Multi-scene fraud phone identification system based on transformer

Abstract

This invention discloses a multi-scenario fraud call identification system based on Transformer, belonging to the field of fraud call identification technology. It solves the technical problem that existing technologies, which filter abnormal numbers through multi-dimensional analysis of caller ID, fail to consider that fraudsters can easily evade blacklists by frequently changing or using fake numbers, potentially leading to an inability to cover all potential fraud sources. This invention preprocesses call recording samples; removes stop words from the call text; converts the call text into text vectors using word embedding technology; constructs a Transformer model using sequence modeling capabilities and a self-attention mechanism to obtain the fraud probability corresponding to the call recording sample; trains the Transformer model using text vectors and fraud probabilities; and dynamically adjusts the hyperparameters and number of layers in the model, introducing domain-specific word embeddings to adapt the model to multiple types of fraud scenarios.

Description

Technical Field

[0001] This invention belongs to the field of fraudulent call identification and involves deep recognition technology, specifically a multi-scenario fraudulent call identification system based on Transformer. Background Technology

[0002] In the era of big data, the amount of data generated is increasing exponentially, and the methods of telecommunications fraud are constantly evolving. Fraudsters use various methods to commit fraud, such as impersonating bank staff, sending fake prize winning messages, and offering fake investment opportunities. The ever-changing methods of these frauds make it difficult for identification work in a single scenario to be comprehensive. Therefore, it is necessary to design a multi-scenario fraud call identification system to deal with various fraud scenarios. A multi-scenario fraud call identification system can process massive amounts of data from multiple scenarios simultaneously and perform rapid analysis and identification through pattern recognition and other technologies. A multi-scenario fraud call identification system can not only help users identify fraudulent calls, but also improve users' awareness of prevention through publicity and education, which helps reduce the number of users who are deceived due to insufficient identification ability. Moreover, fraudsters may be located in different regions or even different countries, colluding with each other to carry out fraud. A multi-scenario fraud call identification system can cover a wider range of regions and countries, providing strong support for combating cross-regional and transnational fraud.

[0003] Existing technology (patent application number CN109698884B) discloses a method for identifying fraudulent calls, including: obtaining local real-time call detail records (CDRs) from an operator; encrypting the phone numbers in the local real-time CDRs to obtain corresponding de-identified CDRs; extracting the caller ID, called number, call duration, and call start time from the de-identified CDRs; analyzing the caller ID to filter out abnormal numbers; analyzing the called number corresponding to the abnormal number to determine whether the user of the called number is a suspected victim; analyzing the specific call process of the abnormal number and the called number corresponding to the abnormal number to determine whether the abnormal number meets the fraud criteria; if it does, decrypting the abnormal number and the called number to determine that the decrypted abnormal number and the called number are respectively the fraudulent number and the victim's number; and outputting the fraudulent number and the victim's number, thereby achieving the identification of fraudulent calls. Existing technology filters out abnormal numbers through multi-dimensional analysis of the caller ID, but it does not consider that fraudsters can easily bypass blacklists by frequently changing numbers or using fake numbers, which may lead to the inability to cover all potential fraud sources.

[0004] This invention provides a multi-scenario fraud call identification system based on Transformer to solve the above-mentioned technical problems. Summary of the Invention

[0005] This invention aims to at least solve one of the technical problems existing in the prior art; to this end, this invention proposes a multi-scenario fraud call identification system based on Transformer, which is used to solve the technical problem that the existing technology filters out abnormal numbers by performing multi-dimensional analysis of the caller ID, but does not take into account that fraudsters can easily avoid the blacklist by frequently changing numbers or using fake numbers, which may lead to the inability to cover all potential fraud sources.

[0006] To achieve the above objectives, a first aspect of the present invention provides a Transformer-based multi-scenario fraud call identification system, comprising:

[0007] Data acquisition and processing module: Retrieves call recording samples from the database, preprocesses the call recording samples, converts the call recording samples into call text; removes stop words from the call text; and uses word embedding technology to convert the call text into text vectors; the call recording samples include: known fraudulent calls and normal call recordings;

[0008] Model building module: Utilizes sequence modeling capabilities and self-attention mechanism to build a Transformer model, obtains the fraud probability corresponding to the call recording samples, and trains the Transformer model using text vectors and fraud probabilities;

[0009] Scene Adaptive Optimization Module: By dynamically adjusting the hyperparameters in the Transformer and introducing domain-specific word embeddings, the model can adapt to different types of fraud scenarios. The hyperparameters include: word embedding dimension, number of heads in multi-head attention, feedforward network dimension, attention dropout ratio, learning rate, and number of encoder layers. Fraud scenarios include: impersonating officials, financial lending, and e-commerce customer service.

[0010] Feedback mechanism module: Adjusts model parameters based on actual recognition results; periodically evaluates model performance and updates the model.

[0011] Preferably, the step of converting the call recording sample into call text includes:

[0012] Retrieve call recording samples and use audio editing software to remove noise and jargon from the call recording samples; divide the call recording samples into several sound segments and extract key features from several sound segments; feature extraction includes operations such as amplitude normalization, frequency response correction, framing, windowing, and start and end point detection of speech.

[0013] The speech parameters of several sound segments are analyzed using an acoustic model; several word sequences and their corresponding probabilities are calculated using a language model; and the final call text is determined based on the speech parameters and the probabilities of the word sequences.

[0014] This invention reduces the impact of noise through preprocessing. By converting call recording samples into call text, the converted text content is more easily retrieved by search engines, increasing information accessibility. Moreover, the text format information is easier to understand and process, and can be reviewed and searched at any time, making information organization and management more convenient.

[0015] Preferably, the removal of stop words from the call text includes:

[0016] Retrieve the call text and use the jieba word segmentation method to divide the call text into several words; obtain the stop word list from the jieba database, match the several words with the stop word list, and remove words that match the stop word list.

[0017] This invention uses the jieba word segmentation method to remove stop words, which can filter out some interjections that have no practical meaning to the model before model training, thereby improving the model's prediction efficiency. By segmenting long sentences into meaningful words, it helps to understand the content of the conversation more accurately.

[0018] Stop word removal can reduce interference from irrelevant words, allowing the analysis to focus more on key information.

[0019] Preferably, the step of converting the call text into a text vector using word embedding technology includes:

[0020] The call text is retrieved, and word embedding technology is used to map the words in the call text to a high-dimensional continuous vector space. The Word2Vec algorithm is then used to transform the words in the high-dimensional continuous vector space into text vectors. The Word2Vec algorithm includes the CBOW model and the Skip-gram model.

[0021] It should be noted that the CBOW model uses context words as input to predict the current word; the Skip-gram model uses the current word to predict context words.

[0022] This invention effectively captures semantic relationships between words through word embedding technology, bringing similar words closer together in the vector space, which helps identify semantic relationships in subsequent natural language processing tasks. Because word embedding technology can capture semantic relationships between words, it provides the model with richer and more effective feature representations.

[0023] Preferably, the construction of the Transformer model using sequence modeling capabilities and self-attention mechanisms includes:

[0024] The system includes an input unit, a position encoder unit, a coding layer unit, and an output unit.

[0025] The input unit is used to input text vectors; cosine position coding is introduced in the position encoder unit to handle differences in sequence order;

[0026] The coding layer unit includes a Multi-Head Attention layer, a Residual Connections layer, a LayerNormalization layer, and a Feed Forward layer;

[0027] The Multi-Head Attention layer enhances the model's attention capabilities by executing multiple self-attention mechanisms; the Residual Connections layer and Layer Normalization together promote stable training and performance improvement of the model; the Feed Forward layer is used to further extract features and increase the model's expressive power.

[0028] The output unit consists of a fully connected linear layer and a softmax function; the fully connected linear layer is used to map the high-level features of the Transformer to the output space; the softmax function is used to predict the category of telecom fraud.

[0029] This invention utilizes the self-attention mechanism in the Transformer model to directly compute the dependency relationship between any two positions, thereby easily capturing long-distance dependencies. This plays a crucial role in processing long texts or tasks that require understanding the relationships between distant words in a sentence.

[0030] Preferably, the introduction of cosine position encoding in the position encoder unit includes:

[0031] Create a random text embedding tensor, construct a positional encoding using cosine positional encoding; copy the positional encoding to match the batch dimension of the text embedding and add them together to obtain the final input containing positional information and retain the positional information of the elements in the sequence.

[0032] It should be noted that all sequence data in this invention have the same length. In practical applications, the pad_sequence function needs to be used to ensure that all sequences have the same length.

[0033] This invention uses positional encoding to preserve the order information of elements in a sequence when processing sequence data in the Transformer architecture. Since the self-attention mechanism itself is position-independent, positional encoding can be added to the input to help the model understand the position of words in a sentence.

[0034] Preferably, the Multi-Head Attention layer enhances the model's attention capabilities by executing multiple self-attention mechanisms, including:

[0035] The embedding matrix of the input sequence is divided into multiple parts, each corresponding to a "head". Self-attention computation is performed on each head. The self-attention computation includes: calculating the query (Q), key (K), and value (V) matrices, and calculating the attention weight of each position in the input sequence to other positions.

[0036] Values ​​in the weighted input sequence are obtained based on the weights, and a contextual representation for each position is generated. Each head learns different dependencies in the input sequence, which correspond to different semantic or syntactic structures. After concatenating the outputs of all heads, a linear transformation is used to reduce the dimensionality to match the dimensionality of the original embedding.

[0037] This invention accelerates the computation process by allowing multiple heads to be processed in parallel since each head is computed independently. By concatenating the outputs of all heads and then reducing the dimensionality through a linear transformation to match the original embedding dimension, this step helps to fuse the features learned by different heads together to generate the final output representation.

[0038] Preferably, the Softmax function is used to predict the category of telecommunications fraud, including:

[0039] Given an n-dimensional vector The vector obtained after performing the Softmax transformation The calculation formula is as follows:

[0040] Through formula The probability value of each element in the vector after the Softmax transformation is calculated; where all probability values ​​are summed to equal 1.

[0041] This invention uses the Softmax function to convert the output of a neural network into a probability distribution, ensuring that the sum of the predicted probabilities of all categories is 1. This means that for input telecom fraud data, the Softmax function can provide an intuitive probabilistic solution, that is, the probability that the model believes the data belongs to each fraud category.

[0042] Preferably, adjusting the model parameters based on the actual recognition results includes:

[0043] The fraud probability values ​​are analyzed based on the fraud probability corresponding to the call recording samples to identify the error types and causes; the model parameters are then adjusted based on the error types and causes, including the learning rate, batch size, and number of iterations.

[0044] Obtain the accuracy of the model in recognizing several types of data, and determine whether the accuracy of several types of data is greater than the accuracy threshold. If yes, add the corresponding type of data to the training set; otherwise, retrieve the corresponding type of data for manual annotation, add the annotated data to the training set, and retrain the model based on the training set.

[0045] It should be noted that, in addition to using the model to identify incorrect samples for backflow learning, new labeled data can be collected periodically to expand the training set; the new labeled data should cover as many possible situations and scenarios as possible to ensure the model's generalization ability; the new labeled data should be added to the training set and the model should be retrained.

[0046] This invention achieves continuous model optimization by adjusting model parameters based on feedback from actual identification results. Simultaneously, it supports back-learning from manually labeled samples, further enhancing the model's ability to identify emerging fraudulent methods.

[0047] Preferably, the periodic performance evaluation and model update of the model includes:

[0048] Obtain several performance metrics of the model within a set time interval; compare these performance metrics with the previous performance metrics; the performance metrics include accuracy, recall, and F1 score.

[0049] Determine if several performance metrics are lower than the previous performance metrics; if yes, update the model; otherwise, continue to analyze the model's performance; evaluate the performance of the updated model, and when the model's performance metrics are greater than the metric threshold, the update is complete; otherwise, continue to update the model.

[0050] It should be noted that the time interval depends on the actual situation; measures to update the model include adjusting hyperparameters, modifying the network structure, and using new training data.

[0051] This invention helps to identify potential problems early, such as data quality degradation, model overfitting, or underfitting, by regularly evaluating model performance. Through regular evaluation, these problems can be identified in a timely manner and corresponding measures can be taken to avoid more serious consequences.

[0052] Compared with the prior art, the beneficial effects of the present invention are:

[0053] 1. This invention applies the revolutionary Transformer model in the field of natural language processing to the telephone fraud identification scenario. It utilizes its powerful sequence processing capabilities and self-attention mechanism to capture long-distance dependencies and effectively analyze the context and semantic information in telephone calls. It can not only identify common fraud patterns, but also quickly adapt to the ever-evolving multi-scenario fraud types by adjusting parameters and introducing domain-specific word embeddings.

[0054] 2. A multi-scenario fraud call identification system based on Transformer, including a data acquisition and preprocessing module, a Transformer model construction module, a multi-head self-attention and position encoding mechanism, a scenario adaptive optimization module, and a feedback mechanism that supports manual feedback. Its feature is that it uses the Transformer architecture to perform deep learning analysis on the content of telephone calls to achieve efficient identification of fraud calls. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic diagram of the modules included in the present invention;

[0057] Figure 2 This is a schematic diagram of the fraudulent call identification process of the present invention;

[0058] Figure 3 This is a schematic diagram illustrating the first specific step in the fraud call identification process of the present invention.

[0059] Figure 4 This is a schematic diagram of the second specific step in the fraud call identification process of the present invention. Detailed Implementation

[0060] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] Please see Figures 1-2 The first aspect of this invention provides a multi-scenario fraud call identification system based on Transformer, comprising:

[0062] Data acquisition and processing module: Retrieves call recording samples from the database, preprocesses the call recording samples, converts the call recording samples into call text; removes stop words from the call text; and uses word embedding technology to convert the call text into text vectors; the call recording samples include: known fraudulent calls and normal call recordings;

[0063] Model building module: Utilizes sequence modeling capabilities and self-attention mechanism to build a Transformer model, obtains the fraud probability corresponding to the call recording samples, and trains the Transformer model using text vectors and fraud probabilities;

[0064] Scene Adaptive Optimization Module: By dynamically adjusting the hyperparameters and number of layers in the Transformer, and introducing domain-specific word embeddings, the model can adapt to different types of fraud scenarios. The hyperparameters include: word embedding dimension, number of heads in multi-head attention, feedforward network dimension, attention dropout ratio, learning rate, and number of encoder layers. Fraud scenarios include: impersonating officials, financial lending, and e-commerce customer service.

[0065] Feedback mechanism module: Adjusts model parameters based on actual recognition results; periodically evaluates model performance and updates the model.

[0066] For example, in the scenario adaptation optimization module, specific domain-specific word embeddings are introduced by dynamically adjusting the hyperparameters and number of layers in the Transformer. For example, impersonating an official requires collecting words and phrases related to public service institutions; impersonating financial credit requires collecting words and phrases related to banks, loans, credit cards, and investment management; and impersonating e-commerce customer service requires collecting words and phrases related to e-commerce platforms, shopping, payment, and logistics. This allows the model to adapt to different types of fraud scenarios. The hyperparameters include: word embedding dimension, number of heads in multi-head attention, feedforward network dimension, attention dropout ratio, learning rate, and number of encoder layers.

[0067] Please see Figure 3The specific steps for identifying fraudulent phone calls are as follows: First, retrieve call recording samples and use audio editing software to remove background noise and interference. Divide the call recording samples into several sound segments and extract key features from each segment. Feature extraction includes amplitude normalization, frequency response correction, framing, windowing, and start / end point detection. Analyze the speech parameters of the sound segments using an acoustic model. Calculate several word sequences and their corresponding probabilities using a language model. Determine the final call text based on the speech parameters and word sequence probabilities. Next, retrieve the call text and use the jieba word segmentation method to divide it into several words. Obtain a stop word list from the jieba database, match the selected words against the stop word list, and remove any words that do not match the stop words. The process involves: retrieving the call text and mapping the words in the call text to a high-dimensional continuous vector space using word embedding technology; then converting the words in the high-dimensional continuous vector space into text vectors using the Word2Vec algorithm; the Word2Vec algorithm includes the CBOW model and the Skip-gram model; input unit, position encoder unit, encoding layer unit, and output unit; the input unit is used to input the text vectors; acquiring the sequence database, creating a random text embedding tensor, and constructing position codes using cosine position encoding; copying the position codes to match the batch dimension of the text embeddings and adding them to obtain the final input containing position information while retaining the position information of elements in the sequence; the encoding layer unit includes a Multi-Head... The model consists of an Attention layer, a Residual Connections layer, a Layer Normalization layer, and a Feed Forward layer. The process involves acquiring a sequence database, segmenting the embeddings of the input sequence into multiple parts, each corresponding to a "head." The embedding matrix is ​​expanded to a higher dimension. Self-attention computation is performed on each head, including calculating the query (Q), key (K), and value (V) matrices, and calculating the attention weights of each position in the input sequence to other positions. Values ​​in the weighted input sequence are then obtained to generate a contextual representation for each position. Each head learns different dependencies in the input sequence, corresponding to different semantic or syntactic structures. After concatenating the outputs of all heads, a linear transformation is applied to reduce the dimensionality to match the original embedding dimension. The Residual Connections layer and Layer Normalization layer jointly promote stable training and performance improvement. The Feed Forward layer is used to further extract features and increase the model's expressive power.

[0068] The output unit consists of a fully connected linear layer and a softmax function; the fully connected linear layer is used to map the high-level features of the Transformer to the output space of the specific task; given an n-dimensional vector The vector obtained after performing the Softmax transformation The calculation formula is as follows: (Using the formula...) The probability value of each element in the vector after the Softmax transformation is calculated; where all probability values ​​are summed to equal 1.

[0069] For example, given a two-minute phone call recording, use audio editing software to remove background noise and other noise from the recording, divide the recording into 12 audio segments, and extract key features from each segment.

[0070] Please see Figure 4 The second specific step in fraud call identification is to obtain the actual fraud probability corresponding to the call recording text; analyze the fraud probability based on the actual fraud probability to identify the error type and cause; and adjust the model parameters according to the error type and cause. The model parameters include the learning rate, batch size, and number of iterations.

[0071] The model obtains the accuracy of identifying several types of data and determines whether the accuracy of several types of data is greater than the accuracy threshold. If yes, the corresponding type of data is added to the training set; if no, the corresponding type of data is retrieved for manual annotation and the annotated data is added to the training set. The model is retrained based on the training set. Several performance indicators of the model within a set time interval are obtained. The performance indicators are compared with the previous performance indicators. The performance indicators include accuracy, recall, and F1 score.

[0072] Determine if several performance metrics are lower than the previous performance metrics; if yes, update the model; otherwise, continue to analyze the model's performance; evaluate the performance of the updated model, and when the model's performance metrics are greater than the metric threshold, the update is complete; otherwise, continue to update the model.

[0073] For example, if the model's accuracy in recognizing several types of data is 95%, and the accuracy threshold is 90%, then the corresponding data types are added to the training set if the accuracy of several types of data is greater than the accuracy threshold. If the model's accuracy, recall, and F1 score are 93%, 90%, and 0.5 within one hour, and the model's accuracy, recall, and F1 score were 95%, 92%, and 0.6 in the previous hour, and all performance metrics are lower than the previous performance metrics, then the model is updated. If the metric thresholds are obtained, where the accuracy is 92%, the recall is 95%, and the F1 score is 0.8, the performance of the updated model is evaluated. If the model's accuracy is greater than the accuracy threshold, but the model's recall and F1 score are less than the metric thresholds, then the model continues to be updated.

[0074] Some of the data in the above formula are calculated by removing dimensions and taking their numerical values. The formula is the closest to the real situation obtained by software simulation of a large amount of collected data. The preset parameters and preset thresholds in the formula are set by those skilled in the art according to the actual situation or obtained through simulation of a large amount of data.

[0075] The working principle of this invention is as follows: This invention obtains call recording samples from a database, preprocesses them to convert them into call text, removes stop words from the call text, converts the call text into text vectors using word embedding technology, constructs a Transformer model using sequence modeling capabilities and a self-attention mechanism, obtains the fraud probability corresponding to the call recording samples, and trains the Transformer model using the text vectors and fraud probabilities. By dynamically adjusting the hyperparameters and number of layers in the Transformer and introducing domain-specific word embeddings, the model adapts to different types of fraud scenarios. Model parameters are adjusted based on actual recognition results, and the model is periodically evaluated and updated.

[0076] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A multi-scenario fraud call identification system based on Transformer, characterized in that, include: The module includes data acquisition and processing, model building, scene adaptation optimization, and feedback mechanism. Data acquisition and processing module: retrieves call recording samples from the database, preprocesses the call recording samples, converts the call recording samples into call text, and removes stop words from the call text; The call text was converted into text vectors using word embedding technology; the call recording samples included: known fraudulent calls and normal call recordings. Model building module: Utilizes sequence modeling capabilities and self-attention mechanisms to build a Transformer model, including: input unit, position encoder unit, encoding layer unit, and output unit; obtains the fraud probability corresponding to the call recording sample, and trains the Transformer model using text vectors and fraud probabilities; Scene Adaptive Optimization Module: By dynamically adjusting the hyperparameters in the Transformer and introducing domain-specific word embeddings, the model can adapt to different types of fraud scenarios. The hyperparameters include: word embedding dimension, number of heads in multi-head attention, feedforward network dimension, attention dropout ratio, learning rate, and number of encoder layers. Fraud scenarios include: impersonating officials, financial lending, and e-commerce customer service. Feedback mechanism module: Adjusts model parameters based on actual recognition results; periodically evaluates model performance and updates the model accordingly; The construction of the Transformer model using sequence modeling capabilities and self-attention mechanisms includes: The input unit is used to input text vectors; cosine position coding is introduced in the position encoder unit to handle differences in sequence order; The coding layer unit includes a Multi-Head Attention layer, a Residual Connections layer, a LayerNormalization layer, and a Feed Forward layer; The Multi-Head Attention layer enhances the model's attention capabilities by implementing multiple self-attention mechanisms; the Residual Connections layer and Layer Normalization jointly promote stable training and performance improvement; the Feed Forward layer is used to further extract features and increase the model's expressive power; the output unit includes a fully connected Linear layer and a Softmax function; the fully connected Linear layer is used to map the high-level features of the Transformer to the output space; the Softmax function is used to predict the category of telecom fraud. The Multi-Head Attention layer enhances the model's attention capabilities by executing multiple self-attention mechanisms, including: The embedding matrix of the input sequence is divided into multiple parts, each corresponding to a "head". Self-attention computation is performed on each head. The self-attention computation includes: calculating the query (Q), key (K), and value (V) matrices, and calculating the attention weight of each position in the input sequence to other positions. Values ​​in the weighted input sequence are obtained based on the weights, and a contextual representation for each position is generated. Each head learns different dependencies in the input sequence, which correspond to different semantic or syntactic structures. After concatenating the outputs of all heads, a linear transformation is used to reduce the dimensionality to match the dimensionality of the original embedding.

2. The Transformer-based multi-scenario fraud call identification system according to claim 1, characterized in that, The process of converting call recording samples into call text includes: Retrieve call recording samples and use audio editing software to remove noise and jargon from the call recording samples; divide the call recording samples into several sound segments and extract key features from several sound segments; feature extraction includes operations such as amplitude normalization, frequency response correction, framing, windowing, and start and end point detection of speech. The speech parameters of several sound segments are analyzed using an acoustic model; several word sequences and their corresponding probabilities are calculated using a language model; and the final call text is determined based on the speech parameters and the probabilities of the word sequences.

3. The Transformer-based multi-scenario fraud call identification system according to claim 1, characterized in that, The removal of stop words from the call text includes: Retrieve the call text and use the jieba word segmentation method to divide the call text into several words; obtain the stop word list from the jieba database, match the several words with the stop word list, and remove words that match the stop word list.

4. The Transformer-based multi-scenario fraud call identification system according to claim 1, characterized in that, The method of converting call text into text vectors using word embedding technology includes: The call text is retrieved, and word embedding technology is used to map the words in the call text to a high-dimensional continuous vector space. The Word2Vec algorithm is then used to transform the words in the high-dimensional continuous vector space into text vectors. The Word2Vec algorithm includes the CBOW model and the Skip-gram model.

5. The Transformer-based multi-scenario fraud call identification system according to claim 1, characterized in that, The introduction of cosine position encoding in the position encoder unit includes: Create a random text embedding tensor, construct a positional encoding using cosine positional encoding; copy the positional encoding to match the batch dimension of the text embedding and add them together to obtain the final input containing positional information and retain the positional information of the elements in the sequence.

6. The Transformer-based multi-scenario fraud call identification system according to claim 1, characterized in that, The Softmax function is used to predict the categories of telecommunications fraud, including: Given an n-dimensional vector After performing the Softmax transformation, the vector is obtained. The calculation formula is as follows: ; The probability value of each element in the vector after the Softmax transformation is calculated using the formula; where the sum of all probability values ​​equals 1.

7. The Transformer-based multi-scenario fraud call identification system according to claim 6, characterized in that, The step of adjusting model parameters based on actual recognition results includes: The fraud probability values ​​are analyzed based on the fraud probability corresponding to the call recording samples to identify the error types and causes; the model parameters are then adjusted based on the error types and causes, including the learning rate, batch size, and number of iterations. Obtain the accuracy of the model in recognizing several types of data, and determine whether the accuracy of several types of data is greater than the accuracy threshold. If yes, add the corresponding type of data to the training set; otherwise, retrieve the corresponding type of data for manual annotation, add the annotated data to the training set, and retrain the model based on the training set.

8. The Transformer-based multi-scenario fraud call identification system according to claim 1, characterized in that, The periodic performance evaluation and model update of the model includes: Obtain several performance metrics of the model within a set time interval; compare these performance metrics with the previous performance metrics; the performance metrics include accuracy, recall, and F1 score. Determine if several performance metrics are lower than the previous performance metrics; if yes, update the model; otherwise, continue to analyze the model's performance; evaluate the performance of the updated model, and when the model's performance metrics are greater than the metric threshold, the update is complete; otherwise, continue to update the model.

Images