Transform-based multi-scene fraud phone recognition system

Abstract

The invention discloses a Transform-based multi-scene fraud phone recognition system, relates to the technical field of fraud phone recognition, and solves the problems that in the prior art, abnormal numbers are screened out by performing multi-dimensional analysis on calling numbers, a fraud can easily avoid a blacklist without considering that the fraud frequently changes the numbers or uses false numbers, and the fraud time is shortened. The technical problem that all potential fraud sources cannot be covered possibly can be solved; the method comprises the steps of preprocessing a call recording sample; stop words in the call text are removed; converting the call text into a text vector by using a word embedding technology; constructing a Transform model by using a sequence modeling capability and a self-attention mechanism, obtaining a fraud probability corresponding to the call recording sample, and training the Transform model by using the text vector and the fraud probability; by dynamically adjusting hyper-parameters and the number of layers in the model, specific field vocabulary embedding is introduced, so that the model adapts to multiple types of fraud scenes.

Description

Technical Field

[0001] The present invention belongs to the field of fraudulent call identification and relates to deep recognition technology, specifically a multi-scenario fraudulent call identification system based on Transformer. Background Art

[0002] In the context of the big data era, the amount of data generated is increasing exponentially, and the methods of telecommunications fraud are endless. Fraudsters will use a variety of means to commit fraud. The constant changes in these fraud methods make it difficult to fully cover the identification work in a single scenario. Therefore, it is necessary to design a multi-scenario fraud call identification system to cope with various fraud scenarios; the multi-scenario fraud call identification system can process massive data in multiple scenarios at the same time, and quickly analyze and identify through technical means such as pattern recognition; the multi-scenario fraud call identification system can not only help users identify fraudulent calls, but also improve users' awareness of prevention through publicity and education, which helps to reduce the situation where users are deceived due to insufficient recognition ability; and fraudsters may be distributed in different regions or even different countries, and collude with each other to commit fraud. The multi-scenario fraud call identification system can cover a wider range of regions and countries, providing strong support for combating cross-regional and cross-border fraud.

[0003] The prior art (invention patent with application number CN109698884B) discloses a method for identifying fraudulent calls, including: obtaining a local real-time call list of an operator, encrypting the telephone number in the local real-time call list, and obtaining a corresponding desensitized call list; extracting the calling number, called number, call duration, and call start time in the desensitized call list; analyzing the calling number to screen out abnormal numbers; analyzing the called number corresponding to the abnormal number to determine whether the user of the called number is a suspected victim; analyzing the specific call process of the abnormal number and the called number corresponding to the abnormal number to determine whether the abnormal number meets the fraud compliance degree, and if so, decrypting the abnormal number and the called number to determine that the decrypted abnormal number and the called number are respectively a fraud number and a victim user number; and outputting the fraud number and the victim user number, thereby realizing the identification of fraudulent calls; the prior art screens out abnormal numbers by performing multi-dimensional analysis on the calling number, but does not take into account that fraudsters can easily avoid blacklists by frequently changing numbers or using false numbers, which may lead to the problem that all potential fraud sources cannot be covered.

[0004] The present invention provides a multi-scenario fraud call identification system based on Transformer to solve the above technical problems. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a multi-scenario fraud call identification system based on Transformer, which is used to solve the technical problem that the prior art screens out abnormal numbers by performing multi-dimensional analysis on the calling number, but does not take into account that fraudsters can easily avoid the blacklist by frequently changing numbers or using fake numbers, which may lead to the failure to cover all potential fraud sources.

[0006] To achieve the above objectives, the first aspect of the present invention provides a multi-scenario fraud call identification system based on Transformer, comprising:

[0007] Data collection and processing module: obtain call recording samples from the database, pre-process the call recording samples, and convert the call recording samples into call text; remove stop words in the call text; use word embedding technology to convert the call text into a text vector; the call recording samples include: known fraudulent calls and normal call recordings;

[0008] Model building module: Use sequence modeling capabilities and self-attention mechanisms to build a Transformer model, obtain the fraud probability corresponding to the call recording sample, and train the Transformer model using text vectors and fraud probability;

[0009] Scenario Adaptive Optimization Module: By dynamically adjusting the hyperparameters and number of layers in Transformer, domain-specific word embedding is introduced to make the model adaptable to different types of fraud scenarios; the hyperparameters include: word embedding dimension, number of heads in multi-head attention, feedforward network dimension, attention dropout ratio, learning rate, and number of encoder layers; fraud scenarios include: impersonating an official, financial credit, and e-commerce customer service;

[0010] Feedback mechanism module: adjust model parameters according to actual recognition result feedback; regularly evaluate model performance and update the model.

[0011] Preferably, converting the call recording sample into call text includes:

[0012] Retrieve call recording samples and use audio editing software to remove noise and hum in the call recording samples; divide the call recording samples into several sound segments and extract key features of the several sound segments; the feature extraction includes the operations of speech amplitude standardization, frequency response correction, framing, windowing, and start and end point detection;

[0013] The speech parameters of several sound clips are analyzed through the acoustic model; several phrase sequences of the sound clips and the corresponding probabilities are calculated through the language model; and the final call text is determined based on the speech parameters and the probability of the phrase sequence.

[0014] The present invention can reduce the impact of noise through preprocessing, and convert call recording samples into call text, so that the converted text content can be more easily retrieved by search engines, thereby increasing the accessibility of information. Moreover, information in text format is easier to understand and process, and can be reviewed and searched at any time, making information organization and management more convenient.

[0015] Preferably, the step of removing stop words from the call text includes:

[0016] Retrieve the call text and divide it into several words using the Jieba word segmentation method; obtain the stop word list from the Jieba database, match several words with the stop word list, and remove the words that are consistent with the stop word list.

[0017] The present invention uses the Jieba word segmentation method to remove stop words, and can filter out some modal particles that have no practical meaning to the model before performing the model training task, thereby improving the model prediction efficiency. By dividing long sentences into meaningful words, it helps to understand the content of the call more accurately.

[0018] Stop word removal can reduce the interference of irrelevant words and make the analysis more focused on key information.

[0019] Preferably, the converting the call text into a text vector by using word embedding technology includes:

[0020] Retrieve the call text, use word embedding technology to map the vocabulary of the call text to a high-dimensional continuous vector space, and use the Word2Vec algorithm to convert the vocabulary in the high-dimensional continuous vector space into a text vector; the Word2Vec algorithm includes: CBOW model and Skip-gram model.

[0021] It should be noted that the CBOW model uses the context word as input to predict the current word; the Skip-gram model uses the current word to predict the context word.

[0022] The present invention can effectively capture the semantic relationship between words through word embedding technology, so that similar words are close to each other in the vector space, which is helpful to identify word meaning relationships in subsequent natural language processing tasks; because word embedding technology can capture the semantic relationship between words, it provides a richer and more effective feature representation for the model.

[0023] Preferably, the construction of the Transformer model using sequence modeling capabilities and self-attention mechanism includes:

[0024] Input unit, position encoder unit, encoding layer unit and output unit;

[0025] The input unit is used to input the text vector; the cosine position encoding is introduced in the position encoder unit to process the difference in sequence order;

[0026] The encoding layer units include Multi-Head Attention layer, Residual Connections layer, LayerNormalization and Feed Forward layer;

[0027] The Multi-Head Attention layer enhances the model’s attention capability by executing multiple self-attention mechanisms; the Residual Connections layer and Layer Normalization jointly promote stable training and performance improvement of the model; the Feed Forward layer is used to further extract features and increase the model’s expressiveness;

[0028] The output unit includes a linear fully connected Linear layer and a Softmax function; the linear fully connected Linear layer is used to map the high-level features of the Transformer to the output space of a specific task; the Softmax function is used to predict the category of telecommunications fraud.

[0029] Through the self-attention mechanism in the Transformer model, the present invention can directly calculate the dependency between any two positions, thereby easily capturing long-distance dependencies, which plays a vital role in processing long texts or tasks that require understanding the relationship between distant words in a sentence.

[0030] Preferably, the introducing cosine position coding in the position encoder unit comprises:

[0031] Get the sequence database, create a random text embedding tensor, and construct the position encoding through cosine position encoding; copy the position encoding to match the batch dimension of the text embedding and add it to obtain the final input containing the position information and retain the position information of the elements in the sequence.

[0032] It should be noted that the length of all sequence data in the present invention is the same. In practical applications, it is necessary to use the pad_sequence function to ensure that the length of all sequences is the same.

[0033] The present invention uses position encoding to retain the order information of elements in the sequence when processing sequence data in the Transformer architecture. Since the self-attention mechanism itself is position-independent, position encoding can be added to the input to help the model understand the position of words in the sentence.

[0034] Preferably, the Multi-Head Attention layer enhances the attention capability of the model by executing multiple self-attention mechanisms, including:

[0035] Obtain a sequence database, split the embedding of the input sequence into multiple parts, and correspond to a "head"; expand the dimension of the embedding matrix to a high dimension; perform self-attention calculation on each head separately; where the self-attention calculation includes: calculating the query (Q), key (K) and value (V) matrices, and calculating the attention weight of each position in the input sequence to other positions;

[0036] The values ​​in the weighted input sequence are obtained according to the weights to generate a contextual representation for each position; each head learns different dependencies in the input sequence, which correspond to different semantic or grammatical structures; after concatenating the outputs of all heads together, a linear transformation is used to reduce the dimension to match the dimension of the original embedding.

[0037] The present invention can process multiple heads in parallel because each head is calculated independently, thereby accelerating the calculation process. After splicing the outputs of all heads together, the dimension is reduced through a linear transformation to match the dimension of the original embedding. This step helps to fuse the features learned by different heads to generate the final output representation.

[0038] Preferably, the Softmax function is used to predict the category of telecommunications fraud, including:

[0039] Given an n-dimensional vector z=(z1,z2,…,z n ), the vector p obtained after Softmax transformation is (p1, p2, ..., p n ); the calculation formula is as follows:

[0040] By formula The probability value of each element in the vector after the Softmax transformation is calculated; the sum of all probability values ​​is equal to 1.

[0041] The present invention can convert the output of the neural network into a probability distribution through the Softmax function, ensuring that the sum of the predicted probabilities of all categories is 1, which means that for the input telecommunications fraud data, the Softmax function can provide an intuitive probability solution, that is, the model believes that the data belongs to each fraud category.

[0042] Preferably, adjusting the model parameters according to the actual recognition result feedback includes:

[0043] Obtain the actual fraud probability corresponding to the call recording text; analyze the fraud probability based on the actual fraud probability, and identify the error type and cause; adjust the model parameters based on the error type and cause; the model parameters include learning rate, batch size, and number of iterations;

[0044] Obtain the accuracy of the model in identifying several types of data, and determine whether the accuracy of several types of data is greater than the accuracy threshold; if yes, add the corresponding type of data to the training set; if not, retrieve the corresponding type of data for manual labeling, and add the labeled data to the training set; retrain the model based on the training set.

[0045] It should be noted that in addition to using samples that the model has misidentified for reflux learning, new labeled data can also be collected regularly to expand the training set; the new labeled data should cover as many possible situations and scenarios as possible to ensure the generalization ability of the model; the new labeled data should be added to the training set and the model should be retrained.

[0046] The present invention achieves continuous optimization of the model by adjusting the model parameters according to the actual recognition result feedback. At the same time, it supports the reflux learning of manually annotated samples to further enhance the model's ability to identify new fraud methods.

[0047] Preferably, the periodic performance evaluation of the model to update the model includes:

[0048] Obtain several performance indicators of the model within a set time interval; compare several performance indicators with the previous performance indicators; wherein the performance indicators include accuracy, recall rate, and F1 score;

[0049] Determine whether several performance indicators are less than the previous performance indicators; if yes, update the model; if not, continue to analyze the performance of the model; evaluate the performance of the updated model, and when the performance indicator of the model is greater than the indicator threshold, the update is completed; otherwise, continue to update the model.

[0050] It should be noted that the time interval depends on the actual situation; measures to update the model include adjusting hyperparameters, modifying the network structure, and using new training data.

[0051] The present invention helps to discover potential problems early by regularly evaluating model performance, such as deterioration in data quality, model overfitting or underfitting, etc. Through regular evaluation, these problems can be discovered in time and corresponding measures can be taken to avoid more serious impacts.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] 1. The present invention applies the revolutionary Transformer model in the field of natural language processing to the telephone fraud identification scenario, using its powerful sequence processing capabilities and self-attention mechanism to capture long-distance dependencies and effectively analyze the context and semantic information in telephone calls. It can not only identify common fraud patterns, but also quickly adapt to the evolving multi-scenario fraud types by adjusting parameters and introducing vocabulary embeddings in specific fields.

[0054] 2. The Transformer-based multi-scenario fraud call identification system includes a data acquisition and preprocessing module, a Transformer model building module, a multi-head self-attention and position encoding mechanism, a scenario adaptive optimization module, and a feedback mechanism that supports artificial reflow. It is characterized by using the Transformer architecture to perform deep learning analysis on telephone call content to achieve efficient identification of fraudulent calls. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A schematic diagram of the modules included in the present invention;

[0057] Figure 2 A schematic diagram of the fraud call identification process of the present invention;

[0058] Figure 3 This is a schematic diagram of the specific step 1 of fraud call identification of the present invention;

[0059] Figure 4 This is a schematic diagram of specific step 2 of fraud call identification according to the present invention. DETAILED DESCRIPTION

[0060] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] See also Figure 1-Figure 2 The first embodiment of the present invention provides a multi-scenario fraud call identification system based on Transformer, including:

[0062] Data collection and processing module: obtain call recording samples from the database, pre-process the call recording samples, and convert the call recording samples into call text; remove stop words in the call text; use word embedding technology to convert the call text into a text vector; the call recording samples include: known fraudulent calls and normal call recordings;

[0063] Model building module: Use sequence modeling capabilities and self-attention mechanisms to build a Transformer model, obtain the fraud probability corresponding to the call recording sample, and train the Transformer model using text vectors and fraud probability;

[0064] Scenario Adaptive Optimization Module: By dynamically adjusting the hyperparameters and number of layers in Transformer, domain-specific word embedding is introduced to make the model adaptable to different types of fraud scenarios; the hyperparameters include: word embedding dimension, number of heads in multi-head attention, feedforward network dimension, attention dropout ratio, learning rate, and number of encoder layers; fraud scenarios include: impersonating an official, financial credit, and e-commerce customer service;

[0065] Feedback mechanism module: adjust model parameters according to actual recognition result feedback; regularly evaluate model performance and update the model.

[0066] For example, in the scenario adaptive optimization module, domain-specific vocabulary embedding is introduced by dynamically adjusting the hyperparameters and number of layers in the Transformer. Among them, impersonating an official requires collecting words and phrases related to public service agencies; impersonating financial credit requires collecting words and phrases related to banks, loans, credit cards, and investment and financial management; impersonating e-commerce customer service requires collecting words and phrases related to e-commerce platforms, shopping, payment, and logistics; thereby making the model adaptable to different types of fraud scenarios. The hyperparameters include: word embedding dimension, the number of heads in multi-head attention, feedforward network dimension, attention dropout ratio, learning rate, and encoder layer number.

[0067] See also Figure 3, the specific steps for identifying fraudulent calls are as follows: 1. Retrieve the call recording sample, and use audio editing software to remove the noise and noise in the call recording sample; divide the call recording sample into several sound segments, and extract the key features of several sound segments; among them, feature extraction includes the operations of speech amplitude standardization, frequency response correction, framing, windowing, and start and end point detection; analyze the speech parameters of several sound segments through the acoustic model; calculate several phrase sequences and corresponding probabilities of the sound segments through the language model; determine the final call text based on the speech parameters and the probability of the phrase sequence; retrieve the call text, and use the jieba word segmentation method to divide the call text into several words; obtain the stop word list from the jieba database, match several words with the stop word list, and eliminate the words that are related to the stop words. The method comprises the following steps: extracting the words of the call text, mapping the words of the call text to a high-dimensional continuous vector space using word embedding technology, and converting the words in the high-dimensional continuous vector space into text vectors using the Word2Vec algorithm; wherein the Word2Vec algorithm includes: a CBOW model and a Skip-gram model; an input unit, a position encoder unit, a coding layer unit, and an output unit; the input unit is used to input the text vector; obtaining a sequence database, creating a random text embedding tensor, and constructing a position encoding using the cosine position encoding method; copying the position encoding to match the batch dimension of the text embedding and adding them together to obtain the final input containing the position information and retain the position information of the elements in the sequence; the coding layer unit includes a Multi-Head Attention layer, Residual Connections layer, Layer Normalization and Feed Forward layer; obtain the sequence database, and divide the embedding of the input sequence into multiple parts, each corresponding to a "head"; by expanding the dimension of the embedding matrix to a high dimension; perform self-attention calculation on each head separately; among which, the self-attention calculation includes: calculating the query (Q), key (K) and value (V) matrices, and calculating the attention weight of each position in the input sequence to other positions; obtaining the value in the weighted input sequence according to the weight, and generating the context representation of each position; each head learns different dependencies in the input sequence, which correspond to different semantic or grammatical structures; after splicing the outputs of all heads together, a linear transformation is used to reduce the dimension to match the dimension of the original embedding; the Residual Connections layer and Layer Normalization jointly promote the stable training and performance improvement of the model; the Feed Forward layer is used to further extract features and increase the expressiveness of the model;

[0068] The output unit includes a linear fully connected Linear layer and a Softmax function; the linear fully connected Linear layer is used to map the high-level features of the Transformer to the output space of a specific task; given an n-dimensional vector z = (z1, z2, ..., z n ), the vector p obtained after Softmax transformation is (p1, p2, ..., p n ); the calculation formula is as follows: The probability value of each element in the vector after the Softmax transformation is calculated; the sum of all probability values ​​is equal to 1.

[0069] For example, there is a two-minute call recording. Use audio editing software to remove the noise and interference in the call recording, divide the call recording sample into 12 sound segments, divide the call recording sample into several sound segments, and extract the key features of several sound segments.

[0070] See also Figure 4 ,The specific step 2 of fraud call identification is to obtain the actual fraud probability corresponding to the call recording text; analyze the fraud probability according to the actual fraud probability, and identify the error type and cause; adjust the model parameters according to the error type and cause; among which, the model parameters include learning rate, batch size, and number of iterations;

[0071] Obtain the accuracy of the model in identifying several types of data, and determine whether the accuracy of several types of data is greater than the accuracy threshold; if yes, add the corresponding type of data to the training set; if no, retrieve the corresponding type of data for manual labeling, and add the labeled data to the training set; retrain the model based on the training set; obtain several performance indicators of the model within a set time interval; compare several performance indicators with the previous performance indicators; among which, the performance indicators include accuracy, recall rate, and F1 score;

[0072] Determine whether several performance indicators are less than the previous performance indicators; if yes, update the model; if not, continue to analyze the performance of the model; evaluate the performance of the updated model, and when the performance indicator of the model is greater than the indicator threshold, the update is completed; otherwise, continue to update the model.

[0073] For example, the accuracy of the model in identifying several types of data is 95%, and the accuracy threshold is 90%. If the accuracy of several types of data is greater than the accuracy threshold, the corresponding type of data is added to the training set; the accuracy of the model within an hour is 93%, the recall rate is 90%, and the F1 score is 0.5, and the accuracy of the model in the previous hour is 95%, the recall rate is 92%, and the F1 score is 0.6. If several performance indicators are less than the previous performance indicators, the model is updated; the indicator threshold is obtained, among which the accuracy rate is 92%, the recall rate is 95%, and the F1 score is 0.8, and the performance of the updated model is evaluated. When the accuracy of the model is greater than the accuracy threshold, the recall rate and F1 score of the model are less than the indicator threshold, the model is continuously updated.

[0074] Part of the data in the above formula is calculated by removing the dimension and taking its numerical value. The formula is a formula closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0075] The working principle of the present invention is as follows: the present invention obtains call recording samples from a database, pre-processes the call recording samples, and then converts the call recording samples into call text; removes stop words in the call text; uses word embedding technology to convert the call text into a text vector; uses sequence modeling capabilities and self-attention mechanisms to build a Transformer model, obtains the fraud probability corresponding to the call recording samples, and trains the Transformer model using text vectors and fraud probabilities; dynamically adjusts the hyperparameters and number of layers in the Transformer, introduces domain-specific vocabulary embedding, and adapts the model to different types of fraud scenarios; adjusts model parameters based on actual recognition result feedback; and regularly performs performance evaluation on the model to update the model.

[0076] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. Transformer-based multi-scenario fraud call identification system, characterized by: include: Data collection and processing module, model building module, scenario adaptability optimization module and feedback mechanism module; Data collection and processing module: obtain call recording samples from the database, pre-process the call recording samples, and convert the call recording samples into call text; remove stop words in the call text; The word embedding technology is used to convert the call text into a text vector. The call recording samples include: known fraudulent calls and normal call recordings. Model building module: Use sequence modeling capabilities and self-attention mechanisms to build a Transformer model, obtain the fraud probability corresponding to the call recording sample, and train the Transformer model using text vectors and fraud probability; Scenario Adaptive Optimization Module: By dynamically adjusting the hyperparameters and number of layers in Transformer, domain-specific word embedding is introduced to make the model adaptable to different types of fraud scenarios; the hyperparameters include: word embedding dimension, number of heads in multi-head attention, feedforward network dimension, attention dropout ratio, learning rate, and number of encoder layers; fraud scenarios include: impersonating an official, financial credit, and e-commerce customer service; Feedback mechanism module: adjust model parameters according to actual recognition result feedback; regularly evaluate model performance and update the model.

2. The Transformer-based multi-scenario fraud call identification system according to claim 1 is characterized in that: The converting the call recording sample into call text includes: Retrieve call recording samples and use audio editing software to remove noise and hum in the call recording samples; divide the call recording samples into several sound segments and extract key features of the several sound segments; the feature extraction includes the operations of speech amplitude standardization, frequency response correction, framing, windowing, and start and end point detection; The speech parameters of several sound clips are analyzed through the acoustic model; several phrase sequences of the sound clips and the corresponding probabilities are calculated through the language model; and the final call text is determined based on the speech parameters and the probability of the phrase sequence.

3. The Transformer-based multi-scenario fraud call identification system according to claim 1 is characterized in that: The step of removing stop words from the call text includes: Retrieve the call text and divide it into several words using the Jieba word segmentation method; obtain the stop word list from the Jieba database, match several words with the stop word list, and remove the words that are consistent with the stop word list.

4. The Transformer-based multi-scenario fraud call identification system according to claim 1 is characterized in that: The method of converting the call text into a text vector by using word embedding technology includes: Retrieve the call text, use word embedding technology to map the vocabulary of the call text to a high-dimensional continuous vector space, and use the Word2Vec algorithm to convert the vocabulary in the high-dimensional continuous vector space into a text vector; the Word2Vec algorithm includes: CBOW model and Skip-gram model.

5. The Transformer-based multi-scenario fraud call identification system according to claim 1 is characterized in that: The Transformer model is constructed by using sequence modeling capabilities and self-attention mechanism, including: Input unit, position encoder unit, encoding layer unit and output unit; The input unit is used to input the text vector; the cosine position encoding is introduced in the position encoder unit to process the difference in sequence order; The encoding layer units include Multi-Head Attention layer, ResidualConnections layer, LayerNormalization and Feed Forward layer; The Multi-Head Attention layer enhances the model’s attention capability by executing multiple self-attention mechanisms; the Residual Connections layer and Layer Normalization jointly promote stable training and performance improvement of the model; the Feed Forward layer is used to further extract features and increase the model’s expressiveness; The output unit includes a linear fully connected Linear layer and a Softmax function; the linear fully connected Linear layer is used to map the high-level features of the Transformer to the output space of a specific task; the Softmax function is used to predict the category of telecommunications fraud.

6. The Transformer-based multi-scenario fraud call identification system according to claim 5 is characterized in that: The method of introducing cosine position coding into the position encoder unit comprises: Get the sequence database, create a random text embedding tensor, and construct the position encoding through cosine position encoding; copy the position encoding to match the batch dimension of the text embedding and add it to obtain the final input containing the position information and preserve the position information of the elements in the sequence.

7. The Transformer-based multi-scenario fraud call identification system according to claim 5 is characterized in that: The Multi-Head Attention layer enhances the model’s attention capabilities by performing multiple self-attention mechanisms, including: Obtain a sequence database, split the embedding of the input sequence into multiple parts, and correspond to a "head"; expand the dimension of the embedding matrix to a high dimension; perform self-attention calculation on each head separately; where the self-attention calculation includes: calculating the query (Q), key (K) and value (V) matrices, and calculating the attention weight of each position in the input sequence to other positions; The values ​​in the weighted input sequence are obtained according to the weights to generate a contextual representation for each position; each head learns different dependencies in the input sequence, which correspond to different semantic or grammatical structures; after concatenating the outputs of all heads together, a linear transformation is used to reduce the dimension to match the dimension of the original embedding.

8. The Transformer-based multi-scenario fraud call identification system according to claim 5 is characterized in that: The Softmax function is used to predict the category of telecommunications fraud, including: Given an n-dimensional vector, the vector obtained after Softmax transformation; the calculation formula is as follows: The probability value of each element in the vector after Softmax transformation is calculated by the formula; where the sum of all probability values ​​is equal to 1.

9. The Transformer-based multi-scenario fraud call identification system according to claim 1 is characterized in that: The adjusting of model parameters according to actual recognition result feedback includes: Obtain the actual fraud probability corresponding to the call recording text; analyze the fraud probability based on the actual fraud probability, and identify the error type and cause; adjust the model parameters based on the error type and cause; the model parameters include learning rate, batch size, and number of iterations; Obtain the accuracy of the model in identifying several types of data, and determine whether the accuracy of several types of data is greater than the accuracy threshold; if yes, add the corresponding type of data to the training set; if not, retrieve the corresponding type of data for manual labeling, and add the labeled data to the training set; retrain the model based on the training set.

10. The Transformer-based multi-scenario fraud call identification system according to claim 1 is characterized in that: The method of regularly evaluating the performance of the model and updating the model includes: Obtain several performance indicators of the model within a set time interval; compare several performance indicators with the previous performance indicators; wherein the performance indicators include accuracy, recall rate, and F1 score; Determine whether several performance indicators are less than the previous performance indicators; if yes, update the model; if not, continue to analyze the performance of the model; evaluate the performance of the updated model, and when the performance indicator of the model is greater than the indicator threshold, the update is completed; otherwise, continue to update the model.

Images