Fraudulent call identification method, device, electronic device and storage medium

Abstract

The present application relates to a method, device, electronic device and storage medium for identifying fraudulent calls. The method includes: obtaining real-time voice data returned by a user device, processing the real-time voice data using a trained voice anti-fraud model, extracting real-time voice features, and determining a real-time anti-fraud template corresponding to the real-time voice features, inputting the real-time voice features and the real-time anti-fraud template into the trained voice anti-fraud model, and outputting the current recognition result of the fraudulent call. The solution provided by the present application can effectively identify and prevent fraudulent calls, and at the same time determine the real-time anti-fraud template associated with the real-time voice features, significantly improve the recognition accuracy of fraudulent calls, and effectively and flexibly deal with complex and changeable fraud methods.

Description

Technical Field

[0001] The present application relates to the field of communication data technology, and in particular to a method, device, electronic device and storage medium for identifying fraudulent calls. Background Art

[0002] With the development and popularization of information technology, telecommunications fraud methods have become more covert and complex, and their harmfulness has continued to increase, posing a great threat to the social integrity system, public order, and personal property safety. In recent years, all sectors of society have paid close attention to the problem of telephone fraud and have taken a series of measures to combat telephone fraud. For example, measures such as establishing an anti-fraud platform, improving laws and regulations, and increasing publicity and education have achieved positive results to a certain extent.

[0003] However, the forms of telephone fraud are constantly evolving, and traditional anti-fraud methods and technical means have gradually shown certain limitations and are unable to cope with increasingly complex fraud techniques.

[0004] To this end, the anti-fraud technology currently used mainly relies on traditional deep learning algorithms, which analyze and identify the voice and text of telephone fraud. However, this type of algorithm has obvious deficiencies in semantic understanding capabilities, low recognition accuracy, and is prone to misjudgment, which seriously affects the user experience. Summary of the invention

[0005] The present application provides a method, device, electronic device and storage medium for identifying fraudulent calls to solve or partially solve the problem of how to improve the accuracy of identifying fraudulent calls.

[0006] The first aspect of the present application provides a method for identifying fraudulent calls, comprising: Get real-time voice data returned by the user's device; Using a trained voice anti-fraud model to extract features from the real-time voice data, obtain real-time voice features, and determine a real-time anti-fraud template corresponding to the real-time voice features; The real-time voice features and the real-time anti-fraud template are input into the trained voice anti-fraud large model, and the current recognition result of the fraudulent call is output.

[0007] In one example, the method of extracting features from the real-time voice data using a trained voice anti-fraud model to obtain real-time voice features and determining a real-time anti-fraud template corresponding to the real-time voice features includes: Extracting the real-time voice features from the real-time voice data using a trained voice anti-fraud model; Determine a preset anti-fraud template associated with the real-time voice feature, and use the preset anti-fraud template as the real-time anti-fraud template; or, The preset anti-fraud template is updated using the real-time voice feature to generate a real-time anti-fraud template corresponding to the real-time voice feature.

[0008] In one example, the training method of the voice anti-fraud model is as follows: Obtain anti-fraud training data set and anti-fraud test data set; Construct the initial speech encoder, initial alignment module and initial speech anti-fraud model; The initial speech encoder is trained using the anti-fraud training data set until the initial speech encoder meets a first set condition, thereby obtaining a target speech encoder; Fixing the parameters of the target speech encoder and the preset text encoder, and using the anti-fraud training data set to train the initial alignment module until the initial alignment module meets the second set condition, thereby obtaining a target alignment module; The target speech encoder, the preset text encoder, the target alignment module and the initial speech anti-fraud model constitute a speech anti-fraud model to be fine-tuned; The anti-fraud training data set and the anti-fraud test data set are used to adjust the speech anti-fraud large model to be fine-tuned to obtain the trained speech anti-fraud large model.

[0009] In one example, the anti-fraud training data set includes original speech data, and the initial speech encoder includes a speech feature compression module and a speech feature extraction module; The speech feature compression module includes a convolution layer and a fully connected layer, wherein the convolution layer is used to extract features and reduce the dimension of the original speech data, and the fully connected layer is used to map the original speech data to a preset dimension and output compressed speech features; The speech feature extraction module includes a multi-head attention network and a feedforward neural network. The multi-head attention network is used to extract speech information at different times or frequencies from the compressed speech features, and the feedforward neural network is used to extract key speech features from the speech information at different times or frequencies.

[0010] In one example, the first set condition includes a first set threshold, and the using the anti-fraud training data set to train the initial speech encoder until the initial speech encoder meets the first set condition to obtain a target speech encoder includes: Using the anti-fraud training data set to train the initial speech encoder to obtain a speech feature vector; The first loss function value corresponding to the speech feature vector is calculated by a first preset loss function until the first loss function value is less than or equal to the first set threshold, the training is terminated, and the target speech encoder is obtained.

[0011] In one example, the second set condition includes a second set threshold, and the initial alignment module is trained using the anti-fraud training data set until the initial alignment module meets the second set condition, and the target alignment module is obtained, including: Inputting the anti-fraud training data set into the target speech encoder and the preset text encoder respectively, and outputting a speech feature vector and a text feature vector; Add the speech feature vector and the text feature vector element by element to generate a fused feature vector; Calculating the matching probability of the fused feature vector through a fully connected layer and an activation function; The second loss function value corresponding to the matching probability is calculated by a second preset loss function until the second loss function value is less than or equal to the second set threshold, the training is terminated, and a target alignment module is obtained.

[0012] In one example, the step of fine-tuning the speech anti-fraud big model to be fine-tuned by using the anti-fraud training data set and the anti-fraud test data set to obtain the trained speech anti-fraud big model includes: Fixing parameters of the target speech encoder, the preset text encoder and the target alignment module; Inputting the anti-fraud training data set into the target speech encoder, the preset text encoder and the target alignment module respectively, and outputting a fused feature vector; The fused feature vector and the preset anti-fraud template are used to perform gradient update on the speech anti-fraud large model to be fine-tuned to obtain a prediction result; The third loss function value corresponding to the prediction result is calculated by a third preset loss function until the third loss function value is less than or equal to a third set threshold, and the training is terminated to obtain the trained speech anti-fraud big model, which is a model tested using the anti-fraud test data set.

[0013] The second aspect of the present application provides a fraudulent call identification device, comprising: A voice data acquisition module is used to acquire real-time voice data returned by the user device; A real-time anti-fraud template determination module is used to extract features from the real-time voice data using a trained voice anti-fraud model to obtain real-time voice features and determine a real-time anti-fraud template corresponding to the real-time voice features; The current recognition result output module is used to input the real-time voice features and the real-time anti-fraud template into the trained voice anti-fraud large model, and output the current recognition result of the fraudulent call.

[0014] A third aspect of the present application provides an electronic device, including: Processor; and The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0015] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0016] The technical solution provided by this application may have the following beneficial effects: In an embodiment of the present application, real-time voice data returned by a user device is obtained, the real-time voice data is processed using a trained voice anti-fraud model, real-time voice features are extracted, and a real-time anti-fraud template corresponding to the real-time voice features is determined, the real-time voice features and the real-time anti-fraud template are input into the trained voice anti-fraud model, and the current recognition result of the fraudulent call is output.

[0017] The technical solution of the present application can effectively identify and prevent fraudulent calls by timely processing real-time voice data through a trained voice anti-fraud model and extracting real-time voice features. At the same time, it can determine the real-time anti-fraud template associated with the real-time voice features, significantly improve the recognition accuracy of fraudulent calls, and effectively and flexibly respond to complex and changeable fraud methods.

[0018] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail the exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0020] Figure 1 It is a flowchart of a method for identifying fraudulent calls shown in an embodiment of the present application; Figure 2 This is a flow chart of a method for identifying fraudulent calls shown in another embodiment of the present application; Figure 3 This is the overall training process of the voice anti-fraud model shown in the embodiment of the present application; Figure 4is a schematic diagram of the structure of a speech encoder shown in an embodiment of the present application; Figure 5 It is a schematic diagram of a telephone anti-fraud process based on a trained voice anti-fraud large model shown in an embodiment of the present application; Figure 6 It is a structural schematic diagram of a fraud call identification device shown in an embodiment of the present application; Figure 7 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0022] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0023] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0024] Telecom fraud, especially telephone fraud, is a serious threat to social integrity and order. Although AI (Artificial Intelligence) technology has played an important role in fraud call detection and malicious website interception, and has improved the efficiency and accuracy of anti-fraud work to a certain extent, the AI ​​technology currently used in telephone fraud generally uses traditional deep learning algorithms, which lack effective semantic understanding capabilities, resulting in low recognition accuracy, a large number of false positives, and a serious impact on user experience.

[0025] In addition, existing voice anti-fraud AI technology generally adopts a two-stage process, which requires first performing voice recognition to obtain text, and then performing recognition through natural language processing (NLP). This process significantly reduces the final accuracy.

[0026] In response to the above problems, an embodiment of the present application provides a method for identifying fraudulent calls, which can significantly improve the accuracy of identifying fraudulent calls.

[0027] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0028] Figure 1 This is a flow chart of a method for identifying fraudulent calls shown in an embodiment of the present application. Figure 1 , the method comprises at least the following steps: Step 101: Acquire real-time voice data returned by a user device.

[0029] In an embodiment of the present application, real-time voice data returned by a user device may be obtained.

[0030] Among them, user equipment refers to electronic equipment capable of communication, including but not limited to smart phones, smart tablets, computers and other electronic devices.

[0031] Real-time voice data refers to the voice signal generated instantly by the user during a call. The voice signal is transmitted dynamically in the form of a stream, usually a voice waveform or encoded voice data.

[0032] Step 102, use the trained voice anti-fraud model to process the real-time voice data, extract the real-time voice features, and determine the real-time anti-fraud template corresponding to the real-time voice features.

[0033] In an embodiment of the present application, a trained voice anti-fraud model is used to process real-time voice data, extract real-time voice features, and determine a real-time anti-fraud template corresponding to the real-time voice features.

[0034] Among them, the trained voice anti-fraud big model refers to a model trained by deep learning methods, which has powerful voice feature extraction and anti-fraud recognition capabilities. This application realizes one-stage end-to-end voice anti-fraud recognition through the voice anti-fraud big model, which greatly improves the accuracy and generalization ability of voice fraud recognition.

[0035] End-to-End (E2E) means that the entire process from data input to data output is directly completed by a unified model or system, without the need for artificially divided steps or independent sub-module intervention in the middle. The core of end-to-end is to enable the overall model to directly learn the mapping relationship between input data and output results, avoiding artificially defined feature extraction or intermediate steps in the middle, thereby simplifying the process and improving efficiency and accuracy.

[0036] In actual applications, the voice anti-fraud model can directly extract real-time voice features from the real-time voice data returned by the user's device without going through the step-by-step processing of traditional voice recognition, such as converting voice into text before performing fraud identification.

[0037] Real-time speech features refer to the high-dimensional feature matrix extracted from audio data by the speech anti-fraud model, which retains key information in the speech, such as the frequency distribution, energy, duration, specific vocabulary, semantic information, speaking speed, intonation, etc., for subsequent matching and classification.

[0038] The real-time anti-fraud template refers to a template that is used to determine whether the call is a fraud call based on the real-time voice data currently received, by comparing it with the existing anti-fraud templates. If the real-time voice features match certain anti-fraud templates, these matching templates are selected as real-time anti-fraud templates. If the matching degree is not high, a new real-time anti-fraud template is generated by updating the existing template, thereby improving the judgment ability of the current voice data.

[0039] Step 103, input the real-time voice features and the real-time anti-fraud template into the trained voice anti-fraud model, and output the current recognition result of the fraudulent call.

[0040] In the embodiment of the present application, the real-time voice features and the real-time anti-fraud template are used as input data, which are input into the trained voice anti-fraud model, and the current recognition result of the fraudulent call is output. The current recognition result may be a classification result or inference output after the voice anti-fraud model processes the real-time voice data, which is used to determine whether the call is a fraud call and related information. For example, the current recognition result is a classification label of "fraud" or "non-fraud".

[0041] In an embodiment of the present application, real-time voice data returned by a user device is obtained, the real-time voice data is processed using a trained voice anti-fraud model, real-time voice features are extracted, and a real-time anti-fraud template corresponding to the real-time voice features is determined, the real-time voice features and the real-time anti-fraud template are input into the trained voice anti-fraud model, and the current recognition result of the fraudulent call is output.

[0042] The technical solution of the present application can effectively identify and prevent fraudulent calls by timely processing real-time voice data through a trained voice anti-fraud model and extracting real-time voice features. At the same time, it can determine the real-time anti-fraud template associated with the real-time voice features, significantly improve the recognition accuracy of fraudulent calls, and effectively and flexibly respond to complex and changeable fraud methods.

[0043] Figure 2 It is a flowchart of a method for identifying fraudulent calls shown in another embodiment of the present application. Figure 2 relatively Figure 1 The technical solution of the embodiment of the present application is described in more detail. The method may include the following steps: Step 201, construct a pre-trained voice anti-fraud model.

[0044] In the embodiment of the present application, the voice anti-fraud model includes the following core modules: a voice encoder, a text encoder, an alignment module and a base model.

[0045] The speech encoder is responsible for extracting features from speech data and converting them into a representation suitable for subsequent processing. Its task is to convert speech data into a high-dimensional, recognizable feature vector, providing sufficient speech information for subsequent model judgment.

[0046] The text encoder is used to process text data related to voice data. The objects to be processed include text information extracted from voice data or other text related to voice content.

[0047] The alignment module is responsible for aligning the outputs of the speech encoder and the text encoder to ensure that the outputs of the two modules can be combined for more accurate analysis and processing. The goal of alignment is to establish a correlation between the speech feature vector and the text feature vector, thereby effectively enhancing the performance of the model in the speech anti-fraud task.

[0048] The base model (also known as the initial voice anti-fraud model) can accept input from the voice encoder and text encoder and perform final fraud identification.

[0049] In the process of building a pre-trained speech anti-fraud model, it is necessary to first build an initial speech encoder, an initial alignment module and an initial speech anti-fraud model, then train the initial speech encoder and the initial alignment module to obtain the target speech encoder and the target alignment module, and then combine the target speech encoder and the target alignment module with the preset text encoder to form the data processing part of the speech anti-fraud model.

[0050] After the pre-trained voice anti-fraud big model receives the anti-fraud training data set and the anti-fraud test data set, it first analyzes the data, and then uses the analyzed data to fine-tune the initial voice anti-fraud big model (base big model) to obtain the final voice anti-fraud big model.

[0051] This application reduces the number of parameters that need to be adjusted in the fine-tuning stage by first optimizing the speech encoder and alignment module to a certain level and then fine-tuning the model. When the base large model is fine-tuned on this basis, it only needs to focus on optimizing its own parameters, which can not only speed up the training process, but also improve the convergence speed of the model.

[0052] Step 202: Use the anti-fraud training data set and the anti-fraud test data set to train and test the voice anti-fraud big model to obtain a trained voice anti-fraud big model.

[0053] In the embodiment of the present application, before training the model and testing the model, it is necessary to collect fraud-related data from multiple data sources, and divide these data into anti-fraud training data sets and anti-fraud test data sets in the form of data pairs. The anti-fraud training data set and the anti-fraud test data set are used to train and test the voice anti-fraud big model to obtain the optimal voice anti-fraud big model, that is, the trained voice anti-fraud big model.

[0054] More specifically, the anti-fraud training dataset consists of speech and text data pairs. There are two types of data pairs: one is the original speech data and the corresponding translated text, which is used for training the speech encoder, and the other is the original speech data and the related description text of whether it is a fraud, which is used to fine-tune the speech anti-fraud model. The anti-fraud test set consists of speech and text data pairs. The data pairs are speech and the related description text of whether it is a fraud.

[0055] In this application, reference is made to Figure 3 , Figure 3 This is the overall training process of the voice anti-fraud large model shown in the embodiment of the present application.

[0056] First, collect voice and text data, and divide them into anti-fraud training data set and anti-fraud test data set.

[0057] Then, an initial speech encoder, an initial alignment module and an initial speech anti-fraud model are constructed, and the initial speech encoder is trained using the anti-fraud training data set until the initial speech encoder meets the first set condition, and the target speech encoder is obtained. The parameters of the target speech encoder and the preset text encoder are fixed, and the initial alignment module is trained using the anti-fraud training data set until the initial alignment module meets the second set condition, and the target alignment module is obtained.

[0058] Next, the speech anti-fraud model to be fine-tuned is composed of the target speech encoder, the preset text encoder, the target alignment module and the initial speech anti-fraud model.

[0059] Finally, the anti-fraud training dataset and the anti-fraud test dataset are used to adjust the fine-tuned voice anti-fraud model to obtain a trained voice anti-fraud model.

[0060] The following will explain the training process of the speech encoder, text encoder, and alignment module one by one in combination with the anti-fraud training dataset and the anti-fraud test dataset: 1) The training process of the speech encoder can be: The speech encoder is designed with a speech feature compression module and a speech feature extraction module. During the training process, the speech feature compression module first extracts features and reduces the dimension of the original speech data through the convolution layer, and then maps the original speech data to the preset dimension through the fully connected layer, and outputs compressed speech features. The speech feature extraction module extracts high-dimensional speech features from the compressed speech features based on the multi-head attention mechanism, residual addition operation, and normalization operation.

[0061] Among them, the speech feature compression module can be composed of multiple convolutional layers and one fully connected layer, and a reshaping operation or a dimensionality reduction operation is added between the output of the convolutional layer and the input of the fully connected layer.

[0062] It can be understood that the convolution layer is used to extract features from the original speech data. The reshaping operation is used to convert the dimension of the convolution layer output data to the dimension that is suitable for the fully connected layer. The fully connected layer is used to map the reshaped data to a fixed feature space, for example, the fixed output feature number is 256, so as to provide standardized input for the subsequent speech feature extraction module. That is, original speech data → convolution layer (extract local features) → reshaping or dimensionality reduction (standardize data dimension) → fully connected layer (map to low-dimensional feature space).

[0063] As an example, see Figure 4 , Figure 4 It is a schematic diagram of the structure of the speech encoder shown in the embodiment of the present application.

[0064] The input of the speech feature compression module is speech feature data, whose matrix dimension is (1, T, F), where T is the number of speech frames obtained by segmenting the audio data into 30ms, F is the Mel spectrum feature with a length of 72, and the output is a speech feature vector of dimension (T / 4, 256).

[0065] The structure of the speech feature compression module can be two convolutions conv1 and conv2, a reshape operation (reshaping operation) and a fully connected layer. Conv1 is a convolution with a size of 3*3, a kernel number of 256, a step size of 2, and a padding of 1. The input is (1, T, F) and the output is (256, T / 2, F / 2); Conv2 is a convolution with a size of 3*3, a kernel number of 256, a step size of 2, and a padding of 1. The input is the output of conv1 and the output is (256, T / 4, F / 4); the reshape operation, the input is (256, T / 4, F / 4), and the output is (T / 4, 256*F / 4); the fully connected layer is a fully connected neural network with an input feature number of 256 and an output feature number of 256. The features after the reshap operation are (T / 4, 256*F / 4), and after full connection, they are (T / 4, 256).

[0066] The speech feature compression module can reduce data redundancy through dimensionality reduction operations while ensuring that feature information is not lost, thereby reducing the computational burden for subsequent modules.

[0067] Among them, the speech feature extraction module can be composed of several multi-head attention networks and a feedforward neural network (FFN) in series, and residual addition operation and normalization operation are designed between the multi-head attention network and the feedforward neural network.

[0068] It can be understood that the multi-head attention network is a network structure that can focus on multiple different parts of the input sequence in parallel, by dividing the input data into multiple "heads", each "head" independently calculates the attention weight, and then integrates this information together to obtain a richer feature representation. This mechanism can help the model capture the relationship between different positions in the input sequence, and is particularly suitable for processing long sequence data such as speech, text, etc.

[0069] The residual addition operation adds the original input to the output after processing by the multi-head attention network or feedforward neural network. This addition operation helps the flow of information and avoids information loss in deep networks.

[0070] The normalization operation is to adjust the input features so that they are within a certain range, such as [0, 1] or [-1, 1].

[0071] The input of the speech feature extraction module is a feature matrix of dimension (T / 4, 256), and the output of the speech feature compression module is a speech feature matrix of dimension (T / 4, 256).

[0072] The structure of the speech feature extraction module can be composed of 12 transformer blocks, where the transformer block is composed of a multi-head attention network, two residual addition and normalization operations, and a feedforward neural network FFN in series. Specifically, the multi-head attention network has 8 attention heads, and the feedforward network is composed of a fully connected layer with an input feature number of 256 and an output feature number of 512 and a fully connected layer with an input feature number of 512 and an output feature number of 256 in series. The multi-head attention network inputs a feature matrix of (T / 4, 256) and outputs a feature matrix of (T / 4, 256). Then, through a residual addition and normalization operation (Add1&Norm1), the Add1 operation is residual addition, and Norm1 normalizes the feature matrix of dimension (T / 4, 256) after residual addition. The feedforward neural network FFN inputs a feature matrix of normalization operation and outputs a feature matrix of dimension (T / 4, 256). Then, through another residual addition and normalization operation (Add2&Norm2), a feature matrix of dimension (T / 4, 256) is obtained.

[0073] During the training process of the speech encoder, the anti-fraud training data set is used to train the initial speech encoder to obtain a speech feature vector. The first loss function value corresponding to the speech feature vector is calculated by a first preset loss function until the first loss function value meets the first set condition, thereby obtaining a target speech encoder.

[0074] The first preset loss function may be a CTC (Connectionist Temporal Classification) loss function, and the calculation formula is as follows:

[0075] L is the CTC loss function value, P is the set of all possible paths, T is the length of the input sequence, is the output annotation label assigned by path p at time step t, and yt is the output probability distribution at time step t.

[0076] The first setting condition may be a first setting threshold. When the first loss function is less than or equal to the first setting threshold, it indicates that the recognition effect of the initial speech encoder is good (the error between the speech feature and the actual annotation label is small), and the initial speech encoder may be used as a target speech encoder.

[0077] As an example, add an output mapping fully connected layer, the input is a feature matrix of (T / 4, 256), and its structure is specifically a fully connected layer with an input feature number of 256 and an output feature number of n, where n is the number of dictionaries of the output text. The CTC loss function is used for training, where CTC is the negative log-likelihood of the sum of the probabilities of the correct path in all possible character paths of the output. For a given input sequence and the corresponding target output sequence, the first loss function loss value is calculated by the CTC loss function. When the first loss function is greater than the first set threshold, it means that the recognition effect of the initial speech encoder has not met expectations, and training continues. When the first loss function is less than or equal to the first set threshold, it means that the recognition effect of the initial speech encoder is good and can be used as a target speech encoder to improve the abstract ability of speech features.

[0078] 2) The training process of the text encoder can be: The text encoder is responsible for converting text data into a high-dimensional vector representation for alignment with speech features or for further processing. Its main goal is to capture the semantic, syntactic, and contextual information of the text so that different text expressions can be mapped to similar vector spaces.

[0079] In the present application, the text encoder can be an open source text encoder. For example, a text encoder of the open source qwen2.5-7b large model is selected, which can be directly used for text feature extraction, has strong generalization capabilities and multi-language support, and reduces the cost of training from scratch.

[0080] Although qwen2.5-7b provides an open source pre-trained model, in order to be more suitable for anti-fraud tasks, its text encoder part can be further fine-tuned.

[0081] As an example, prepare an anti-fraud text dataset, such as common fraud scripts, keyword lists, and non-fraudulent normal speech conversion texts. After word segmentation and encoding, decompose the sentences into subwords or word units, use word embedding to convert the text units into vector representations, load the text encoder weights of qwen2.5-7b as initialization parameters for training, and the trained text encoder converts the sentences into high-dimensional feature vectors of fixed dimensions to capture their semantic and contextual information.

[0082] 3) The training process of the alignment module can be: The alignment module is composed of fully connected layers with different feature numbers. For example, a speech feature vector alignment module is designed, which is composed of a fully connected layer with an input feature number of 256 and an output feature number of 2024, and a fully connected layer with an input feature number of 2024 and an output feature number of 3584 in series. Its input is the feature matrix output by the speech encoder (T / 4, 256), and its output is the alignment feature matrix (T / 4, 3584).

[0083] During the training process of the alignment module, the anti-fraud training data set is respectively input into the target speech encoder and the preset text encoder, and the speech feature vector and the text feature vector are output. The speech feature vector and the text feature vector are added element by element to generate a fused feature vector. The matching probability of the fused feature vector is calculated through the fully connected layer and the activation function. The second loss function value corresponding to the matching probability is calculated through the second preset loss function until the second loss function value meets the second set condition, and the target alignment module is obtained.

[0084] Among them, the calculation formula of the second loss function is as follows:

[0085] N is the number of speech and text data pairs, yi is the true label of the i-th sample (1 for match, 0 for mismatch), and pi is the probability that the model predicts that the i-th sample is a match.

[0086] It should be noted that during training, the parameters of the speech encoder and text encoder are fixed and do not participate in gradient updates, while the remaining modules participate in gradient updates.

[0087] The second setting condition may be a second setting threshold. When the second loss function is less than or equal to the second setting threshold, it indicates that the alignment effect of the initial alignment module is good (the error between the speech feature and the true label is small), and the initial alignment module may be used as a target alignment module.

[0088] As an example, matching loss is used for alignment training. Matching loss is the loss for judging whether a speech feature space and a text feature space match. It is realized by a fully connected layer with an input feature number of 3584 and an output feature number of 1. Specifically, a speech feature vector with a dimension of 3584 is obtained by adding each vector in the alignment feature matrix, and a text feature vector with a dimension of 3584 is obtained by adding each vector in the feature matrix output by the text encoder. Then, the speech feature vector and the text feature vector are added to obtain a fusion feature, and then the fusion feature vector is input to the fully connected layer, and the output of the fully connected layer is input to sigmoid (activation function) to obtain the probability p of speech-text matching. The loss value of the second loss function is calculated by the loss function. When the second loss function is greater than the second set threshold, it means that the alignment effect of the initial alignment module has not met expectations, and training continues. When the second loss function is less than or equal to the second set threshold, it means that the alignment effect of the initial alignment module is good and can be used as a target alignment module, so that the large language model can also process speech data, and end-to-end recognition of telephone anti-fraud is realized, and recognition accuracy is improved.

[0089] By training the speech encoder, text encoder, and alignment module encoder in steps, we can avoid making the overall model too complex and difficult to tune. In particular, training the speech encoder and alignment module separately can ensure the stability and efficiency of these two modules. This will help to clarify the optimization goals and improve model performance when fine-tuning the base large model.

[0090] Furthermore, after the speech encoder, text encoder and alignment module are trained, the fine-tuning process of the base large model can be: Fix the parameters of the target speech encoder, the preset text encoder and the target alignment module, input the anti-fraud training data set into the target speech encoder, the preset text encoder and the target alignment module respectively, output the fused feature vector, use the fused feature vector and the preset anti-fraud template to perform gradient update on the speech anti-fraud big model to be fine-tuned, obtain the prediction result, calculate the third loss function value corresponding to the prediction result by the third preset loss function, until the third loss function value meets the third set condition, obtain the trained speech anti-fraud big model, the trained speech anti-fraud big model is the model that completes the test using the anti-fraud test data set.

[0091] Among them, the calculation formula of the third loss function is as follows:

[0092] The vocabulary of the large model contains V words. For a given input, the model predicts the probability distribution of the next word P=[p1,p2,...,pV], where pi is the probability that the model predicts that the i-th word is the next word. The true distribution Q=[q1,q2,...,qV] is a one-hot encoded vector, where qi is 1, indicating that the next word is the i-th word, and the rest are 0.

[0093] The third setting condition may be a third setting threshold. When the third loss function is less than or equal to the third setting threshold, it indicates that the fine-tuning effect of the base large model is good and the base large model can be used as the optimal voice anti-fraud large model.

[0094] As an example, the trained voice anti-fraud model consists of a trained voice encoder, a trained alignment module, a qwen2.5-7b text encoding, and a qwen2.5-7b base model that needs fine-tuning.

[0095] The speech alignment feature matrix of the original speech data is obtained through the speech encoder and alignment module, and the text feature matrix is ​​obtained through the text encoder. The speech alignment feature matrix and the text feature matrix are used as input data and input into the qwen2.5-7b base large model that needs to be fine-tuned for reasoning.

[0096] Fine-tuning using the voice and text scam training datasets involves at least the following steps: First, design the input prompt (template) text. The prompt is as follows: "You are a voice anti-fraud expert, responsible for judging whether the voice involves fraud. You will follow a fixed format output: Chinese, the output is a JSON string, and the JSON contains three keys: judgment and reason. Judge: whether the voice involves fraud, if yes, please answer "1", if not, please answer "0"; reason: explain why you make such a judgment, if judge is "0", reason is empty; class: what type of fraud is it, if judge is "0", class is empty", through this prompt, set the role of the big model and set the big model for structured output.

[0097] Next, the voice anti-fraud model was fine-tuned. Only the open source qwen2.5-7b base model was gradient updated for all parameters, and the parameters of other modules were frozen and did not participate in the gradient update.

[0098] Among them, the setting of the loss objective function uses the cross entropy loss to calculate the cross entropy difference between the next word that appears in the real anti-fraud training sample and the predicted next word that appears, as follows:

[0099] The vocabulary of the large model contains V words. For a given input, the model predicts the probability distribution of the next word P=[p1,p2,...,pV], where pi is the probability that the model predicts that the i-th word is the next word. The true distribution Q=[q1,q2,...,qV] is a one-hot encoded vector, where qi is 1, indicating that the next word is the i-th word, and the rest are 0.

[0100] Furthermore, the voice data and the prompt with anti-fraud capabilities that can be updated in time are used as the input of the voice anti-fraud model for inference to obtain the final result. If a new type of fraud occurs, the anti-fraud capabilities of the voice anti-fraud model can be updated in time through the prompt.

[0101] For example, if a new type of fraud involving fake orders and rebates occurs, the prompt can be set as follows: "You are a voice anti-fraud expert, responsible for determining whether the voice involves fraud. Please fully understand the following new frauds and identify the type of fraud.

[0102] New frauds involving rebates for order brushing: This is the most varied and fastest-changing type of fraud, mainly involving complex frauds such as recruiting part-time order brushing workers and online pornography to induce order brushing. After gaining the trust of the victim, the fraudsters lure the victim into doing tasks by saying "the more you top up, the more rebates you get", and then use excuses such as "consecutive orders" and "card orders" to lure the victim into continuously transferring money.

[0103] You will follow a fixed format output: Chinese, the output is a json string, and the json contains three keys: judgement and reason, judge: whether the voice involves fraud, if it involves fraud, please answer "1", if it does not involve fraud, please answer "0"; reason: explain why you make such a judgment, if judge is "0", reason is empty; class: what type of fraud it is, if judge is "0", class is empty.

[0104] This application designs an anti-fraud prompt and uses a speech text training set related to telephone fraud to fine-tune the large language model to obtain a large vertical model for telephone anti-fraud, thereby greatly improving the anti-fraud identification capability.

[0105] Step 203: Use the trained voice anti-fraud model to extract real-time voice features from the real-time voice data.

[0106] In the actual process of fraud call identification, in order to achieve real-time monitoring of fraudulent voices, the speech encoder of the trained speech anti-fraud model is used to extract key real-time speech features from the received real-time speech data.

[0107] The speech feature compression module in the speech encoder uses a convolutional network to extract time domain and frequency domain features, and generates compact time series features through dimensionality reduction operations. The speech feature extraction module uses a multi-head attention mechanism to capture the long- and short-term dependencies of speech, combines residual connections to improve feature representation capabilities, and enhances stability through normalization operations. Finally, the speech encoder outputs a high-dimensional feature vector of fixed length, which comprehensively represents the intonation, speech speed, pauses and other information of the speech, providing key data support for fraud risk analysis.

[0108] Step 204: Use the real-time voice feature to update the preset anti-fraud template, and generate a real-time anti-fraud template corresponding to the real-time voice feature.

[0109] After obtaining the real-time voice features, the real-time voice features are matched with the existing anti-fraud templates. If the match is successful, the fraudulent behavior is identified based on the existing templates. If the match is incomplete or new features are found, the new features are marked and the update step is entered.

[0110] For example, classification analysis and machine learning can be combined to dynamically integrate new features into existing anti-fraud templates. The system adjusts template rules based on the importance of features, optimizes semantic recognition and behavior pattern detection capabilities, and verifies the accuracy and applicability of new templates through reinforcement learning and simulation testing, achieving adaptive evolution of templates.

[0111] Step 205, input the real-time voice features and the real-time anti-fraud template into the trained voice anti-fraud model, and output the current recognition result of the fraudulent call.

[0112] The real-time voice features and real-time anti-fraud templates are used as input data and input into the trained voice anti-fraud model to output the current recognition results of fraudulent calls.

[0113] As an example, the current recognition results may include: whether there is fraud (for example, outputting "1" to indicate that fraud is involved, and outputting "0" to indicate that fraud is not involved); if fraud is involved, the model will further analyze its type (such as fake order fraud, impersonating customer service, etc.) and give a corresponding type label.

[0114] As an example, refer to Figure 5 , Figure 5 It is a schematic diagram of the telephone anti-fraud process based on the trained voice anti-fraud large model shown in the embodiment of the present application.

[0115] S51, input part Voice data: Voice input provided by the user, which can be voice commands, conversations, or other voice signals, needs to be converted into a format that can be processed by the machine.

[0116] Text data: Another input independent of voice data, which may be text information related to voice data, or it may be completely independent text input, which needs to be processed and understood.

[0117] S52, data processing part Trained speech encoder: extracts key information from speech and provides high-quality feature representation for subsequent processing.

[0118] qwen2.5-7b text encoder: This module is responsible for processing the input text data and converting the text into an embedded representation or feature vector.

[0119] Trained alignment module: This module ensures that the representation of speech and text data is semantically aligned, helping the model better understand the relationship between the two. Through alignment, the fusion of speech and text information becomes more accurate.

[0120] S53, core model part The fine-tuned qwen2.5-7b base model: receives input from the speech encoder and text encoder, and deeply understands and processes this data.

[0121] S54, output section Recognition result: The final output result generated by the system after being processed by the model.

[0122] It should be noted that the embodiments of the present application include but are not limited to the above examples. It is understandable that under the guidance of the ideas of the embodiments of the present application, technical personnel in this field can make settings according to actual conditions, and the present application does not limit this.

[0123] In an embodiment of the present application, a pre-trained voice anti-fraud big model is constructed, and the voice anti-fraud big model is trained and tested using an anti-fraud training data set and an anti-fraud test data set to obtain a trained voice anti-fraud big model, and the trained voice anti-fraud big model is used to extract real-time voice features from real-time voice data, and the real-time voice features are used to update a preset anti-fraud template to generate a real-time anti-fraud template corresponding to the real-time voice features, and the real-time voice features and the real-time anti-fraud template are input into the trained voice anti-fraud big model, and the current recognition result of the fraudulent call is output, so that in the face of ever-changing fraud methods, the recognition ability can be improved by continuously updating the anti-fraud template, the recognition efficiency of new frauds can be improved, and the current recognition result of the fraudulent call can be accurately output.

[0124] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a fraudulent call identification device, electronic device and corresponding embodiments.

[0125] Figure 6Schematic diagram of a fraud call identification device shown in an embodiment of the present application. Figure 6 , the device comprises at least the following modules: The voice data acquisition module 601 is used to acquire the real-time voice data returned by the user equipment; The template determination module 602 is used to extract features from real-time voice data using a trained voice anti-fraud model, obtain real-time voice features, and determine a real-time anti-fraud template corresponding to the real-time voice features; The result output module 603 is used to input the real-time voice features and the real-time anti-fraud template into the trained voice anti-fraud model, and output the current recognition result of the fraudulent call.

[0126] As an optional example of the present application, the template determination module 602 is used to: Use the trained voice anti-fraud model to extract real-time voice features from real-time voice data; Determine a preset anti-fraud template associated with the real-time voice feature, and use the preset anti-fraud template as the real-time anti-fraud template; or, The preset anti-fraud template is updated using the real-time voice features to generate a real-time anti-fraud template corresponding to the real-time voice features.

[0127] As an optional example of this application, the training method of the voice anti-fraud large model includes: A data acquisition module is used to acquire anti-fraud training data sets and anti-fraud test data sets; The first construction module is used to construct an initial speech encoder, an initial alignment module and an initial speech anti-fraud model; A speech coding training module, used to train an initial speech coder using an anti-fraud training data set until the initial speech coder meets a first set condition, thereby obtaining a target speech coder; An alignment training module, used to fix the parameters of the target speech encoder and the preset text encoder, and train the initial alignment module using the anti-fraud training data set until the initial alignment module meets the second set condition, thereby obtaining a target alignment module; The second construction module is used to form a speech anti-fraud model to be fine-tuned by using a target speech encoder, a preset text encoder, a target alignment module and an initial speech anti-fraud model; The model fine-tuning module is used to adjust the voice anti-fraud large model to be fine-tuned using the anti-fraud training data set and the anti-fraud test data set to obtain a trained voice anti-fraud large model.

[0128] As an optional example of the present application, the anti-fraud training data set includes original speech data, and the initial speech encoder includes a speech feature compression module and a speech feature extraction module; The speech feature compression module includes a convolutional layer and a fully connected layer. The convolutional layer is used to extract features and reduce the dimension of the original speech data. The fully connected layer is used to map the original speech data to a preset dimension and output compressed speech features. The speech feature extraction module includes a multi-head attention network and a feedforward neural network. The multi-head attention network is used to extract speech information at different times or frequencies from compressed speech features, and the feedforward neural network is used to extract key speech features from speech information at different times or frequencies.

[0129] As an optional example of the present application, the first setting condition includes a first setting threshold, and the speech coding training module is used to: The initial speech encoder is trained using the anti-fraud training data set to obtain the speech feature vector; The first loss function value corresponding to the speech feature vector is calculated by a first preset loss function until the first loss function value is less than or equal to a first set threshold value, the training is terminated, and the target speech encoder is obtained.

[0130] As an optional example of the present application, the second setting condition includes a second setting threshold, and an alignment training module is used to: The anti-fraud training data set is input into the target speech encoder and the preset text encoder respectively, and the speech feature vector and the text feature vector are output; Add the speech feature vector and the text feature vector element by element to generate a fused feature vector; Calculate the matching probability of the fused feature vector through the fully connected layer and activation function; The second loss function value corresponding to the matching probability is calculated by the second preset loss function until the second loss function value is less than or equal to the second set threshold, the training is terminated, and the target alignment module is obtained.

[0131] As an optional example of the present application, the model fine-tuning module is used to: Fix the parameters of the target speech encoder, the preset text encoder, and the target alignment module; The anti-fraud training data set is input into the target speech encoder, the preset text encoder and the target alignment module respectively, and the fused feature vector is output; The fused feature vector and the preset anti-fraud template are used to perform gradient update on the fine-tuned voice anti-fraud model to obtain the prediction result; The third loss function value corresponding to the prediction result is calculated by the third preset loss function until the third loss function value is less than or equal to the third set threshold, the training is terminated, and the trained speech anti-fraud model is obtained. The trained speech anti-fraud model is a model that has been tested using the anti-fraud test data set.

[0132] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0133] Figure 7 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.

[0134] See also Figure 7 , the electronic device 700 includes a memory 710 and a processor 720 .

[0135] The processor 720 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0136] The memory 710 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. Among them, ROM may store static data or instructions required by the processor 720 or other modules of the computer. The permanent storage may be a readable and writable storage device. The permanent storage may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 710 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 710 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0137] The memory 710 stores executable codes, and when the executable codes are processed by the processor 720 , the processor 720 can execute part or all of the methods described above.

[0138] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0139] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0140] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for identifying fraudulent calls, characterized in that: include: Get real-time voice data returned by the user's device; Using a trained voice anti-fraud model to extract features from the real-time voice data, obtain real-time voice features, and determine a real-time anti-fraud template corresponding to the real-time voice features; The real-time voice features and the real-time anti-fraud template are input into the trained voice anti-fraud large model, and the current recognition result of the fraudulent call is output.

2. The method according to claim 1, characterized in that The method of using the trained voice anti-fraud big model to extract features from the real-time voice data to obtain real-time voice features, and determining a real-time anti-fraud template corresponding to the real-time voice features, includes: Extracting the real-time voice features from the real-time voice data using a trained voice anti-fraud model; Determine a preset anti-fraud template associated with the real-time voice feature, and use the preset anti-fraud template as the real-time anti-fraud template; or, The preset anti-fraud template is updated using the real-time voice feature to generate a real-time anti-fraud template corresponding to the real-time voice feature.

3. The method according to claim 1, characterized in that The training method of the voice anti-fraud model is as follows: Obtain anti-fraud training data set and anti-fraud test data set; Construct the initial speech encoder, initial alignment module and initial speech anti-fraud model; The initial speech encoder is trained using the anti-fraud training data set until the initial speech encoder meets a first set condition, thereby obtaining a target speech encoder; Fixing the parameters of the target speech encoder and the preset text encoder, and using the anti-fraud training data set to train the initial alignment module until the initial alignment module meets the second set condition, thereby obtaining a target alignment module; The target speech encoder, the preset text encoder, the target alignment module and the initial speech anti-fraud model constitute a speech anti-fraud model to be fine-tuned; The anti-fraud training data set and the anti-fraud test data set are used to adjust the speech anti-fraud large model to be fine-tuned to obtain the trained speech anti-fraud large model.

4. The method according to claim 3, characterized in that The anti-fraud training data set includes original speech data, and the initial speech encoder includes a speech feature compression module and a speech feature extraction module; The speech feature compression module includes a convolution layer and a fully connected layer, wherein the convolution layer is used to extract features and reduce the dimension of the original speech data, and the fully connected layer is used to map the original speech data to a preset dimension and output compressed speech features; The speech feature extraction module includes a multi-head attention network and a feedforward neural network. The multi-head attention network is used to extract speech information at different times or frequencies from the compressed speech features, and the feedforward neural network is used to extract key speech features from the speech information at different times or frequencies.

5. The method according to claim 3, characterized in that: The first set condition includes a first set threshold, and the anti-fraud training data set is used to train the initial speech encoder until the initial speech encoder meets the first set condition, and a target speech encoder is obtained, including: Using the anti-fraud training data set to train the initial speech encoder to obtain a speech feature vector; The first loss function value corresponding to the speech feature vector is calculated by a first preset loss function until the first loss function value is less than or equal to the first set threshold, the training is terminated, and the target speech encoder is obtained.

6. The method according to claim 3, characterized in that The second set condition includes a second set threshold, and the initial alignment module is trained using the anti-fraud training data set until the initial alignment module meets the second set condition, thereby obtaining a target alignment module, including: Inputting the anti-fraud training data set into the target speech encoder and the preset text encoder respectively, and outputting a speech feature vector and a text feature vector; Add the speech feature vector and the text feature vector element by element to generate a fused feature vector; Calculating the matching probability of the fused feature vector through a fully connected layer and an activation function; The second loss function value corresponding to the matching probability is calculated by a second preset loss function until the second loss function value is less than or equal to the second set threshold, the training is terminated, and a target alignment module is obtained.

7. The method according to claim 3, characterized in that The step of fine-tuning the speech anti-fraud big model to be fine-tuned by using the anti-fraud training data set and the anti-fraud test data set to obtain the trained speech anti-fraud big model includes: Fixing parameters of the target speech encoder, the preset text encoder and the target alignment module; Inputting the anti-fraud training data set into the target speech encoder, the preset text encoder and the target alignment module respectively, and outputting a fused feature vector; The fused feature vector and the preset anti-fraud template are used to perform gradient update on the speech anti-fraud large model to be fine-tuned to obtain a prediction result; The third loss function value corresponding to the prediction result is calculated by a third preset loss function until the third loss function value is less than or equal to a third set threshold, and the training is terminated to obtain the trained speech anti-fraud big model, which is a model tested using the anti-fraud test data set.

8. A device for identifying fraudulent calls, characterized in that: include: A voice data acquisition module is used to acquire real-time voice data returned by the user device; A template determination module, used to extract features from the real-time voice data using a trained voice anti-fraud model, obtain real-time voice features, and determine a real-time anti-fraud template corresponding to the real-time voice features; The result output module is used to input the real-time voice features and the real-time anti-fraud template into the trained voice anti-fraud model and output the current recognition result of the fraudulent call.

9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 7.

Images