Small sample telecommunication anti-fraud method and device for rapid variation information

Abstract

The invention discloses a small sample telecommunication anti-fraud method and device for rapid variation information. The method comprises the following steps: constructing a feature extractor by using a general large model; using new data to collect a support data set; collecting feature vectors of the support data set by using a general large model; obtaining a representation vector expression of the support data set, and assembling a feature matrix; and training the extended small model to adapt to the new data risk. The method comprises the following steps: firstly, collecting several to dozens of unequal support data sets of small samples, then carrying out numerical feature conversion on support texts by utilizing a super-large-scale language model, and reducing the requirements of large samples while keeping text semantics; dimensionality reduction is carried out in a sample space of a supporting text, the dimensionality which contributes to new risk identification precision to the greatest extent is reserved, and finally, a small model is trained in a new feature space to complete new risk identification. The invention provides a small sample telecommunication fraud risk identification method which is capable of sustainable development and rapid iteration, aiming at the problem of rapid variation telecommunication anti-fraud identification.

Description

Technical Field

[0001] The present invention belongs to the field of small sample learning in the field of deep learning, and relates to a telecommunications anti-fraud method and device that uses a deep neural network model of small sample learning to cope with rapid mutations. Background Art

[0003] In the past decade, deep learning has achieved remarkable results in various tasks, which also depends on the construction and development of massive labeled samples. However, collecting a large number of samples means a large amount of risk leakage for the victim group, which should be avoided as much as possible. Therefore, how to use as few samples as possible to cover new fraud problems after changes is a major difficulty in fraud risk identification. While ensuring the effectiveness of the extracted deep semantic logical features, reducing the demand for a large number of samples in traditional machine learning is what this patent hopes to solve. Summary of the invention

[0004] To solve the above problems, the present invention provides a telecommunications anti-fraud method and device for rapidly changing information. Traditional machine learning requires a large number of samples. However, collecting a large number of samples means a large amount of risk spillover for the victim group. Therefore, in the early stage of new risks, using as few samples as possible to ensure the effectiveness of the extracted deep semantic logical features, cover the new fraud problems after changes, and effectively respond to the emerging new fraud risks. The present invention proposes a telecommunications anti-fraud method and system that uses a deep neural network model with small sample learning to respond to rapidly changing information.

[0005] The first aspect of the present invention relates to a telecommunications anti-fraud method for rapidly changing information, comprising the following steps:

[0006] 1. Use a general large model to build a feature extractor;

[0007] The Bert language model is used to generate features for text, converting the original text information into digital vector expressions for later fraud identification algorithm calculations.

[0008] 2. Use new data to collect supporting data sets;

[0009] For the missed risk samples that cannot be identified by the original model, negative samples are collected, and for the normal samples that are incorrectly identified by the original model, positive samples are collected.

[0010] 3. Collect feature vectors supporting the dataset using a general large model;

[0011] The samples in the newly emerged supporting data sets that cannot be correctly identified are input into the large model one by one and converted into supporting data set representation vectors as the basis for subsequent model optimization.

[0012] 4. Obtain the representation vector expression of the supporting data set and assemble the feature matrix;

[0013] The collected support data feature vectors are reduced in dimension using the principal component analysis method to obtain a more concentrated support data set feature matrix.

[0014] 5. Train and expand small models to adapt to new data risks;

[0015] A small neural network is further trained on the feature matrix using the supporting dataset to generate an expanded small model to identify new data risks.

[0016] The first aspect of the present invention relates to a small sample telecommunications anti-fraud device for rapidly changing information, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the small sample telecommunications anti-fraud method for rapidly changing information of the present invention.

[0017] A computer-readable storage medium of the present invention stores a program, which, when executed by a processor, implements a small sample telecommunications anti-fraud method of the present invention for rapidly changing information.

[0018] The present invention takes into account the dependence of traditional deep neural network models on a large number of samples. Collecting a large number of samples means a large number of risk leaks. How to use as few samples as possible to cover the new fraud problems after changes is the problem of risk identification that the present invention focuses on solving. The present invention first collects a supporting data set of small samples, usually ranging from a few to dozens of items, and then uses a very large-scale language model to convert the supporting text into numerical features, retaining the semantics of the text while reducing the demand for large samples; then, the sample space of the supporting text is reduced in dimension, retaining the dimensions that contribute most to the accuracy of new risk identification, and finally, a small model is trained in the new feature space to complete the identification of new risks. In response to the problem of rapidly changing telecommunications anti-fraud identification, the present invention provides a sustainable and rapidly iterable small sample telecommunications fraud risk identification method.

[0019] The advantages of the present invention are:

[0020] Supporting data sets for emerging risks in telecommunications fraud are collected at the earliest stage of problem discovery. By using a large model to extract and express features on the supporting data set, and by using small samples to quickly build and expand small models to cover new risks, there is no need to retrain the original large model, but the encoding capacity of the large model is directly utilized to ensure the validity of the extracted deep semantic logical features, while reducing the demand for a large number of samples in traditional machine learning. The implementation cost is low, providing a feasible solution for the rapid identification of telecommunications fraud that changes rapidly in actual scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] FIG1( a ) is a schematic diagram of the model training process of the present invention when an original discrimination method already exists.

[0022] FIG1( b ) is a schematic diagram of the model training process of the present invention when there is no original discriminant model.

[0023] FIG. 2( a ) is a schematic diagram of the model prediction process of the present invention when an original discrimination method already exists.

[0024] FIG. 2( b ) is a schematic diagram of the model prediction process of the present invention when there is no original discriminant model. DETAILED DESCRIPTION

[0025] The following is a clear and complete explanation and description of the technical solution of the present invention in conjunction with the accompanying drawings, and an exemplary implementation scheme of the present invention is described. This detailed description should not be considered as a limitation of the present invention, but should be understood as a more detailed description of certain aspects and features of the present invention when applied.

[0026] Example 1

[0027] The present invention provides a telecommunication anti-fraud identification method for rapidly changing information, comprising the following steps:

[0028] 1. Use the general large model to build a feature extractor.

[0029] At present, the text big model can describe the semantic features of the text very well through supervised or unsupervised learning of massive samples. Using the general text big model for feature extraction, the text data can be converted into numerical features suitable for further processing or analysis. The present invention adopts the BERT model, which can dynamically generate word embeddings according to the context. These embeddings not only consider the word itself, but also combine the context, making the feature representation richer. We convert the original telecommunication text t into embedded features Here i Each Chinese character, after feature conversion, The dimension is 768×2048. Here 768 is the feature dimension of each Chinese character, and 2048 is the length of the sample sentence L. If the length of the sample exceeds L, the effective text length can be increased by removing commonly used auxiliary words and punctuation marks. If the length exceeds L, the sentence can also be truncated.

[0030]

[0031] 2. Use new data to collect supporting data sets.

[0032] Case 1: If a telecommunications fraud identification model has been prepared, it can be used directly as the original model. When collecting supporting data sets, the newly added risks collected externally (such as samples obtained by reporting, spot checks, etc., and samples near the discrimination threshold) are manually verified and checked one by one through the original model. For the missed risk samples that the original model cannot identify, negative samples are collected, and for the normal samples that the original model incorrectly identifies, positive samples are collected. We denote the new data set as S.

[0033] Case 2: If there is no prepared original model, you can use the original data set for training to get a basic model. The original data set is usually an accumulation of violation data that has accumulated over time, and has positive and negative sample labels. The original data set is generally very large in scale and is the basic data of telecommunications units. Unlike the supporting data set for new risks, the supporting data is the latest risk data and is very small in scale. It is usually in the earliest stage when new risks appear, usually a few or dozens of items, and the timeliness is data that has never appeared in history within the same day or a few days, and the original model cannot recognize it. Generally speaking, if new risks are not intercepted as quickly as possible, they will spread rapidly, causing the original model to fail quickly.

[0034] If there is no original model and the original data set needs to be used to build the original model, the BERT model can be used to extract features from the original data set, and then a linear classifier can be used for binary classification training. The linear classifier can use the Cross-entropy+Sigmoid method, y label is the true label of the sample, with a dimension of 2, is the label vector calculated by the model, with the same dimension as y label The cross entropy loss function is: Loss = -y label log(y), using the gradient descent method, we can get the parameters (w0, b0) corresponding to the original model M0. After obtaining the original model M0, we can pass the newly added risks collected from the outside (such as samples obtained by reporting, spot checks, etc., and samples near the discrimination threshold) through the original model M0 one by one, as in case 1, and perform manual verification (only the maximum dimension of y needs to be determined). For the missed risk samples that the original model cannot identify, negative samples are collected, and for the normal samples that the original model incorrectly identifies, positive samples are collected. Thus, a new data set S is obtained.

[0035] 3. Use the general large model to collect feature vectors supporting the data set.

[0036] The samples in the newly appeared support dataset S that cannot be correctly identified are input into the BERT model one by one and converted into support dataset representation vectors as the basis for subsequent model optimization. s, the transformed representation vector is Its dimensions are The size is 768×2048, and the preprocessing method is the same, including punctuation and auxiliary word filtering, and length truncation. Each supporting data corresponds to a Assuming that I pieces of support data are collected, we can get a support data feature matrix of size I×768×2048

[0037]

[0038] 4. Obtain the representation vector expression of the supporting data set and assemble the feature matrix.

[0039] For the converted Dimensionality reduction is needed because we have no way to collect massive samples. For a sample space of 768×2048, dozens of supporting data set samples are not enough. Therefore, principal component analysis is used to reduce the dimension to a describable range and maximize the distinguishing ability. The specific high-dimensional principal component analysis calculation method is as follows. For tensors Its dimension is I×768×2048. There are three modes of dimensionality reduction. We choose to reduce the dimension in the first dimension.

[0040] First, the tensor is expanded into a matrix along the first dimension. Here we will Convert to a matrix of I×(768×2048) i=1,2,…,I,j=1,2,…,J,k=1,2,…,K,J=768,K=2048,that is, the tensor Each slice of Flattened to one row, χ is a matrix of size I×(768×2048).

[0041] Then, calculate the covariance matrix C of χ. Here The dimensions of C are (JK) × (JK). Matrix multiplication is performed here.

[0042] Then, calculate the eigenvalue λ of the covariance matrix C and the corresponding eigenvector v. Take the largest top_n features λ in λ top_n and the corresponding eigenvector v top_n , where V top_n The dimension of is (JK) × top_n. Here, top_n is selected based on the size of each component of λ. Eigenvalues ​​with λ<0.1 are discarded. top_n is the number of remaining eigenvalues, which usually does not exceed 10.

[0043] Finally, the original sample features X and v top_nMultiply them to get the reduced eigenvector χ down The dimension of χ is I×(JK), where I is the number of samples in the supporting dataset and χ down The dimension is I×top_n, which is reduced from JK to top_n.

[0044] 5. Train and expand small models to adapt to new data risks.

[0045] Using the feature χ obtained after dimensionality reduction in the support dataset down And the corresponding label ψ, train a small classifier M1, the corresponding parameters are (w1, b1). Here we use Cross-entropy+softmax as the model structure, and replace the inner product in the neural network with cosine similarity calculation. The specific process is as follows, Predicted classification results:

[0046] y=Softmax(sim(w1,χ down )+b1)

[0047] Cross entropy loss function: Loss = -ψlog(y), using the gradient descent method, we can get the parameters (w1, b1) corresponding to the original model M1. When judging whether a sample is illegal, we only need to judge the maximum dimension of y. This small neural network and supporting data set are further trained on the feature matrix to generate an extended small model to identify new data risks. Because the amount of data here is small, similarity calculation is used instead of dot multiplication and inner product. After a significant dimensionality reduction, the features can also match the dimensions of a simple neural network model, and there will be no overfitting problem.

[0048] 6. The overall process is divided into three parts: model training, model prediction, and model update.

[0049] (1) Model training

[0050] For the case where there is already an original discriminant model M0:

[0051] First, the existing discriminant model M0 is used to filter data in real time, and new data is extracted from the generated risk-free data or suspected data to form a support data set S (the method is the same as in case 1 of step 2). Then, the general open source large model BERT is used to generate features for each data in the support data set

[0052] (As shown in step 3), the feature dimension is high at this time, but the number is small. Then, the feature dimension is reduced (as shown in step 4). Then, the ultra-small scale extended model M1 is trained (as shown in step 5). The overall process is shown in Figure 1(a).

[0053] For the case without the original discriminant model M0:

[0054] First, the original discriminant model M0 needs to be trained using the huge amount of historical data (mainly using steps 1 and 2). Then, just like the previous case, the trained discriminant model M0 is used to filter the data in real time, and new data is extracted from the generated risk-free data or suspected data to form the support data set S. Then, the general open source large model BERT is used to generate features for each piece of data in the support data set. (As in step 3), the feature dimension is high, but the number is small. Next, the feature is reduced to obtain χ down (As shown in step 4). Then, the ultra-small-scale extended model M1 is obtained by training (as shown in step 5). The overall process is shown in Figure 1(b).

[0055] (2) Model prediction

[0056] For the already trained ultra-small-scale extended model M1 and the original discriminant model M0, in the actual application process, first, the text segment to be detected is first input into the original BERT model to obtain the feature vector (As in step 1). Then, the feature vector is input into the original discriminant model M0 to calculate For samples judged as risk by M0 (e.g., when y≥0.5), they are directly judged as risk samples. If the original model is not trained in step 2, it is directly judged based on the model output without calculating the y value. Next, for samples judged as not risky by M0 (e.g., when y<0.5), the feature To reduce the dimension, With v top_n Multiply them together to reduce the dimension to top_n and get the vector χ down Then, the new feature vector is input into the model M1, and y=Softmax(sim(w1,χ down )+b1), those M1 judged as risk (y≥0.5) are directly judged as risk samples. The overall process is shown in Figure 2.

[0057] (3) Model update

[0058] In the case where the supporting data set gradually increases over time, model M1 can be updated according to the steps of model training, and model M0 can be kept unchanged. Because the scale of model M1 is small, the cost of its training is small. After multiple updates of model M1, model M0 can be updated synchronously once according to the steps of model training. For example, the update frequency of model M1 can be kept daily, and model M0 can be updated monthly. Model updates can maintain the robustness of the overall telecommunications anti-fraud system.

[0059] Example 2

[0060] This embodiment involves a small sample telecommunications anti-fraud device for rapidly changing information, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a small sample telecommunications anti-fraud method for rapidly changing information in embodiment 1.

[0061] Example 3

[0062] This embodiment relates to a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, a small sample telecommunications anti-fraud method for rapidly changing information according to Embodiment 1 is implemented.

[0063] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to the equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A small sample telecommunications anti-fraud method for rapidly changing information, comprising the following steps: 1) Use a general large model to build a feature extractor; Use the Bert language model to generate features for text; convert the original text information into digital vector expressions for later fraud identification algorithm calculations; 2) Use new data to collect supporting data sets; For the missed risk samples that the original model cannot identify, negative samples are collected, and for the normal samples that the original model incorrectly identifies, positive samples are collected; 3) Collect feature vectors of supporting data sets using a general large model; The samples in the newly appeared support data set that cannot be correctly identified are input into the large model one by one and converted into support data set representation vectors as the basis for subsequent model optimization; 4) Obtain the representation vector expression of the supporting data set and assemble the feature matrix; The collected support data feature vectors are reduced in dimension using the principal component analysis method to obtain a more concentrated support data set feature matrix; 5) Train and expand small models to adapt to new data risks; A small neural network is further trained on the feature matrix along with the supporting dataset to generate an expanded small model to identify new data risks.

2. A small sample telecommunications anti-fraud method for rapidly changing information as claimed in claim 1, characterized in that: Step 1 specifically includes: Replace the original telecom text t with embedded features Here i Each Chinese character, after feature conversion, The dimension is the feature dimension of each Chinese character × the length of the sample sentence; the feature dimension of each Chinese character is 768, and the length of the sample sentence L is 2048; if the length of the sample exceeds L, the effective text length is increased by removing commonly used auxiliary words and punctuation marks, or the sentence is truncated; 3. A small sample telecommunications anti-fraud method for rapidly changing information as claimed in claim 1, characterized in that: Step 2 specifically includes: Case 1: If the telecommunications fraud identification model has been prepared, it can be used directly as the original model. When collecting the supporting data set, the newly added risks collected from the outside are manually checked one by one through the original model. For the missed risk samples that the original model cannot identify, negative samples are collected, and for the normal samples that the original model incorrectly identifies, positive samples are collected. The new data set is recorded as S. Case 2: If there is no prepared original model, you can use the original data set for training to get a basic model; the original data set is usually the accumulation of violation data accumulated over time, with positive and negative sample labels; the original data set is generally very large in scale and is the basic data of telecommunications units; different from the supporting data set of new risks, the supporting data is the latest risk data, with a very small scale, usually in the earliest stage of new risks, usually a few or dozens of data, with a timeliness of the day or a few days, which has never appeared in history and cannot be identified by the original model; usually, if new risks are not intercepted at the fastest speed, they will spread rapidly, causing the original model to fail quickly; If there is no original model and the original data set needs to be used to build the original model, the BERT model is used to extract features from the original data set, and then a linear classifier is used for binary classification training; the linear classifier can use the Cross-entropy+Sigmoid method, y label is the true label of the sample, with a dimension of 2, is the label vector calculated by the model, with the same dimension as y label Consistent; cross entropy loss function is: Loss = -y label log(y), and the gradient descent method can be used to obtain the parameters (w0, b0) corresponding to the original model M0. After obtaining the original model M0, as in case 1, the newly added risks collected from the outside can be manually verified through the original model M0 one by one. For the missed risk samples that the original model cannot identify, negative samples are collected, and for the normal samples that the original model incorrectly identifies, positive samples are collected. Thus, a new data set S is obtained.

4. A small sample telecommunications anti-fraud method for rapidly changing information as claimed in claim 1, characterized in that: Step 3 specifically includes: The samples in the newly appeared support dataset S that cannot be correctly identified are input into the BERT model one by one and converted into the support dataset representation vector as the basis for subsequent model optimization; as the single text t in the support dataset s , the transformed representation vector is Its dimensions are Keep consistent, preprocessing methods are also consistent, including punctuation and auxiliary word filtering, and length truncation; each supporting data corresponds to a Assuming that I pieces of support data are collected, we can get a support data feature matrix of size I×768×2048 5. A small sample telecommunications anti-fraud method for rapidly changing information as claimed in claim 1, characterized in that: Step 4 specifically includes: For the converted The principal component analysis method is used to reduce the dimension to a describable range and maximize the distinguishing ability; the specific high-dimensional principal component analysis calculation method is as follows; for the tensor Its dimension is I×768×2048. There are three modes of dimensionality reduction. We choose to reduce the dimension in the first dimension. First, the tensor Expanded along the first dimension into a matrix; here Convert to a matrix of I×(768×2048) i=1,2,…,I,j=1,2,…,J,k=1,2,…,K,J=768,K=2048,that is, the tensor Each slice of Flatten into one row, so X is a matrix of I×(768×2048); Then, calculate the covariance matrix C of χ; here The dimension of C is (JK)×(JK); matrix multiplication is performed here; Then, calculate the eigenvalue λ of the covariance matrix C and the corresponding eigenvector v; take the largest top_n features λ in λ top_n and the corresponding eigenvector v top_n , where v top_n The dimension is (JK)×top_n; here, the selection of top_n is based on the size of each component of λ, and the eigenvalues ​​with λ<0.1 are discarded, and top_n is the number of remaining eigenvalues; Finally, the original sample features χ and v top_n Multiply them to get the reduced eigenvector χ down ; The dimension of χ is I×(JK), where I is the number of samples in the supporting data set, and χ down The dimension is I×top_n, which is reduced from JK to top_n.

6. A small sample telecommunications anti-fraud method for rapidly changing information as claimed in claim 1, characterized in that: Step 5 specifically includes: Using the feature χ obtained after dimensionality reduction in the support dataset down and the corresponding label ψ, train a small classifier M1 with the corresponding parameters (w1, b1); Cross-entropy+softmax is used as the model structure, and the inner product in the neural network is replaced by cosine similarity calculation; the specific process is as follows, Predicted classification results: y=Softmax(sim(w1,x) down )+b1) Cross entropy loss function: Loss = -ψlog(y), using the gradient descent method, we can get the parameters (w1, b1) corresponding to the original model M1; when judging whether a sample is illegal, we only need to judge the maximum dimension of y; this small neural network and supporting data set are further trained on the feature matrix to generate an extended small model to identify new data risks; because the amount of data here is small, similarity calculation is used instead of dot multiplication and inner product. After a large reduction in dimensionality, the features can match the dimensions of a simple neural network model and will not cause overfitting problems.

7. A small sample telecommunications anti-fraud method for rapidly changing information as claimed in claim 1, characterized in that: It is divided into three parts: model training, model prediction, and model update; (1) Model training For the case where the original discriminant model M0 already exists: First, the existing discriminant model M0 is used to perform real-time data filtering, and new data is extracted from the generated risk-free data or suspected data to form a support data set S; Then, the general open source large model BERT is used to generate features for each data in the supporting dataset. At this time, the feature dimension is high, but the number is small; then, the feature dimension is reduced; then, the ultra-small scale extended model M1 is trained; In the case where there is no original discriminant model M0: First, it is necessary to use the huge amount of historical accumulated data to train the original discriminant model M0; then, just like the previous case, the trained discriminant model M0 is used to perform real-time data filtering, and new data is extracted from the generated risk-free data or suspected data to form the support data set S; Then, we use the general open source large model BERT to generate features for each piece of data in the supporting dataset. At this time, the feature dimension is high, but the number is small; then, the feature dimension is reduced to obtain χ down ; Then, train to obtain an ultra-small scale extended model M1; (2) Model prediction For the already trained ultra-small-scale extended model M1 and the original discriminant model M0, in the actual application process, first, the text segment to be detected is first input into the original BERT model to obtain the feature vector Then, the feature vector is input into the original discriminant model M0 to calculate For samples judged as risk by M0, they are directly judged as risk samples. For the case where the original model is not trained in step 2, it is directly judged based on the model output without calculating the y value. Then, for samples judged as not risky by M0, the feature To reduce the dimension, With v top_n Multiply them together to reduce the dimension to top_n and get the vector χ down ; Then, the new feature vector is input into the model M1, and y=Softmax(sim(w1,χ down )+b1), M1 judged as risk is directly judged as a risk sample; (3) Model update When the supporting data set gradually increases over time, model M1 can be updated according to the steps of model training, and model M0 can be kept unchanged; because the scale of model M1 is small, the cost of its training is small; after multiple updates of model M1, model M0 can be updated synchronously once according to the steps of model training; for example, the update frequency of model M1 can be kept at daily updates, and model M0 can be updated every month; Model updates can maintain the robustness of the overall telecommunications anti-fraud system.

8. A small sample telecommunications anti-fraud method for rapidly changing information as claimed in claim 1, characterized in that: The newly added risks collected externally in step 3 include samples obtained through reporting, spot checks, etc., as well as samples near the discrimination threshold.

9. A small sample telecommunications anti-fraud method for rapidly changing information as claimed in claim 1, characterized in that: The manual verification check described in step 2 only needs to determine the maximum dimension of y, and the top_n described in step 4 is the number of remaining eigenvalues, which does not exceed 10.

10. Implement the small sample telecommunication anti-fraud device for fast-changing information as claimed in claim 1, characterized in that: It includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement a small sample telecommunications anti-fraud method for rapidly changing information as described in any one of claims 1-9.

Images