A method and apparatus for small-sample telecommunications fraud prevention targeting rapidly changing information.

Abstract

A method and apparatus for small-sample telecommunications fraud detection targeting rapidly changing information is disclosed. The method includes: constructing a feature extractor using a general large model; collecting a supporting dataset using new data; collecting feature vectors of the supporting dataset using the general large model; obtaining the representation vector expression of the supporting dataset and assembling a feature matrix; and training an extended small model to adapt to new data risks. This invention first collects a small-sample supporting dataset, typically ranging from a few to dozens of data points. Then, it uses a large-scale language model to perform numerical feature transformation on the supporting text, preserving the text semantics while reducing the need for large samples. Next, it performs dimensionality reduction in the sample space of the supporting text, retaining the dimension that contributes most to the accuracy of new risk identification. Finally, it trains a small model in the new feature space to complete the identification of new risks. This invention provides a sustainable and rapidly iterative small-sample telecommunications fraud risk identification method for the problem of rapidly changing telecommunications fraud detection.

Description

Technical Field

[0001] This invention belongs to the few-shot learning direction in the field of deep learning, and relates to a method and device for combating rapidly changing telecommunications fraud by using a deep neural network model learned from few shots. Background Technology

[0002] Over the past decade, deep learning has achieved remarkable results in various tasks, which relies on the construction and development of massive amounts of labeled samples. However, collecting a large number of samples means a significant risk exposure for the victim population, which should be avoided as much as possible. Therefore, how to cover new and evolving fraud problems with as few samples as possible is a major challenge in fraud risk identification. This patent aims to solve the problem of reducing the need for large amounts of samples in traditional machine learning while ensuring the effectiveness of the extracted deep semantic and logical features. Summary of the Invention

[0003] To address the aforementioned problems, this invention provides a method and apparatus for combating telecommunications fraud based on rapidly evolving information. Traditional machine learning requires a large number of samples. However, collecting a large number of samples means significant risk spillover for the victim population. Therefore, in the early stages of emerging risks, it is crucial to utilize as few samples as possible to cover the changed new fraud problems while ensuring the effectiveness of the extracted deep semantic and logical features, effectively responding to the constantly emerging new fraud risks. This invention proposes a method and system for combating rapidly evolving telecommunications fraud using a deep neural network model learned from a small sample size.

[0004] The first aspect of this invention relates to a telecommunications anti-fraud method for rapidly changing information, comprising the following steps:

[0005] 1. Construct a feature extractor using a general large model;

[0006] The BERT language model is used to generate features from the text. The raw text information is converted into a digital vector representation for later use in fraud detection algorithms.

[0007] 2. Utilize new data collection to support the dataset;

[0008] For samples at risk of being missed that the original model could not identify, negative samples were collected, while for normal samples that were incorrectly identified by the original model, positive samples were collected.

[0009] 3. Use a general large model to collect feature vectors supporting the dataset;

[0010] Newly emerging examples from the support dataset that cannot be correctly identified are input one by one into the large model and transformed into support dataset representation vectors, which serve as the basis for subsequent model optimization.

[0011] 4. Obtain the representation vectors of the supporting dataset and assemble the feature matrix;

[0012] The collected support data feature vectors are then dimensionality-reduced using principal component analysis to obtain a more concentrated support dataset feature matrix.

[0013] 5. Train extended small models to adapt to new data risks;

[0014] By using a small neural network and supporting dataset to further train on the feature matrix, an expanded small model is generated to identify new data risks.

[0015] The first aspect of the present invention relates to a small-sample telecommunications fraud prevention device for rapidly changing information, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a small-sample telecommunications fraud prevention method for rapidly changing information according to the present invention.

[0016] The present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a small-sample telecommunications fraud prevention method for rapidly changing information.

[0017] This invention addresses the dependence of traditional deep neural network models on large datasets, which implies significant risk exposure. Therefore, this invention focuses on how to cover novel fraud scenarios with minimal sample size. First, a small support dataset, typically consisting of a few to dozens of data points, is collected. Then, a large-scale language model is used to perform numerical feature transformation on the support text, preserving its semantics while reducing the need for large samples. Next, dimensionality reduction is performed in the sample space of the support text, retaining the dimensions that contribute most to the accuracy of new risk identification. Finally, a small model is trained in the new feature space to identify the new risks. This invention provides a sustainable and rapidly iterative method for identifying telecommunications fraud risks using a small sample dataset, addressing the rapidly evolving problem of telecommunications fraud detection.

[0018] The advantages of this invention are:

[0019] By collecting supporting datasets at the earliest stage of problem discovery for emerging risks in telecommunications fraud, and using large models for feature extraction and representation on these supporting datasets, and rapidly building extended small models to cover new risks using small samples, this approach avoids the need to retrain the original large model. Instead, it directly utilizes the encoding capabilities of the large model, ensuring the effectiveness of the extracted deep semantic and logical features while reducing the need for large numbers of samples required by traditional machine learning. This low-cost approach provides a feasible solution for the rapid identification of telecommunications fraud in rapidly changing real-world scenarios. Attached Figure Description

[0020] Figure 1(a) is a schematic diagram of the model training process of the present invention when the original discrimination method already exists.

[0021] Figure 1(b) is a schematic diagram of the model training process of the present invention when there is no original discriminant model.

[0022] Figure 2(a) is a schematic diagram of the prediction process of the model of the present invention when the original discrimination method already exists.

[0023] Figure 2(b) is a schematic diagram of the prediction process of the present invention when there is no original discriminant model. Detailed Implementation

[0024] The technical solution of the present invention will be clearly and completely explained and described below with reference to the accompanying drawings, and exemplary embodiments of the present invention will be illustrated. This detailed description should not be regarded as a limitation of the present invention, but should be understood as a more detailed description of the application of certain aspects and characteristics of the present invention.

[0025] Example 1

[0026] This invention provides a telecommunications fraud identification method for rapidly changing information, comprising the following steps:

[0027] 1. Construct a feature extractor using a general large model.

[0028] Currently, large-scale text models, through supervised or unsupervised learning on massive samples, can already describe the semantic features of text very well. Using general-purpose large-scale text models for feature extraction can transform text data into numerical features suitable for further processing or analysis. This invention employs the BERT model, which can dynamically generate word embeddings based on context. These embeddings not only consider the words themselves but also the contextual information, resulting in richer feature representations. We replace the original telecommunications text t with embedded features. Here t i Each Chinese character, after feature conversion, The dimension is 768×2048. Here, 768 is the feature dimension of each Chinese character, and 2048 is the length L of the sample sentence. If the length of the sample exceeds L, the effective text length can be increased by removing commonly used auxiliary words and punctuation marks. Sentence truncation can also be performed if the length exceeds L.

[0029]

[0030] 2. Use new data collection to support the dataset.

[0031] Scenario 1: If a pre-prepared telecom fraud identification model is available, it can be used directly as the original model. When collecting supporting datasets, newly added risks collected externally (such as samples obtained through reports, spot checks, etc., and samples near the discrimination threshold) are manually verified against the original model one by one. For missed risk samples that the original model cannot identify, negative samples are collected; conversely, for normal samples that the original model incorrectly identifies, positive samples are collected. We denote the new dataset as S.

[0032] Scenario 2: If a prepared original model is unavailable, a basic model can be trained using the original dataset. The original dataset typically consists of accumulated violation data, with both positive and negative sample labels. It is generally very large and represents the foundational data for telecommunications companies. This differs from the supporting dataset for newly emerging risks. Supporting data consists of the latest risk data, which is very small in scale, usually appearing in the very early stages of a new risk, typically consisting of only a few or dozens of entries. This data is historically unprecedented and cannot be identified by the original model within the current day or a few days. In most cases, if new risks are not intercepted as quickly as possible, they will spread rapidly, causing the original model to quickly become ineffective.

[0033] If an original model is unavailable and needs to be built using the original dataset, the BERT model can be used to extract features from the original dataset, followed by binary classification training using a linear classifier. The linear classifier can employ a Cross-entropy + Sigmoid approach. label These are the actual labels of the samples, with a dimension of 2. It is the label vector calculated by the model, with dimensions and y. label Consistent. The cross-entropy loss function is: Loss = -y label Log(y) can be used to obtain the parameters (w0, b0) of the original model M0 using gradient descent. After obtaining the original model M0, as in Case 1, newly added risks collected externally (such as samples obtained through reports, spot checks, etc., and samples near the discrimination threshold) are manually verified one by one through the original model M0 (only the maximum dimension of y needs to be determined). For missed risk samples that the original model cannot identify, negative samples are collected; simultaneously, for normal samples that the original model misidentifies, positive samples are collected. This results in a new dataset S.

[0034] 3. Use a general large model to collect feature vectors supporting the dataset.

[0035] Newly emerging examples from the support dataset S that cannot be correctly identified are input one by one into the BERT main model, transformed into support dataset representation vectors, which serve as the basis for subsequent model optimization. Each text t in the support dataset... sThe transformed representation vector is Its dimensions and The resolution remains consistent at 768×2048, and the preprocessing methods are also consistent, including punctuation and auxiliary word filtering, as well as length truncation. Each supporting data point corresponds to one... Assuming I support data points are collected, a support data feature matrix of size I×768×2048 can be obtained.

[0036]

[0037] 4. Obtain the representation vectors of the supporting dataset and assemble the feature matrix.

[0038] For the converted Dimensionality reduction is necessary because we cannot collect massive amounts of samples. For a sample space of 768×2048, a few dozen supporting datasets are insufficient. Therefore, principal component analysis (PCA) is used to reduce the dimensionality to a describable range while maximizing discriminative power. The specific high-dimensional PCA calculation method is as follows. For tensors... Its dimensions are I×768×2048, and there are three modes for dimensionality reduction. We choose to perform dimensionality reduction in the first dimension.

[0039] First, the tensor Expand along the first dimension into a matrix. Here we will... Convert to an I×(768×2048) matrix i = 1, 2, ..., I, j = 1, 2, ..., J, k = 1, 2, ..., K, J = 768, K = 2048, which is the tensor. Each slice Flattened into a single row, χ is an I×(768×2048) matrix.

[0040] Then, calculate the covariance matrix C of χ. Here The dimension of C is (JK)×(JK). Matrix multiplication is being performed here.

[0041] Next, calculate the eigenvalues ​​λ and corresponding eigenvectors v of the covariance matrix C. Then, select the top n largest eigenvalues ​​λ. top_n and the corresponding feature vector v top_n V here top_n The dimension is (JK)×top_n. Here, the selection of top_n is based on the magnitude of each component of λ. Eigenvalues ​​with λ < 0.1 are discarded, and top_n is the number of remaining eigenvalues, which usually does not exceed 10.

[0042] Finally, the original sample features X and v are compared. top_nMultiplication yields the dimensionality-reduced feature vector χ. down The dimension of χ is I×(JK), where I is the number of samples in the supporting dataset, and χ is... down The dimension is I×top_n, which is reduced from JK to top_n.

[0043] 5. Train extended small models to adapt to new data risks.

[0044] Using the features χ obtained after dimensionality reduction from the support dataset down Given the corresponding label ψ, a small classifier M1 is trained with parameters (w1, b1). Here, Cross-entropy + softmax is used as the model structure, and the inner product in the neural network is replaced with cosine similarity calculation. The specific process is as follows. Predicted classification results:

[0045] y = Softmax(sim(w1, χ) down )+b1)

[0046] The cross-entropy loss function is Loss = -ψlog(y). Using gradient descent, the parameters corresponding to the original model M1 are (w1, b1). When determining whether a sample violates a rule, only the maximum dimension of y needs to be considered. This small neural network and its supporting dataset are further trained on the feature matrix to generate an expanded small model to identify new data risks. Because the amount of data is small, similarity calculation is used instead of dot product. After significant dimensionality reduction, the feature dimensions match those of the simple neural network model, avoiding overfitting.

[0047] 6. The overall process is divided into three parts: model training, model prediction, and model update.

[0048] (1) Model Training

[0049] For cases where an original discriminant model M0 already exists:

[0050] First, real-time data filtering is performed using the existing discriminant model M0. New data is extracted from the generated risk-free or suspected data to form the support dataset S (the method is the same as in step 2, case one). Then, the general-purpose open-source large model BERT is used to generate features for each data point in the support dataset.

[0051] (As in step 3), at this point, the feature dimension is high, but the number is small. Next, the features are dimensionality reduced (as in step 4). Then, a very small-scale extended model M1 is trained (as in step 5). The overall process is shown in Figure 1(a).

[0052] For cases where there is no original discriminant model M0:

[0053] First, the original discriminant model M0 needs to be trained using the massive amount of historically accumulated data (mainly through steps 1 and 2). Then, consistent with the previous approach, the trained discriminant model M0 is used for real-time data filtering, extracting new data from the generated risk-free or suspected data to form the support dataset S. Finally, the general-purpose open-source large model BERT is used to generate features for each data point in the support dataset. (As in step 3), at this point, the feature dimension is high, but the number is small. Next, the features are dimensionality reduced to obtain χ². down (As in step 4). Next, a very small-scale extended model M1 is trained (as in step 5). The overall process is shown in Figure 1(b).

[0054] (2) Model Prediction

[0055] For the already trained, very small-scale extended model M1 and the original discriminant model M0, in practical applications, the text segment to be detected is first input into the original BERT model to obtain the feature vector. (As in step 1). Then, the feature vector is input into the original discriminant model M0 to calculate... For samples where M0 indicates risk (e.g., y≥0.5), they are directly classified as risky. If the original model was not trained in step 2, the judgment is made directly based on the model output, without needing to calculate the y value. Next, for samples where M0 indicates no risk (e.g., y<0.5), the features... To reduce the dimensionality, With v top_n Multiplying reduces the dimension to top_n, resulting in the vector χ. down Then, the new feature vector is input into model M1, and y = Softmax(sim(w1, χ) is calculated. down For M1, which is judged as risk (y≥0.5), it is directly judged as a risky sample. The overall process is shown in Figure 2.

[0056] (3) Model update

[0057] As the supporting dataset gradually increases over time, model M1 can be updated according to the model training steps, while keeping model M0 unchanged. Because model M1 is small, its training cost is low. After several updates to model M1, model M0 can be updated synchronously according to the model training steps. For example, model M1 can be updated daily, and model M0 can be updated monthly. Model updates help maintain the robustness of the overall telecommunications anti-fraud system.

[0058] Example 2

[0059] This embodiment relates to a small-sample telecommunications anti-fraud device for rapidly mutating information, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they implement a small-sample telecommunications anti-fraud method for rapidly mutating information according to Embodiment 1.

[0060] Example 3

[0061] This embodiment relates to a computer-readable storage medium storing a program that, when executed by a processor, implements a small-sample telecommunications anti-fraud method for rapidly changing information as described in Embodiment 1.

[0062] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms described in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A small-sample telecommunications fraud prevention method for rapidly changing information, comprising the following steps: Step 1: Construct a feature extractor using a general large model; The BERT language model is used to generate features from the text; the original text information is converted into digital vector representations for subsequent fraud detection algorithm calculations. Step 2: Use new data collection to support the dataset; For samples with missed detection risks that the original model cannot identify, negative samples are collected; at the same time, for normal samples that the original model misidentifies, positive samples are collected. Step 3: Use a general large model to collect feature vectors supporting the dataset; The newly emerging examples in the support dataset that cannot be correctly identified are input one by one into the general large model and transformed into support dataset representation vectors, which serve as the basis for subsequent model optimization. Step 4: Obtain the representation vectors of the supporting dataset and assemble the feature matrix; The collected support data feature vectors are dimensionality reduced using principal component analysis to obtain a more concentrated support dataset feature matrix. Step 5: Train an expanded small model to adapt to new data risks; By using a small neural network and supporting dataset to further train on the feature matrix, an expanded small model is generated to identify new data risks.

2. The method for small-sample telecommunications fraud prevention targeting rapidly changing information as described in claim 1, characterized in that, Step 1 specifically includes: The original telecommunications text Replace with embedded features ;here Each Chinese character, after feature conversion, The dimension is the feature dimension of each Chinese character multiplied by the length of the sample sentence; the feature dimension of each Chinese character is 768, and the length of the sample sentence is... If the sample length exceeds By removing commonly used particles and punctuation marks, the effective text length can be increased, or sentences can be truncated. 。 3. The method for small-sample telecommunications fraud prevention targeting rapidly changing information as described in claim 1, characterized in that, Step 2 specifically includes: Scenario 1: If a telecom fraud identification model has already been prepared, it can be used directly as the original model. When collecting supporting datasets, each newly added risk collected from external sources is manually verified and checked through the original model. For missed risk samples that the original model cannot identify, negative samples are collected, while for normal samples that the original model misidentifies, positive samples are collected. The new dataset is denoted as S. Scenario 2: If there is no prepared original model, a basic model can be trained using the original dataset. The original dataset is usually an accumulation of violation data over time, with positive and negative sample labels. The original dataset is generally very large in scale and is the basic data of telecommunications units. Unlike the supporting dataset for newly emerging risks, the supporting data is the latest risk data, which is very small in scale. It is usually in the very early stage when a new risk just appears, usually consisting of a few or dozens of data points. The timeliness is that it is data that has never appeared in history within the same day or a few days, and the original model cannot identify it. Under normal circumstances, if new risks are not intercepted as quickly as possible, they will spread rapidly, causing the original model to quickly become ineffective. If an original model is unavailable and needs to be built using the original dataset, the BERT model is used to extract features from the original dataset, followed by binary classification training using a linear classifier. The linear classifier can be implemented using a Cross-entropy + Sigmoid approach. These are the actual labels of the samples, with a dimension of 2. It is the label vector calculated by the model, with dimensions and Consistent; the cross-entropy loss function is: The original model can be obtained using the gradient descent method. The corresponding parameters are ( ); to obtain the original model Then, as in scenario one, the newly identified risks collected from external sources can be processed one by one through the original model. Manual verification is performed; negative samples are collected for samples that the original model cannot identify, and positive samples are collected for normal samples that the original model misidentifies; thus, a new dataset S is obtained.

4. The method for small-sample telecommunications fraud prevention targeting rapidly changing information as described in claim 3, characterized in that, Step 3 specifically includes: Newly emerging examples from the support dataset S that cannot be correctly identified are input one by one into the BERT main model, transformed into support dataset representation vectors, which serve as the basis for subsequent model optimization; and as individual texts in the support dataset. The transformed representation vector is Its dimensions and To maintain consistency, the preprocessing methods are also consistent, including punctuation and auxiliary word filtering, and length truncation; each supporting data corresponds to one Assuming I support data points are collected, a support data feature matrix of size I×768×2048 can be obtained. ; 。 5. A small-sample telecommunications fraud prevention method for rapidly changing information as described in claim 4, characterized in that, Step 4 specifically includes: For the converted Principal component analysis (PCA) is employed to reduce the dimensionality to a describable range and maximize discriminative power. The specific calculation method for high-dimensional PCA is as follows; for tensors... Its dimensions are I×768×2048, and there are three modes for dimensionality reduction. We choose to perform dimensionality reduction in the first dimension. First, the tensor Expanding along the first dimension into a matrix; here we will Convert to an I×(768×2048) matrix = , , Tensor Each slice Flatten it into a single line, so It is an I×(768×2048) matrix; Then, calculate The covariance matrix C; here The dimension of C is (JK). (JK); This is where matrix multiplication is performed; Next, calculate the eigenvalues ​​of the covariance matrix C. With the corresponding feature vector ;Pick The largest Features and the corresponding feature vectors ,here The dimension is ;here The choice is based on The magnitude of each component, for Eigenvalues ​​with a value less than 0.1 are discarded, and top_n is the number of remaining eigenvalues; Finally, the original sample features and Multiplication yields the dimensionality-reduced feature vector. ; The dimension is I×(JK), where I is the number of samples in the supporting dataset. The dimension is I× From JK to dimensionality reduction .

6. The method for small-sample telecommunications fraud prevention targeting rapidly changing information as described in claim 1, characterized in that, Step 5 specifically includes: Features obtained by dimensionality reduction from the supporting dataset and corresponding tags Train a small classifier The corresponding parameters are ( Here, Cross-entropy + softmax is used as the model structure, and the inner product in the neural network is replaced with cosine similarity calculation; the specific process is as follows, sim( )= Predicted classification results: Cross-entropy loss function: The original model can be obtained using the gradient descent method. The corresponding parameters are ( When determining whether a sample violates regulations, it is only necessary to judge... The maximum dimension is sufficient; this small neural network and the supporting dataset are further trained on the feature matrix to generate an extended small model to identify new data risks; because the amount of data here is very small, similarity calculation is used instead of dot product inner product. After a significant dimensionality reduction, the feature dimension can match that of a simple neural network model, and there will be no overfitting problem.

7. The method for small-sample telecommunications fraud prevention targeting rapidly changing information as described in claim 1, characterized in that, It is divided into three parts: model training, model prediction, and model update. (1) Model training For those who already have an original discriminant model The situation is as follows: First, using existing discriminative models Real-time data filtering is performed, and new data is extracted from the generated risk-free or suspected data to form a supporting dataset S; Then, the general-purpose open-source large model BERT is used to generate features for each data point in the supporting dataset. At this point, the feature dimension is high, but the number of features is small; next, the features are dimensionality reduced; then, a very small-scale extended model is trained. ; For those without an original discriminant model The situation is as follows: First, it is necessary to train the original discriminant model using the massive amount of historically accumulated data. Then, just like in the previous case, the pre-trained discriminant model is used. Real-time data filtering is performed, and new data is extracted from the generated risk-free or suspected data to form the supporting dataset S. Then, the general-purpose open-source large model BERT is used to generate features for each data point in the supporting dataset. At this point, the feature dimension is high, but the number of features is small; next, the features are dimensionality reduced to obtain... Next, a very small-scale extended model is trained. ; (2) Model prediction For already trained, very small-scale extended models and the original discriminant model In practical applications, the text segment to be detected is first input into the original BERT model to obtain the feature vector. Then, the feature vectors are input into the original discriminant model. In the middle, calculation ,for Instances judged as risky are directly identified as risky examples. For cases where the original model was not trained in step 2, the judgment is made directly based on the model output without further computation. Value; then, for Examples that are determined not to be risky are identified by their features. To reduce dimensionality, and Multiplication reduces the dimension to the top_n, resulting in a vector. ; Then, the new feature vector is input into the model. ,calculate ,for The direct determination of a case as risky is a risk sample; (3) Model update For situations where the supporting dataset gradually increases over time, the model can be... Update the model according to the training steps and maintain the model. Unchanged; because the model The scale is small, and its training cost is low; after multiple model iterations... After the update, the model can be updated synchronously once according to the model training steps. For example, it can preserve the model. The update frequency is daily, and the model... It is updated monthly; model updates help maintain the robustness of the overall telecom fraud prevention system.

8. A small-sample telecommunications fraud prevention method for rapidly changing information as described in claim 3, characterized in that, The newly added risks collected externally include reports, samples obtained through random sampling, and samples near the discrimination threshold.

9. A small-sample telecommunications fraud prevention method for rapidly changing information as described in claim 5, characterized in that, The manual verification check only needs to determine The maximum dimension, where top_n is the number of remaining feature values, not exceeding 10.

10. A small-sample telecommunications anti-fraud device for rapidly changing information, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a small-sample telecommunications anti-fraud method for rapidly changing information as described in any one of claims 1-9.

Images