Abnormal sample labeling method and device

Abstract

The invention discloses an abnormal sample labeling method and device. The method comprises the following steps: acquiring to-be-labeled sample data; multiple models in a pre-trained labeling model are adopted to label the to-be-labeled sample data, a target labeling result is jointly determined according to labeling results output by the multiple models, the labeling model comprises the multiple models, and the labeling result is determined according to the labeling results output by the multiple models. The plurality of models at least comprise one of the following models: a first model used for labeling the to-be-labeled sample data according to the text type sample features, a second model used for labeling the to-be-labeled sample data according to the time type sample features, and a third model used for labeling the to-be-labeled sample data according to the coding type sample features; and outputting a target labeling result.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to an anomaly sample labeling method and apparatus. Background Technology

[0002] As the number of telecommunications users continues to expand, more and more users are being harassed by abnormal messages. In recent years, telecommunications operators have gradually begun to use neural network models to monitor abnormal messages, achieving good results. However, as abnormal messages become increasingly complex and rapidly changing, their characteristics have become scattered and diverse. Therefore, deep learning-based abnormal message identification models have been introduced. However, in practical applications, there is a vast amount of data on different types of abnormal messages. Relying solely on the resources of operators, they can only obtain accurate, high-quality, small-batch sample data for certain scenarios. This leads to a scarcity of high-quality labeled data and high costs of manual annotation during the training of deep learning models for different domain types, resulting in low accuracy in identifying abnormal messages. Summary of the Invention

[0003] This application provides an abnormal sample labeling method and apparatus to at least solve the technical problem in the related art that the accuracy of abnormal information identification is low due to the small number of samples.

[0004] According to one aspect of the embodiments of this application, an anomaly sample annotation method is provided, comprising: acquiring sample data to be annotated; annotating the sample data to be annotated using multiple models in a pre-trained annotation model, and jointly determining a target annotation result based on the annotation results output by the multiple models, wherein the annotation model includes multiple models, and the multiple models include at least one of the following: a first model for annotating the sample data to be annotated based on text-type sample features, a second model for annotating the sample data to be annotated based on time-type sample features, a third model for annotating the sample data to be annotated based on coding-type sample features; and outputting the target annotation result.

[0005] Optionally, the annotation model is determined by: constructing a base model, which includes: the base model of the first model, the base model of the second model, and the base model of the third model; obtaining a training dataset, which includes: a first dataset, a second dataset, and a third dataset, wherein the first dataset includes: text samples, the second dataset includes: time samples, and the third dataset includes: encoded samples, the text samples include: abnormal text content, the time samples include: number activation time and call initiation period, and the encoded samples include: the age of the number owner, the location of the number, and the number of numbers under the name of the number owner; and training the base models of the first model, the second model, and the third model using the first dataset, the second dataset, and the third dataset respectively to obtain the annotation model.

[0006] Optionally, constructing a base model includes: constructing a base model for the first model, a base model for the second model, and a base model for the third model, respectively. The base model for the first model includes, in sequence, an input layer, a convolutional layer, a max pooling layer, a fully connected layer, and a classification layer. The base model for the second model includes a bidirectional long short-term memory model. The base model for the third model includes a one-dimensional wide-kernel convolutional neural network.

[0007] Optionally, the sample data to be labeled is labeled using a pre-trained labeling model to obtain target labeling results, including: using the first model, the second model, and the third model in the labeling model to predict the type of the sample data to be labeled, respectively, to obtain a first prediction result, a second prediction result, and a third prediction result; determining the sample data in the sample data to be labeled that have the same first prediction result, second prediction result, and third prediction result as successfully labeled target sample data; and determining the prediction result of the target sample data as the labeling result of the target sample data.

[0008] Optionally, the method further includes: receiving a word vector matrix using the input layer of the first model, and performing multiple convolution operations on the word vector matrix using the convolutional layer of the first model to obtain multiple feature maps, wherein the word vector matrix is ​​obtained by preprocessing the sample data to be labeled; selecting the feature vector with the largest mid-value among the multiple feature maps using the max pooling layer of the first model to determine it as the target feature vector; fusing the multiple target feature vectors using the fully connected layer of the first model to obtain a fused vector; and making a prediction based on the fused vector using the classification layer of the first model to obtain the first prediction result.

[0009] Optionally, the method further includes: using the bidirectional long short-term memory model in the second model to extract features from the sample data to be labeled, obtaining bidirectional features; using an attention mechanism to perform attention weighting on the bidirectional features, obtaining a feature vector; and using the classification layer in the second model to make a prediction based on the feature vector, obtaining the second prediction result.

[0010] Optionally, the method further includes: storing the target sample data into different datasets according to the type of sample data, thereby expanding the training dataset, wherein the types of sample data include: text sample data, time sample data, and encoded sample data.

[0011] According to another aspect of the embodiments of this application, an anomaly sample labeling device is also provided, comprising: an acquisition module for acquiring sample data to be labeled; a labeling module for labeling the sample data to be labeled using multiple models in a pre-trained labeling model, and jointly determining a target labeling result based on the labeling results output by the multiple models, wherein the labeling model includes multiple models, and the multiple models include at least one of the following: a first model for labeling the sample data to be labeled based on text-type sample features, a second model for labeling the sample data to be labeled based on time-type sample features, and a third model for labeling the sample data to be labeled based on coding-type sample features; and an output module for outputting the target labeling result.

[0012] According to another aspect of the embodiments of this application, a computer device is also provided, including: a memory and a processor, wherein the memory is used to store program instructions; and the processor, connected to the memory, is used to execute the above-described abnormal sample labeling method.

[0013] According to another aspect of the embodiments of this application, a computer program product is also provided, including computer instructions that, when executed by a processor, implement the above-described abnormal sample labeling method.

[0014] In this embodiment, sample data to be labeled is acquired; multiple models in a pre-trained labeling model are used to label the sample data to be labeled, and the target labeling result is determined based on the labeling results output by the multiple models. The labeling model includes multiple models, at least one of the following: a first model for labeling the sample data to be labeled based on text-type sample features, a second model for labeling the sample data to be labeled based on time-type sample features, and a third model for labeling the sample data to be labeled based on coding-type sample features; the target labeling result is output. By having multiple models in the labeling model label the sample data to be labeled according to different types of features, the purpose of quickly and accurately labeling the sample data is achieved, resulting in a rapid increase in the number of sample data. This solves the technical problem in related technologies where the accuracy of anomaly identification is low due to a small number of samples. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0016] Figure 1 This is a hardware structure block diagram of a computer terminal for implementing an anomaly sample annotation method according to an embodiment of this application;

[0017] Figure 2 This is a flowchart of an abnormal sample annotation method according to an embodiment of this application;

[0018] Figure 3 This is a flowchart illustrating the construction of an anomaly sample annotation model according to an embodiment of this application;

[0019] Figure 4 This is a structural diagram of an abnormal sample labeling device according to an embodiment of this application. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] The information collected in this application embodiment is information and data authorized by the user or fully authorized by all parties. The collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant regions, and necessary confidentiality measures have been taken. It does not violate public order and good morals, and provides corresponding operation entry points for users to choose to authorize or reject the automated decision results. If the user chooses to reject, the process will proceed to the expert decision-making process.

[0023] To address the problems existing in related technologies, this application provides an anomaly sample annotation method, which can be run on... Figure 1 The computer terminal shown is explained below.

[0024] The abnormal sample labeling method provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal for implementing an anomaly sample annotation method is shown. Figure 1 As shown, the computer terminal 10 may include one or more processors (shown as 102a, 102b, ..., 102n in the figure) (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions connected via wired and / or wireless networks. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0025] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10. As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0026] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the anomaly sample labeling method in this embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the aforementioned anomaly sample labeling method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0027] The transmission module 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission module 106 may be a radio frequency (RF) module, used for wireless communication with the Internet.

[0028] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10.

[0029] It should be noted here that, in some optional embodiments, the above... Figure 1 The computer terminal shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1This is only one instance of a specific particular instance, and is intended to illustrate the types of components that may exist in the aforementioned computer terminal.

[0030] In the above operating environment, this application provides an embodiment of an anomaly sample labeling method. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than that shown here.

[0031] Figure 2 This is a flowchart of an abnormal sample annotation method according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0032] Step S202: Obtain the sample data to be labeled;

[0033] Step S204: The sample data to be labeled is labeled using multiple models in the pre-trained labeling model, and the target labeling result is determined jointly based on the labeling results output by the multiple models. The labeling model includes multiple models, and the multiple models include at least one of the following: a first model for labeling the sample data to be labeled based on text-type sample features, a second model for labeling the sample data to be labeled based on time-type sample features, and a third model for labeling the sample data to be labeled based on coding-type sample features.

[0034] Step S206: Output the target annotation results.

[0035] Through steps S202 to S206 above, sample data to be labeled is obtained; multiple models in a pre-trained labeling model are used to label the sample data to be labeled, and the target labeling result is determined based on the labeling results output by the multiple models. The labeling model includes multiple models, at least one of the following: a first model for labeling the sample data to be labeled based on text-type sample features, a second model for labeling the sample data to be labeled based on time-type sample features, and a third model for labeling the sample data to be labeled based on coding-type sample features; the target labeling result is output. By having multiple models in the labeling model label the sample data to be labeled according to different types of features, the purpose of quickly and accurately labeling the sample data is achieved, resulting in a rapid increase in the number of sample data. This solves the technical problem in related technologies where the accuracy of anomaly identification is low due to a small number of samples. A detailed explanation follows.

[0036] In some embodiments of this application, such as Figure 3 As shown, the annotation model is determined in the following way: A base model is constructed, which includes: the base model of the first model, the base model of the second model, and the base model of the third model; a training dataset is obtained, which includes: a first dataset, a second dataset, and a third dataset. The first dataset includes: text samples; the second dataset includes: time samples; and the third dataset includes: encoded samples. The text samples include: abnormal text content; the time samples include: number activation time and call initiation period; and the encoded samples include: the age of the number owner, the location of the number, and the number of numbers under the name of the number owner; the base models of the first model, the second model, and the third model are trained using the first dataset, the second dataset, and the third dataset, respectively, to obtain the annotation model.

[0037] Specifically, based on the data type of the sample data and the different hierarchical feature relationships, the data information sample of an abnormal information, such as an abnormal text message or abnormal call received by a mobile phone, is divided into three dimensions: text sample data, time sample data, and encoded sample data.

[0038] Abnormal content and scripts, as text-based sample data, are the most direct anomaly analysis data and exhibit clear logical connections. SIM card activation time and call initiation time, as time-based sample data, are understandably targeted at the elderly, women, and children in real-world applications. These calls are typically scheduled within specific time windows, such as during normal working hours when the elderly or women are unaccompanied, or on non-holidays. This type of data, when statistically analyzed periodically, shows a clear long-term time-series dependency. SIM card owner's age, location, and number of SIM cards under their name, as coded sample data, represent standardized data features that exhibit characteristics of abnormal users, and these features are correlated.

[0039] In some embodiments of this application, the specific steps for constructing the base model include: constructing the base model of the first model, the base model of the second model, and the base model of the third model respectively. The base model of the first model includes, in sequence, an input layer, a convolutional layer, a max pooling layer, a fully connected layer, and a classification layer. The base model of the second model includes a bidirectional long short-term memory model. The base model of the third model includes a one-dimensional wide-kernel convolutional neural network.

[0040] Specifically, for text-based sample data, the TextCNN (Text Convolutional Neural Network) model is selected as the base model. TextCNN utilizes multiple kernel sizes to extract key information from sentences for natural language analysis. The model consists of an input layer, convolutional layers, max-pooling layers, fully connected layers, and a classification layer. The specific model structure is as follows:

[0041] 1) Input layer: Receives the word vector matrix wj obtained by preprocessing the original word embedding matrix using the CBoW algorithm;

[0042] 2) Convolutional layer: Traverse different convolutional kernels (kernels with widths of 1, 3, and 5 respectively), perform 3 convolution operations to obtain feature map a:

[0043] in,

[0044]

[0045] In the formula, f represents the activation function, k is the weight matrix, wj represents the word vector matrix of the j-th word, and b represents the bias value. Let j represent the j-th feature map.

[0046] 3) Max Pooling Layer: Select the vector Vmax with the largest value in the output feature map of the convolutional layer, i.e.:

[0047]

[0048] It's understandable that max pooling layers are used to select the most salient features from the feature map. There are various ways to select the most salient features. For example, max pooling layers select the maximum value of each pooling window in the convolutional layer's output feature map as the output of that window. These maximum values ​​are usually considered the most salient features extracted by that layer because they represent the most prominent information within the local region. Feature importance evaluation: In models such as decision trees, random forests, or gradient boosting trees, feature importance scores can be used to determine which features have the greatest impact on the model's prediction results. These features with high importance scores are considered the "most salient features." Attention mechanism: In sequence models such as LSTM and Transformer, the attention mechanism assigns weights to each input element. Features with high weights are considered the "most salient features" that are most explanatory to the current task or output. Feature selection algorithms: Algorithms such as those based on variance, mutual information, and recursive feature elimination (RFE) are used to filter out a set of features from a large number of features that are most explanatory to the target variable; these features are the so-called "most salient features."

[0049] 4) Fully connected layer: It accepts the output vector of the max pooling layer and maps the features learned by the previous few layers to the sample label space; in addition, in order to prevent the model from overfitting and speed up the training efficiency, the Dropout mechanism is used to discard some neurons.

[0050] 5) Classification layer: The Softmax function is used to normalize the results obtained from the fully connected layer to obtain the prediction results.

[0051] For time-related sample data, a Bi-LSTM-based model was selected as the base model. Bi-LSTM is highly sensitive to time series features and can acquire both positive and negative information features. The model's feature extraction part consists of a two-layer Bi-LSTM network, the feature optimization part consists of an attention mechanism layer, and the classification and recognition part consists of two fully connected layers and one classification layer.

[0052] The Bi-LSTM network consists of two independent LSTM networks. Data enters each LSTM network through a forward and a backward path, respectively. The two LSTM networks then output the extracted forward and backward features together. The information processing flow of the short-term memory network is as follows: first, using the external state ht-1 from the previous time step and the input from the current time step, the input gate, forget gate, output gate, and candidate states are calculated.

[0053] The hidden layer features obtained after transformation by Bi-LSTM layers are as follows:

[0054]

[0055] In the formula, This represents the feature of the nth hidden layer.

[0056] An attention mechanism is introduced to filter and optimize the features extracted by the Bi-LSTM network, highlighting the contribution and importance of key features to the fault identification and classification results. The specific construction process of the attention mechanism for adaptive dynamic feature weighting fusion is as follows:

[0057] R

[0058]

[0059]

[0060] In the formula, R is the attention scoring function, ∂i represents the attention weight coefficient, f() represents the activation function, m is the feature representation obtained after weighted fusion, di represents the hidden layer state corresponding to the input sequence, c represents the bias value, and Ws represents the weight matrix of the variables. This represents the sum of attention scores for all time steps. This represents the attention score at the i-th time step.

[0061] The fully connected layer with the Tanh activation function receives the output feature vector m from the attention layer, and the forward propagation formula is as follows:

[0062]

[0063] In the formula, m represents the output of the attention layer, N represents the output of the fully connected layer, and g represents the activation function.

[0064] After the fully connected layer performs feature space transformation through weighted computation, the classification layer, with the softmax activation function as the input, outputs the predicted target category. :

[0065]

[0066] For coded sample data, a one-dimensional wide-kernel convolutional neural network (CNN) is used as the base model. Coded data has a one-dimensional characteristic, and a one-dimensional convolutional network can extract features more effectively and has better interpretability. The overall architecture of the wide-kernel one-dimensional convolutional network is similar to that of the CNN model and will not be described here. The main difference is that in the filtering stage, the first convolutional kernel of the one-dimensional wide-kernel CNN is wider, while subsequent convolutional kernels are smaller. Large convolutional kernels can more effectively extract short-term features, playing a role similar to the short-time Fourier transform. Multiple layers of small convolutional kernels increase the network depth, which helps to obtain a good representation of the input features and improve performance.

[0067] In some embodiments of this application, the specific steps of using a pre-trained annotation model to annotate the sample data to be annotated and obtaining the target annotation result include: using the first model, the second model, and the third model in the annotation model to predict the type of the sample data to be annotated, respectively, to obtain a first prediction result, a second prediction result, and a third prediction result; determining the sample data in the sample data to be annotated that has the same first prediction result, second prediction result, and third prediction result as the successfully annotated target sample data; and determining the prediction result of the target sample data as the annotation result of the target sample data.

[0068] The three models perform the following recognition processes on the input sample data: The first model uses an input layer to receive a word vector matrix, and a convolutional layer in the first model performs multiple convolution operations on the word vector matrix to obtain multiple feature maps. The word vector matrix is ​​obtained by preprocessing the sample data to be labeled. A max-pooling layer in the first model selects the feature vector with the largest mid-value among the multiple feature maps as the target feature vector. A fully connected layer in the first model fuses the multiple target feature vectors to obtain a fused vector. A classification layer in the first model makes a prediction based on the fused vector to obtain the first prediction result. The second model uses a bidirectional long short-term memory model to extract features from the sample data to be labeled, obtaining bidirectional features. An attention mechanism is used to weight the bidirectional features to obtain a feature vector. A classification layer in the second model makes a prediction based on the feature vector to obtain the second prediction result.

[0069] Based on the information dimensions of three types of sample data, three characteristic initial training sets were generated from the existing small amount of labeled data. These sets were used to train three base classifiers: TextCNN (the first model), BILSTRM (the second model), and a one-dimensional wide-kernel convolutional neural network (the third model). TextCNN is responsible for extracting core semantic features of the text, BILSTRM focuses on extracting temporal features, and the one-dimensional wide-kernel convolutional neural network deeply mines one-dimensional features of the encoding class. By using differentiated training sets, the three base models learn and extract multi-level fraud-related features from different dimensions. Subsequently, these models predict and label samples in the unlabeled dataset. Data with consistent labeling (consistent review) and their labels were obtained and included in different training libraries according to three types: text-based sample data, temporal feature-based sample data, and encoding-based sample data. These data were continuously used to iterate, expand, and update the training sets of each base model. The training sets were then used for continuous professional training and optimization of the three base models. This process continuously improved the quality and scale of pseudo-labels, automatically generating a large amount of reliable labeled data until the model performance reached convergence.

[0070] In the process of classifying the labeled sample data, the target sample data is stored in different datasets according to the type of sample data to expand the training dataset. The types of sample data include: text sample data, time sample data, and encoded sample data.

[0071] To better illustrate the anomaly sample annotation method proposed in this application, another specific embodiment is provided below for further explanation. Step 1, Data Type Classification: Text-based sample data: includes abnormal content and dialogue, possessing logical correlation. Time-related sample data: such as SIM card activation time, call initiation time, etc., exhibiting periodicity and long-term dependency characteristics. Encoded sample data: includes SIM card owner's age, location, number of SIM cards under their name, etc., possessing standardized characteristics of abnormal users. Collect and organize the above three types of data to form a preliminary sample set. Step 2, Basic Model Selection and Construction: a. Text-based model: using TextCNN, utilizing convolutional kernels of different sizes to extract key information. The input layer receives the preprocessed word vector matrix. The convolutional layer uses convolutional kernels of different sizes for feature extraction. The max pooling layer selects the most salient features. The fully connected layer performs feature fusion, applying Dropout to prevent overfitting. The classification layer uses the Softmax function for category prediction. b. Time-related model: using Bi-LSTM, focusing on time series features. The Bi-LSTM layer extracts forward and reverse time characteristics. The process involves several steps: 1. **Introducing an attention mechanism to optimize features and highlight key information.** 2. **Multi-layer fully connected layers for feature space transformation.** 3. **Softmax classification layer outputs the predicted category.** The coding model uses a one-dimensional wide-kernel convolutional neural network, suitable for one-dimensional feature extraction. Wide convolutional kernels extract short-term features, followed by smaller convolutional kernels to deepen the network.** Step 3: **Model Training Initialization:** Using a small amount of high-quality labeled data, three initial training sets with different characteristics are generated for the basic training of their respective models. Step 4: **Semi-supervised learning annotation implementation:** a. **Differentiated training:** Each model performs prediction annotations on unlabeled data. b. **Consistent review:** Samples with consistent annotation results from the three models are selected. These samples and their labels are categorized into their respective training libraries for subsequent training set expansion. c. **Iterative optimization:** The expanded training sets are used to iteratively train and fine-tune the three base models. The quality of pseudo-labels is continuously improved, training samples are expanded, and model convergence is accelerated. d. **Result evaluation:** The model performance is monitored to ensure that it meets expectations or converges, ensuring annotation quality. Step 5: **Model integration and application:** The three trained models each possess high prediction accuracy in different dimensions. By integrating the three models, a complete automatic annotation system for anomaly information samples is constructed.

[0072] Through the above steps, this automatic annotation method for abnormal information samples can make full use of the limited amount of existing high-quality annotation data. By adopting the ideas of semi-supervised learning and multi-view co-training, it can automatically generate a large amount of reliable annotation data, thereby effectively improving the performance of professional anti-fraud models for telecommunications network fraud detection, reducing labor costs, and enhancing the system's generalization ability and real-time response capability.

[0073] Figure 4An anomalous sample labeling device is shown, the device comprising:

[0074] Module 40 is used to acquire sample data to be labeled;

[0075] The annotation module 42 is used to annotate the sample data to be annotated using multiple models in a pre-trained annotation model, and to jointly determine the target annotation result based on the annotation results output by the multiple models. The annotation model includes multiple models, and the multiple models include at least one of the following: a first model for annotating the sample data to be annotated based on text-type sample features, a second model for annotating the sample data to be annotated based on time-type sample features, and a third model for annotating the sample data to be annotated based on coding-type sample features.

[0076] Output module 44 is used to output the target annotation results.

[0077] The aforementioned anomaly sample labeling device acquires sample data to be labeled; uses multiple models in a pre-trained labeling model to label the sample data, and jointly determines the target labeling result based on the labeling results output by the multiple models. The labeling model includes multiple models, at least one of the following: a first model for labeling the sample data based on text-type sample features, a second model for labeling the sample data based on time-type sample features, and a third model for labeling the sample data based on coding-type sample features; and outputs the target labeling result. By using multiple models in the labeling model to label the sample data based on different types of features, the device achieves the goal of quickly and accurately labeling the sample data, rapidly increasing the sample data volume, and thus solving the technical problem in related technologies where the accuracy of anomaly information identification is low due to a small number of samples.

[0078] The annotation module 42 includes a training submodule, used to determine the annotation model in the following ways: constructing a base model, which includes: the base model of the first model, the base model of the second model, and the base model of the third model; obtaining a training dataset, wherein the training dataset includes: a first dataset, a second dataset, and a third dataset, the first dataset including: text samples, the second dataset including: time samples, and the third dataset including: encoded samples, the text samples including: abnormal text content, the time samples including: number activation time and call initiation period, and the encoded samples including: the age of the number owner, the location of the number, and the number of numbers under the name of the number owner; training the base models of the first model, the second model, and the third model using the first dataset, the second dataset, and the third dataset respectively to obtain the annotation model.

[0079] The training submodule includes a construction unit for building a base model, which includes building the base model of the first model, the base model of the second model, and the base model of the third model respectively. The base model of the first model includes, in sequence, an input layer, a convolutional layer, a max pooling layer, a fully connected layer, and a classification layer. The base model of the second model includes a bidirectional long short-term memory model. The base model of the third model includes a one-dimensional wide-kernel convolutional neural network.

[0080] The annotation module 42 further includes an annotation submodule, used to annotate the sample data to be annotated using a pre-trained annotation model to obtain target annotation results, including: using the first model, the second model and the third model in the annotation model to predict the type of the sample data to be annotated, respectively, to obtain a first prediction result, a second prediction result and a third prediction result; determining the sample data in the sample data to be annotated that have the same first prediction result, second prediction result and third prediction result as successfully annotated target sample data; and determining the prediction result of the target sample data as the annotation result of the target sample data.

[0081] The annotation submodule includes a first annotation unit, a second annotation unit, and an expansion unit. The first annotation unit receives a word vector matrix using the input layer of the first model and performs multiple convolution operations on the word vector matrix using the convolutional layer of the first model to obtain multiple feature maps. The word vector matrix is ​​obtained by preprocessing the sample data to be annotated. The first unit uses a max-pooling layer in the first model to select the feature vector with the largest mid-value among the multiple feature maps as the target feature vector. The first unit uses a fully connected layer in the first model to fuse the multiple target feature vectors to obtain a fused vector. Finally, the first unit uses a classification layer in the first model to make a prediction based on the fused vector to obtain the first prediction result.

[0082] The second labeling unit is used to extract features from the sample data to be labeled using the bidirectional long short-term memory model in the second model to obtain bidirectional features; to apply attention weighting to the bidirectional features using an attention mechanism to obtain a feature vector; and to make a prediction based on the feature vector using the classification layer in the second model to obtain the second prediction result.

[0083] An expansion unit is used to store the target sample data into different datasets according to the type of sample data, thereby expanding the training dataset. The types of sample data include: text sample data, time sample data, and encoded sample data.

[0084] It should be noted that, Figure 4 The abnormal sample labeling device shown is used to perform Figure 2 The above explanations regarding the abnormal sample labeling method also apply to this abnormal sample labeling device, and will not be repeated here.

[0085] This application also provides a computer device, including: a memory and a processor, wherein the memory is used to store program instructions; and the processor, connected to the memory, is used to execute the above-described abnormal sample labeling method.

[0086] The anomaly sample labeling method executed by the aforementioned computer equipment involves: acquiring sample data to be labeled; labeling the sample data using multiple models in a pre-trained labeling model; and jointly determining the target labeling result based on the labeling results output by the multiple models. The labeling model includes multiple models, at least one of the following: a first model for labeling the sample data based on text-type sample features; a second model for labeling the sample data based on time-type sample features; and a third model for labeling the sample data based on coding-type sample features. The target labeling result is then output. By using multiple models in the labeling model to label the sample data based on different types of features, the method achieves the goal of quickly and accurately labeling the sample data, rapidly increasing the amount of sample data, and thus solving the technical problem in related technologies where the accuracy of anomaly information identification is low due to a small number of samples.

[0087] This application also provides a computer program product, including computer instructions that, when executed by a processor, implement the steps of the abnormal sample labeling method in this application.

[0088] The aforementioned computer program product's method for annotating abnormal samples involves: acquiring sample data to be labeled; using multiple models in a pre-trained annotation model to annotate the sample data; and jointly determining the target annotation result based on the annotation results output by the multiple models. The annotation model includes multiple models, at least one of the following: a first model for annotating the sample data based on text-type sample features; a second model for annotating the sample data based on time-type sample features; and a third model for annotating the sample data based on coding-type sample features. The target annotation result is then output. By having multiple models in the annotation model annotate the sample data according to different types of features, the method achieves the goal of quickly and accurately annotating the sample data, rapidly increasing the amount of sample data, and thus solving the technical problem in related technologies where the accuracy of abnormal information identification is low due to a small number of samples.

[0089] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0090] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0092] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0093] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0094] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0095] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An abnormal sample labeling method, characterized in that, The method comprises the following steps: obtaining sample data to be labeled; using a plurality of models in a pre-trained labeling model to label the sample data to be labeled, and determining a target labeling result based on the labeling results output by the plurality of models, wherein the labeling model comprises a plurality of models, and the plurality of models at least include one of the following: a first model for labeling the sample data to be labeled according to text type sample characteristics, a second model for labeling the sample data to be labeled according to time type sample characteristics, and a third model for labeling the sample data to be labeled according to coding type sample characteristics; outputting the target labeling result.

2. The method of claim 1, wherein, The labeling model is determined by the following method: constructing a basic model, wherein the basic model comprises a basic model of the first model, a basic model of the second model, and a basic model of the third model; obtaining a training data set, wherein the training data set comprises a first data set, a second data set, and a third data set, the first data set comprises text type samples, the second data set comprises time type samples, and the third data set comprises coding type samples, the text type samples comprise abnormal text content, the time type samples comprise number activation time and call initiation period, and the coding type samples comprise the age of the number owner, the number ownership, and the number of all numbers under the name of the number owner; training the basic model of the first model, the basic model of the second model, and the basic model of the third model using the first data set, the second data set, and the third data set respectively to obtain the labeling model.

3. The method of claim 2, wherein, Constructing a basic model comprises: respectively constructing the basic model of the first model, the basic model of the second model, and the basic model of the third model, wherein the basic model of the first model comprises an input layer, a convolution layer, a max-pooling layer, a full connection layer, and a classification layer in sequence, the basic model of the second model comprises a bidirectional long short-term memory model, and the basic model of the third model comprises a one-dimensional wide kernel convolutional neural network.

4. The method of claim 1, wherein, Using the pre-trained labeling model to label the sample data to be labeled to obtain a target labeling result comprises: respectively using the first model, the second model, and the third model in the labeling model to predict the type of the sample data to be labeled to obtain a first prediction result, a second prediction result, and a third prediction result respectively; determining sample data in which the first prediction result, the second prediction result, and the third prediction result in the sample data to be labeled are consistent as target sample data with successful labeling; determining the prediction result of the target sample data as the labeling result of the target sample data.

5. The method of claim 4, wherein, The method further comprises: receiving a word vector matrix using the input layer in the first model, and performing a plurality of convolution operations on the word vector matrix using the convolution layer in the first model to obtain a plurality of feature maps, wherein the word vector matrix is obtained by preprocessing the sample data to be labeled; Adopting the maximum pooling layer in the first model selects the value maximum feature vector in the plurality of feature maps as a target feature vector; Adopting the full connection layer in the first model fuses a plurality of the target feature vectors to obtain a fusion vector; Adopting the classification layer in the first model predicts according to the fusion vector to obtain the first prediction result.

6. The method of claim 4, wherein, The method further comprises: Adopting the bidirectional long short-term memory model in the second model to extract features of the sample data to be labeled to obtain bidirectional features; Adopting an attention mechanism to perform attention weighting on the bidirectional features to obtain a feature vector; Adopting the classification layer in the second model to predict according to the feature vector to obtain the second prediction result.

7. The method of claim 4, wherein, The method further comprises: Storing the target sample data according to the types of sample data into different data sets to expand the training data set, wherein the types of sample data include text sample data, time sample data, and code sample data. 8.An abnormal sample labeling apparatus, characterized by comprising: Comprise: An acquisition module is configured to acquire sample data to be labeled; A labeling module is configured to label the sample data to be labeled by using a plurality of models in a pre-trained labeling model, and to determine a target labeling result according to labeling results output by the plurality of models, wherein the labeling model comprises a plurality of models, and the plurality of models at least include one of the following: a first model configured to label the sample data to be labeled according to text sample features, a second model configured to label the sample data to be labeled according to time sample features, and a third model configured to label the sample data to be labeled according to code sample features; An output module is configured to output the target labeling result.

9. A computer device, comprising: Comprise: A memory and a processor, wherein the memory is configured to store program instructions; The processor is connected with the memory and is configured to execute the abnormal sample labeling method in any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that, The computer instructions are executed by the processor to implement the abnormal sample labeling method in any one of claims 1 to 7.

Images