Sample data screening method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of machine learning, and discloses a sample data screening method and device, computer equipment and a storage medium. The method comprises the steps of obtaining multiple pieces of first text data, and inputting the first text data into a to-be-trained model to obtain a first semantic code and first probability distribution; according to the current corpus and the first semantic code, matching the first text data to obtain a plurality of second text data with similar semantics, and determining to obtain a plurality of second annotation data; calculating a probability difference between the first text data and the plurality of second text data according to the first probability distribution and the plurality of second annotation data to obtain a value score of the first text data; and processing the first text data according to the value score of the first text data to obtain sample data. The probability difference is used for screening, so that the sample data beneficial to training the to-be-trained model can be obtained more efficiently and conveniently.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a method, device, computer equipment and storage medium for screening sample data. Background Art

[0002] Natural Language Processing (NLP) technology is being widely used in financial services, retail, smart healthcare and other fields. In human-computer interaction scenarios such as voice intelligent agents and chat box AI customer service, financial institutions such as securities and banks have widely adopted Natural Language Understanding (NLU) models for semantic recognition, so as to more efficiently and accurately determine customer intentions and provide more convenient, smooth and more in line with needs. However, since natural language understanding models are often pre-trained based on specific corpora, and it is difficult for any corpus to cover all situations in the real world, the applicability and accuracy of the model obtained through pre-training alone are relatively limited, and further training is required using data generated in the business.

[0003] However, the data generated in actual business is often large in scale, and the data that the model actually predicts incorrectly is often rare. In order to screen sample data that is conducive to further training of the model, not only a large amount of manual labeling is required, but the labeled data obtained in the end is mostly consistent with the model prediction results, which has limited effect in further training of the model and may even produce negative effects such as overfitting. How to efficiently screen sample data to improve model performance has become a difficult problem. Summary of the invention

[0004] The present application discloses a method, device, computer equipment and storage medium for screening sample data, which solve the problems of limited efficiency in screening sample data and limited quality of screened sample data.

[0005] In a first aspect, the present application provides a method for screening sample data, comprising:

[0006] Acquire a plurality of first text data, input the first text data into a model to be trained, and obtain a first semantic code and a first probability distribution;

[0007] According to the current corpus and the first semantic encoding, the first text data is matched to obtain a plurality of second text data with similar semantics, and a plurality of second annotated data is determined to be obtained, wherein the plurality of second annotated data is in one-to-one correspondence with the plurality of second text data;

[0008] Calculating the probability difference between the first text data and the plurality of second labeled data according to the first probability distribution and the plurality of second labeled data to obtain a value score of the first text data;

[0009] The first text data is processed according to the value score of the first text data to obtain sample data.

[0010] In a second aspect, the present application provides a device for screening sample data, comprising:

[0011] A data acquisition module, used to acquire a plurality of first text data, input the first text data into a model to be trained, and obtain a first semantic code and a first probability distribution;

[0012] A semantic matching module, used for matching the first text data according to the current corpus and the first semantic encoding, obtaining a plurality of second text data with similar semantics, and determining to obtain a plurality of second annotated data;

[0013] a difference analysis module, configured to calculate the probability difference between the first text data and the plurality of second text data according to the first probability distribution and the plurality of second annotated data, so as to obtain a value score of the first text data;

[0014] The value assessment module is used to process the first text data according to the value score of the first text data to obtain sample data.

[0015] In a third aspect, the present application provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein the memory stores a strategy model, wherein when the computer program is executed by the processor, a method for screening sample data as provided in any embodiment of the present application is implemented.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor implements a method for screening sample data as provided in any embodiment of the present application.

[0017] The above-mentioned sample data screening method, device, computer equipment and storage medium obtain multiple first text data, input the first text data into the model to be trained, and obtain the first semantic code and the first probability distribution; match the first text data according to the current corpus and the first semantic code, obtain multiple second text data with similar semantics, and determine to obtain multiple second annotated data; calculate the probability difference between the first text data and the multiple second text data according to the first probability distribution and the multiple second annotated data, and obtain the value score of the first text data; process the first text data according to the value score of the first text data to obtain sample data. Screening sample data that is easy to be mispredicted by the model from the perspective of the probability distribution difference predicted by the model, rather than directly annotating a large amount of text data, can save manpower, material resources and computing resources, obtain sample data that is conducive to training the model at a lower cost of computing power, manpower and material resources, and improve the degree of automation and intelligence of sample data screening.

[0018] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 is a schematic flow chart of the steps of a method for screening sample data provided in an embodiment of the present application;

[0021] Figure 2 It is a structural diagram of an intent recognition model provided in an embodiment of the present application;

[0022] Figure 3 is a schematic flow chart of the steps of a value score calculation method provided in an embodiment of the present application;

[0023] Figure 4 It is a schematic flow chart of the steps of a method for screening sample data provided in an embodiment of the present application;

[0024] Figure 5 is a schematic flow chart of the steps of another method for screening sample data provided in an embodiment of the present application;

[0025] Figure 6 is a structural schematic diagram of a sample data screening device provided in an embodiment of the present application;

[0026] Figure 7 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application.

[0027] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0030] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0031] It should be understood that, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "third" are used to distinguish between identical or similar items with substantially identical functions and effects. For example, the first data and the third data are only used to distinguish different data, and their sequence is not limited. Those skilled in the art can understand that words such as "first" and "third" do not limit the quantity and execution order, and words such as "first" and "third" do not necessarily limit them to be different.

[0032] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0033] To facilitate understanding of the embodiments of the present application, some words involved in the embodiments of the present application are briefly explained below.

[0034] 1. BERT (Bidirectional Encoder Representations from Transformers): A pre-trained language representation model based on the Transformer architecture. Its core feature is that it can capture the bidirectional contextual information of the text, so it performs well in various natural language processing (NLP) tasks. Its input representation often includes Token Embeddings that convert words in the text into word embedding vectors, Segment Embeddings that distinguish the two sentences in a sentence pair and are usually used for question-answering tasks, and Position Embeddings that provide the model with information about the position of words in the sentence.

[0035] 2. K-Nearest Neighbors (KNN): A basic classification and regression method, commonly used in statistics and machine learning. The core idea of ​​the KNN algorithm is that the category of a sample can be determined by its nearest neighbor samples. In the KNN model, "nearest neighbors" refers to the K samples closest to the sample in the feature space.

[0036] 3. Euclidean distance: A measurement method used in mathematics to describe the distance between two points. It is also the most intuitive distance concept in daily life. In Euclidean space, the Euclidean distance between two points is the length of the straight line segment connecting the two points.

[0037] 4. KL divergence (Kullback-Leibler Divergence): A measure of the difference between two probability distributions, also known as relative entropy. KL divergence measures the expected value of the extra bits required to obtain information in the probability distribution P relative to the extra bits required to use a better distribution Q. The mathematical expression of KL divergence is:

[0038]

[0039] In machine learning, P can be defined as the probability distribution generated by the model, and Q as the real data distribution.

[0040] 5. Transformer architecture: The Transformer model consists of an encoder and a decoder, both of which contain multiple identical layers. Each layer consists of a self-attention sublayer and a feed-forward sublayer, with residual connections and layer normalization between these sublayers.

[0041] 6. Token Embedding Layer: In the Transformer model, the token embedding layer is the first step in processing the input data. This layer converts the words in the text into vectors of fixed dimensions to capture the semantic information of the words.

[0042] 7. Positional Encoding Layer: An important component in Transformer, it provides the model with information about the position of words in the sequence. This is because the self-attention mechanism of Transformer itself does not contain information about the order of words. Positional encoding is usually implemented by adding a vector of the same dimension as the word embedding, which contains information about the position of the word.

[0043] 8. Multi-Head Attention Layer: One of the core components of Transformer, it is used to learn information in different representation subspaces in parallel, allowing the model to focus on different parts of the input sequence at the same time, thereby better capturing complex dependencies.

[0044] 9. Residual Layer: A key component in Transformer that helps the model effectively transfer information in deep networks by adding a skip connection, which helps solve the gradient disappearance problem in deep networks.

[0045] 10. Batch Normalization: In machine learning, it refers to a technique used to improve the speed and stability of model training. It is usually located after each sublayer in the model, and reduces internal covariate shift by normalizing the input of the module to prevent changes in the input distribution of the network layer from adversely affecting the training process.

[0046] 11.Softmax: A commonly used activation function, the Softmax function is used to convert the raw scores of the output layer into probability distributions that represent the model's prediction confidence for each possible output.

[0047] 12. Activation layer: An important part of the neural network, it introduces nonlinear characteristics so that the network can learn and perform more complex tasks. You can use activation functions such as ReLU, Sigmoid, Leaky ReLU, Softmax, etc. to introduce nonlinear factors, thereby enhancing the expression ability of deep neural networks. Among them, ReLU is a non-saturated activation function, which can alleviate the problem of gradient disappearance caused by the large number of layers in deep neural networks and accelerate the convergence speed.

[0048] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0049] Machine learning technology is being used more and more widely in financial services, retail, smart healthcare and other fields. Among them, natural language processing technology (NLP) has been widely used in human-computer interaction scenarios such as voice intelligent agents and chat box AI customer service. For example, the natural language understanding model (NLU) is used to identify customer intent, thereby providing more convenient, smoother and more customer-friendly services. However, natural language understanding models are often pre-trained based on specific corpora and cannot fully cover all real-world situations. Therefore, it is necessary to use new data in the business to further train the model to improve accuracy and versatility.

[0050] The amount of text data generated by the business is often large, and the model has already accurately predicted most of the data. Continuing to label this data and use it for training will, firstly, have limited effect on improving model performance, secondly, lead to unnecessary consumption of manpower and material resources, and thirdly, a large amount of inefficient data involved in training may lead to overfitting of the model, thereby reducing the versatility and accuracy of the model. Therefore, it is necessary to filter out the data that is truly effective in improving the applicability and accuracy of the model from the text data generated by the business for further training of the model.

[0051] In addition, although the existing natural language understanding models have their own advantages, in actual industrial and commercial applications, upgrading or replacing natural language understanding models often involves development work such as updating the model architecture, reconstructing the loss function, large-scale parameter updates, and business system reconstruction. First, retraining the model consumes a lot of computing resources. Second, the test results of the new model are often based on a standard sample library, and its model accuracy may not be able to surpass the old model that is constantly iterated using business data. Third, the models used in different scenarios and different fields have different architectural designs and are difficult to replace directly. Therefore, it is necessary to screen the text data generated by the business and improve the screening method of sample data to improve the versatility of this solution and save resource consumption.

[0052] To solve the above problems, the present application embodiment proposes a method for screening sample data. Figure 1 , Figure 1 It is a schematic flowchart of the steps of a method for screening sample data provided in an embodiment of the present application.

[0053] like Figure 1 As shown, the sample data screening method specifically includes steps S101 to S104.

[0054] S101. Acquire a plurality of first text data, input the first text data into a model to be trained, and obtain a first semantic code and a first probability distribution.

[0055] The model to be trained may include an embedder, an encoder and a classifier. The encoder may include a multi-head self-attention layer, a residual layer and a normalization layer. The embedder may include a word embedding layer and a position encoding layer.

[0056] Exemplarily, a user may obtain multiple first text data, input the first text data into the encoder of the model to be trained, and obtain the first semantic code corresponding to the first text data; input the first semantic code into the classifier of the model to be trained, and obtain the first probability distribution.

[0057] It should be noted that the first probability distribution can be a classification result (such as an intent classification result) or a result distribution (such as the probability distribution corresponding to each intent classification result obtained by the model based on the first text data).

[0058] Exemplarily, in the text data intent recognition scenario, the model to be trained can be an intent recognition model. The first text data with the third category of intent is input into the model to be trained. After computational reasoning, the model to be trained can obtain a classification result, such as [0 0 1 0]. The classification result can indicate that the intent classification result of the first text data input into the model to be trained is the third category of intent. After computational reasoning, the model to be trained can also obtain a result distribution [0 0.1 0.7 0.2]. The result distribution can indicate that the probability that the intent classification result of the first text data input into the model to be trained is the first category of intent is 0, the probability of the second category of intent is 0.1, the probability of the third category of intent is 0.7, and the probability of the fourth category of intent is 0.2. Among them, the result distribution can be obtained by obtaining the latent expression vector of the last layer of the classifier of the model to be trained.

[0059] In some embodiments, see Figure 2 ,like Figure 2 As shown, Figure 2 It is a structural diagram of an intent recognition model provided in an embodiment of the present application. The intent recognition model can be used to infer the intent category or the first probability distribution based on the input first text data. The intent recognition model can be used to implement the above step S101. The model to be trained can be an intent recognition model. The intent recognition model can include an embedder, multiple encoders and a classifier; wherein the encoder includes a multi-head self-attention layer, a residual layer and a normalization layer.

[0060] Further, such as Figure 2 As shown, the embedder included in the intent recognition model may include a word embedding layer and a position encoding layer, wherein the word embedding layer can be used to convert words or tags in text data into numerical vectors to obtain word unit encoding; the position encoding layer is used to add position information to the word unit encoding to obtain a comprehensive encoding containing the position information, and output it to the encoder.

[0061] Further, such as Figure 2 As shown, the encoder included in the intent recognition model may include an embedding layer, a position encoding layer, a multi-head attention layer, a residual layer, a normalization layer and a feedforward network layer, wherein the multi-head attention layer can be used to perform multi-head attention calculation on the comprehensive encoding of the input, obtain the attention weight, and perform weighted summation to obtain the attention output; the residual layer can be used to perform weighted summation of the input and output of the previous layer to obtain the residual output; the normalization layer can be used to normalize the residual output to obtain a normalized output; the feedforward network layer can be used to transform the output of the previous layer through linear changes and nonlinear activations to obtain a further transformed feature representation.

[0062] Furthermore, the intent recognition model may include multiple encoders, each of which may take the output of a previous encoder or the output of an embedder as input, and may be used to generate a first semantic code and input the first semantic code into a classifier.

[0063] Further, such as Figure 2 As shown, the classifier included in the intent recognition model may include a fully connected layer, an activation layer and an output layer, wherein the activation function that can be adopted by the activation layer includes Softmax, and Softmax can be used to convert the feature representation into a probability distribution; the output layer can be used to output the intent category or the first probability distribution predicted by the intent recognition model based on the input first text data, and the first probability distribution may include the probability estimation of various intent categories by the intent recognition model.

[0064] S102: Match the first text data according to the current corpus and the first semantic encoding to obtain a plurality of second text data with similar semantics, and determine to obtain a plurality of second annotated data.

[0065] The plurality of second annotation data and the plurality of second text data are in one-to-one correspondence.

[0066] Furthermore, the above-mentioned current corpus may include multiple third text data, third semantic codes corresponding to the third text data, and third annotation data corresponding to the third text data, and the second text data includes the first N third text data whose corresponding third semantic codes in the current corpus are slightly different from the first semantic codes, where N is a positive integer.

[0067] Exemplarily, in a text data intent recognition scenario, the current corpus may include multiple third text data with annotated intent classification results, the semantic encoding corresponding to the third text data is the third semantic encoding, and the intent classification result corresponding to the third text data is the third annotated data. For example, for one of the third text data a1, its corresponding semantic encoding is the third semantic encoding b1, and its corresponding intent classification result is the third annotated data c1.

[0068] Specifically, the third semantic code may be obtained by inputting the third text data into an encoder of the model to be trained.

[0069] It should be noted that the plurality of second text data with the closest semantics may include the plurality of second text data with the highest degree of similarity (or the smallest difference) in semantic coding to the first semantic coding. The measure of the degree of similarity between semantic codings may include Euclidean distance and / or Manhattan distance.

[0070] Exemplarily, the first semantic code may be input into a KNN model, and the KNN model may be used to perform matching in the current corpus according to the Euclidean distance between the semantic codes to determine a plurality of second text data with similar semantics.

[0071] Among them, the KNN model is trained based on the current corpus, and the second text data includes the first N third text data whose corresponding third semantic codes in the current corpus are most similar to the first semantic codes (or have the smallest difference and are semantically similar), and N is a positive integer.

[0072] Exemplarily, the training method of the KNN model may be to use the third semantic encoding S = [S1, S2, ..., Sn] of the third text data in the current corpus and the corresponding third annotation data D = [D1, D2, ..., Dn] as a training set to train the KNN model, where n is the total number of data items.

[0073] Furthermore, the Euclidean distance can be calculated according to the first semantic code and the third semantic code by the following formula:

[0074]

[0075] The component on the i dimension, n represents the total dimension of the semantic encoding, and d represents the Euclidean distance.

[0076] It should be noted that, by calculating the Euclidean distance, the difference between semantics can be measured more clearly, and by matching the KNN model, the second text data that is semantically closer to the first text data and its corresponding second annotation data can be efficiently found. The second annotation data may include the intent category of the second text data.

[0077] S103: Calculate the probability difference between the first text data and the plurality of second labeled data according to the first probability distribution and the plurality of second labeled data, and determine the value score of the first text data.

[0078] Among them, the difference between the first probability distribution and the multiple second labeled data can be calculated by KL divergence (Kullback-Leibler Divergence), JS divergence (Jensen-Shannon Divergence), Wasserstein distance (Wasserstein Distance), maximum mean discrepancy (Maximum Mean Discrepancy, MMD), Wasserstein-1 distance (Earth Mover's Distance, EMD), total variation distance (Total Variation Distance) or Bhattacharyya distance.

[0079] Exemplarily, the difference between the first probability distribution and multiple second annotated data can be calculated by JS divergence, and the specific steps include: obtaining the first probability distribution and the multiple second annotated data corresponding to it; taking the second annotated data as Q distribution and the corresponding first probability distribution as P distribution, and calculating the JS divergence of the multiple second annotated data respectively to obtain the JS divergence of the multiple second annotated data; calculating the average value based on the JS divergence of the multiple second annotated data to obtain the value score of the corresponding first text data.

[0080] It should be noted that, compared with KL divergence, JS divergence is bounded and symmetric, and can be applied to situations where two distributions have no overlap or very little overlap, and can provide a stable measure for the predicted distribution or labeled distribution of multiple models, thereby improving the versatility of the sample data screening method. On the other hand, from the perspective of computer device resources, KL divergence can consume less computing resources than JS divergence, so it can be applied to some devices with weaker performance.

[0081] In some embodiments, see Figure 3 , Figure 3 1 is a schematic flow chart of the steps of a value score calculation method provided in an embodiment of the present application. The value score calculation method can be used to implement the above step S103. Figure 3 As shown, the value score calculation method specifically includes steps S103a to S103b.

[0082] S103a, calculating the KL divergence of each second labeled data relative to the first probability distribution to obtain multiple KL divergences.

[0083] Exemplarily, the calculation method of KL divergence can be expressed as:

[0084]

[0085] Here, the first probability distribution is taken as P distribution, the second labeled data is taken as Q distribution, KL represents KL divergence, P i It can represent the components of P distributed in dimension i, Q iIt can represent the component of Q distribution in dimension i. For example, in the intent recognition model, the first probability distribution P = [0.1, 0.2, 0.3, 0.4], which means that the model predicts the probability of the first intent category as 0.1, the probability of the second intent category as 0.2, the probability of the third intent category as 0.3, and the probability of the fourth intent category as 0.4 based on the first text data input. If one of the corresponding second annotation data is Q = [0, 0, 0, 1], the actual annotation in the corpus is the fourth intent category. When multiple second annotation data are obtained, different second annotation data are used as Q distributions to calculate multiple KL divergences.

[0086] S103b: Calculate an average value based on multiple KL divergences to obtain a value score of the first text data.

[0087] Among them, the average value can be calculated by arithmetic mean, geometric mean and weighted mean.

[0088] Through the KNN model, multiple second text data that are semantically similar to the first text data are found, and then the KL divergence of the second annotation data corresponding to the multiple second text data relative to the first probability distribution is calculated respectively, and finally the average KL divergence is used as the value assessment score of the current first text data. Data with small semantic differences but large deviations between the model prediction results and the actual annotations can be obtained. These data can often produce greater effects in model iterations. Screening out data with poor effects in iterations can improve the efficiency of manual annotation and data screening.

[0089] S104. Process the plurality of first text data according to the plurality of value scores corresponding to the plurality of first text data to obtain sample data.

[0090] Exemplarily, multiple first text data may be arranged from large to small according to value points to obtain a rearranged sequence; a preset number of first text data in the rearranged sequence may be used as data to be annotated, and the data to be annotated may be annotated to obtain sample data.

[0091] In some embodiments, see Figure 4 , Figure 4 1 is a schematic flow chart of the steps of a method for screening sample data provided in an embodiment of the present application. Figure 4 As shown, the sample data screening method can be used to implement the above step S104. The sample data screening method can include steps S104a to S104c.

[0092] S104a, determining first M first text data with higher value scores among a plurality of first text data according to the value scores of the first text data, and obtaining data to be annotated.

[0093] Wherein, M is a positive integer.

[0094] It should be noted that since the distribution of value points may be uneven, a threshold condition can be added to ensure that the obtained sample data can have a positive effect on improving the performance of the model and prevent the introduction of inefficient data during the training process from causing the model to overfit.

[0095] Exemplarily, the data to be annotated may be screened out from the first M first text data with higher value scores according to a preset value range.

[0096] S104b, label the data to be labeled to obtain sample data.

[0097] Exemplarily, the data to be labeled can be labeled by manual labeling or machine labeling to obtain sample data.

[0098] In some embodiments, see Figure 5 , Figure 5 FIG. 1 is a flowchart showing the steps of another method for screening sample data provided by an embodiment of the present application. Figure 5 As shown, the sample data screening method can be used to implement the above step S104. The sample data screening method can include steps S104c to S104d.

[0099] S104c. According to the value score of the first text data, determine the first text data whose value score is within a preset value range among the multiple first text data to obtain the data to be marked.

[0100] Exemplarily, the value score of each first text data can be used to determine whether the value score is within a preset value score range, thereby obtaining first text data whose value score is within the preset value score range; and the obtained first text data is stored in a new data structure as data to be annotated.

[0101] It should be noted that the above-mentioned preset value score range can be manually preset or generated according to the value score calculation method and corresponding experience data.

[0102] Exemplarily, according to the method for calculating the value score, if the greater the difference between the first probability distribution and the second labeled data, the smaller the value score, then the preset value score range can select a range with a smaller value score.

[0103] S104d, label the data to be labeled to obtain sample data.

[0104] Exemplarily, the data to be labeled can be labeled by manual labeling or machine labeling to obtain sample data.

[0105] It should be noted that the above Figure 4 The sample data shown is screened in the same way as Figure 5 The sample data screening methods shown can be combined.

[0106] In some embodiments, based on multiple value scores corresponding to multiple first text data, multiple first text data are processed to obtain sample data, which may specifically include the following steps: based on the sizes of the multiple value scores, the first text data whose value scores are within a preset value score range and belong to the top M first text data with higher value scores among the multiple first text data are determined to obtain data to be labeled, wherein M is a positive integer; and the data to be labeled are labeled to obtain sample data.

[0107] In some embodiments, before the step of acquiring multiple first text data, inputting the first text data into the model to be trained, and obtaining the first semantic code and the first probability distribution corresponding to the first text data, the following steps can be performed: acquiring the current time and the preset execution time range, and determining whether the current time belongs to the preset execution time range; if the current time belongs to the preset execution time range, performing the step of acquiring multiple first text data, inputting the first text data into the model to be trained, and obtaining the first semantic code and the first probability distribution corresponding to the first text data.

[0108] The current time and the preset execution time range may be obtained by reading local cache, network communication and / or calling an interface.

[0109] By determining whether the current time falls within the preset execution time range, the sample data acquisition task can be executed when the user's business system is idle or the business service pressure is low, thereby improving the security and stability of the acquisition process, allowing the user's business system to operate normally and the model iteration and upgrade to proceed in an orderly manner.

[0110] like Figure 6 As shown, Figure 6 1 is a schematic diagram of the structure of a sample data screening device provided in an embodiment of the present application, wherein the sample data screening device is used to execute the aforementioned sample data screening method, wherein the sample data screening device can be configured in a terminal or a server.

[0111] like Figure 6 As shown, the sample data screening device 100 includes a data acquisition module 101 , a semantic matching module 102 , a difference analysis module 103 , and a value assessment module 104 .

[0112] The data acquisition module 101 is used to acquire a plurality of first text data, input the first text data into the model to be trained, and obtain a first semantic code and a first probability distribution.

[0113] The semantic matching module 102 is used to match the first text data according to the current corpus and the first semantic encoding to obtain a plurality of second text data with similar semantics, and determine to obtain a plurality of second annotated data.

[0114] The difference analysis module 103 is used to calculate the probability difference between the first text data and the plurality of second text data according to the first probability distribution and the plurality of second annotated data, so as to obtain the value score of the first text data.

[0115] The value evaluation module 104 is used to process the first text data according to the value score of the first text data to obtain sample data.

[0116] In some embodiments, the sample data screening device 100 also includes a conditional judgment module, which is used to obtain the current time and the preset execution time range, and judge whether the current time belongs to the preset execution time range, before obtaining multiple first text data, inputting the first text data into the model to be trained, and obtaining the first semantic code and the first probability distribution corresponding to the first text data; if the current time belongs to the preset execution time range, execute the steps of obtaining multiple first text data, inputting the first text data into the model to be trained, and obtaining the first semantic code and the first probability distribution.

[0117] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0118] The above-mentioned device can be implemented in the form of a computer program. Figure 7 Runs on the computer device shown.

[0119] See also Figure 7 , Figure 7 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server. Figure 7 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0120] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any one of the sample data screening methods.

[0121] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0122] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any method for screening sample data.

[0123] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will appreciate that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0124] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0125] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0126] Acquire multiple first text data, input the first text data into the model to be trained, and obtain a first semantic code and a first probability distribution; match the first text data according to the current corpus and the first semantic code to obtain multiple second text data with similar semantics, and determine to obtain multiple second annotated data, wherein the multiple second annotated data are in one-to-one correspondence with the multiple second text data; calculate the probability difference between the first text data and the multiple second text data according to the first probability distribution and the multiple second annotated data to obtain the value score of the first text data; process the first text data according to the value score of the first text data to obtain sample data.

[0127] In some embodiments, when the processor calculates the probability difference between the first text data and the plurality of second labeled data according to the first probability distribution and the plurality of second labeled data to obtain the value sharing of the first text data, it is specifically used to implement:

[0128] The KL divergence of each of the second labeled data relative to the first probability distribution is calculated to obtain multiple KL divergences; and an average value is calculated based on the multiple KL divergences to obtain the value score of the first text data.

[0129] In some embodiments, when the processor is used to process the first text data according to the value score of the first text data to obtain sample data, it is specifically used to implement:

[0130] According to the value scores of the first text data, the first M first text data with higher value scores among the multiple first text data are determined to obtain data to be labeled, where M is a positive integer; and the data to be labeled are labeled to obtain sample data.

[0131] In some embodiments, when the processor is used to process the first text data according to the value score of the first text data to obtain sample data, it is specifically used to implement:

[0132] According to the value score of the first text data, the first text data whose value score is within a preset value score range is determined among the multiple first text data to obtain data to be annotated; and the data to be annotated is annotated to obtain sample data.

[0133] In some embodiments, before the processor is used to obtain a plurality of first text data and input the first text data into the model to be trained to obtain the first semantic encoding and the first probability distribution, it is used to implement:

[0134] Obtain the current time and the preset execution time range, and determine whether the current time belongs to the preset execution time range; if the current time belongs to the preset execution time range, execute the step of obtaining multiple first text data, inputting the first text data into the model to be trained, and obtaining a first semantic code and a first probability distribution.

[0135] In some embodiments, when the processor is used to match the first text data according to the current corpus and the first semantic encoding to obtain a plurality of second text data with similar semantics, it is specifically used to implement:

[0136] The first semantic encoding is input into a KNN model, wherein the KNN model is trained based on the current corpus; and according to the Euclidean distance between the semantic encodings, the KNN model is used to perform matching in the current corpus to determine a plurality of second text data with similar semantics.

[0137] In some embodiments, the model to be trained includes at least one embedder, multiple encoders and a classifier, the embedder includes at least a word embedding layer and a position encoding layer, and the encoder includes at least a multi-head self-attention layer, a residual layer and a normalization layer.

[0138] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a computer program. The computer program includes program instructions. The processor executes the program instructions to implement any sample data screening method provided in the embodiments of the present application.

[0139] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0140] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for screening sample data, characterized in that: include: Acquire a plurality of first text data, input the first text data into a model to be trained, and obtain a first semantic code and a first probability distribution; According to the current corpus and the first semantic encoding, the first text data is matched to obtain a plurality of second text data with similar semantics, and a plurality of second annotated data is determined to be obtained, wherein the plurality of second annotated data is in one-to-one correspondence with the plurality of second text data; Calculating the probability difference between the first text data and the plurality of second labeled data according to the first probability distribution and the plurality of second labeled data to obtain a value score of the first text data; The first text data is processed according to the value score of the first text data to obtain sample data.

2. The method according to claim 1, characterized in that The step of calculating the probability difference between the first text data and the plurality of second labeled data according to the first probability distribution and the plurality of second labeled data to obtain the value score of the first text data includes: Calculate the KL divergence of each of the second labeled data with respect to the first probability distribution to obtain multiple KL divergences; An average value is calculated based on the multiple KL divergences to obtain a value score of the first text data.

3. The method according to claim 1, characterized in that The step of processing the first text data according to the value score of the first text data to obtain sample data includes: According to the value scores of the first text data, determine the first M first text data with higher value scores among the plurality of first text data to obtain data to be annotated, where M is a positive integer; The data to be labeled is labeled to obtain sample data.

4. The method according to claim 3, characterized in that The step of processing the first text data according to the value score of the first text data to obtain sample data includes: According to the value score of the first text data, determine the first text data whose value score is within a preset value range among the plurality of first text data, and obtain the data to be annotated; The data to be labeled is labeled to obtain sample data.

5. The method according to any one of claims 1 to 4, characterized in that: Before acquiring a plurality of first text data and inputting the first text data into the model to be trained to obtain a first semantic code and a first probability distribution, the method further includes: Obtain the current time and the preset execution time range, and determine whether the current time falls within the preset execution time range; If the current time belongs to the preset execution time range, the step of acquiring a plurality of first text data is executed, inputting the first text data into the model to be trained, and obtaining a first semantic code and a first probability distribution.

6. The method according to any one of claims 1 to 4, characterized in that: The matching of the first text data according to the current corpus and the first semantic encoding to obtain a plurality of second text data with similar semantics includes: Inputting the first semantic encoding into a KNN model, wherein the KNN model is trained according to the current corpus; According to the Euclidean distance between the semantic codes, the KNN model is used to perform matching in the current corpus to determine a plurality of second text data with similar semantics.

7. The method according to any one of claims 1 to 4, characterized in that: The model to be trained includes at least one embedder, multiple encoders and a classifier, the embedder includes at least a word embedding layer and a position encoding layer, and the encoder includes at least a multi-head self-attention layer, a residual layer and a normalization layer.

8. A device for screening sample data, characterized in that: include: A data acquisition module, used to acquire a plurality of first text data, input the first text data into a model to be trained, and obtain a first semantic code and a first probability distribution; A semantic matching module, configured to match the first text data according to the current corpus and the first semantic encoding, obtain a plurality of second text data with similar semantics, and determine to obtain a plurality of second annotated data, wherein the plurality of second annotated data is in one-to-one correspondence with the plurality of second text data; a difference analysis module, configured to calculate the probability difference between the first text data and the plurality of second text data according to the first probability distribution and the plurality of second annotated data, so as to obtain a value score of the first text data; The value assessment module is used to process the first text data according to the value score of the first text data to obtain sample data.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the method for screening sample data according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the method for screening sample data according to any one of claims 1 to 7.

Images