Sample data screening method, model training method, device, server and storage medium

Abstract

The present application relates to the field of artificial intelligence and discloses a method for screening sample data, a model training method, an apparatus, a server and a storage medium, wherein the screening method constructs a corpus database to obtain second text data to be predicted, and uses an initial semantic recognition model to classify the second text data to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data and a second probability distribution corresponding to the second text intent; according to the second text vector, multiple first text data matching the second text data are determined from the corpus database as target text data, and a first probability distribution of the first text intent corresponding to the target text data is obtained; the probability distribution difference between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data is calculated respectively, and the target sample data is screened out from the second text data according to the probability distribution difference.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a sample data screening method, model training method, device, server, and storage medium. Background Art

[0002] Natural language understanding (NLU) is a key area of ​​artificial intelligence technology, primarily aimed at enabling computers to understand human language. NLU is commonly used in fields such as question-answering systems, intelligent customer service systems, and knowledge graphs. The core principle of NLU is to achieve semantic understanding by using semantic recognition algorithms and models to classify and predict input text. Therefore, the recognition accuracy of the algorithm model is a crucial evaluation metric.

[0003] Generally, when an algorithm model is just built, the natural language recognition accuracy is low, and iterative optimization is required to ensure that the recognition accuracy of the algorithm model meets the preset requirements, thereby meeting actual production needs.

[0004] The typical algorithm model optimization process can be summarized as raw data extraction, raw data cleaning, manual labeling, training set updates, model training, and testing. The key to this optimization process is to obtain as many examples of model recognition errors (also known as bad case samples) as possible, adding them to the training set for the algorithm model to learn, thereby improving the algorithm model's performance more quickly and effectively.

[0005] Currently, a common industry practice is to select business log data from a specific time period, randomly extract model input text and corresponding recognition results, and then manually annotate the text. If the manual annotation results disagree with the model recognition results, these are filtered out as bad case samples and then added to the training set to retrain the model. However, the proportion of bad case samples obtained through this method is unstable. In other words, the number of valid bad case samples obtained by annotating sample data from different dates or batches fluctuates greatly. A large amount of manpower is wasted on annotating invalid sample data, resulting in low output efficiency. Summary of the Invention

[0006] The main purpose of the embodiments of the present application is to provide a sample data screening method, model training method, device, server and storage medium, aiming to efficiently screen effective sample data, and ultimately apply the effective sample data to model training, which can effectively improve the model training efficiency and model recognition accuracy.

[0007] In a first aspect, the present application provides a semantic recognition model training method, the method comprising:

[0008] Obtaining a first text data set, obtaining a first text vector corresponding to each first text data in the first text data set, and obtaining a first text intent corresponding to each first text data in the first text data set;

[0009] Constructing a corpus database based on the first text data, the first text vector corresponding to the first text data, and the first text intent;

[0010] Obtaining second text data to be predicted, and classifying the second text data using the initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data, and a second probability distribution corresponding to the second text intent;

[0011] Determining, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data, and obtaining a first probability distribution of the first text intent corresponding to the target text data;

[0012] respectively calculating the probability distribution differences between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data, and filtering out target sample data from the second text data according to the probability distribution differences;

[0013] The initial semantic recognition model is trained using at least the target sample data until a target semantic recognition model is obtained.

[0014] In a second aspect, the present application provides a method for screening sample data, comprising:

[0015] Obtaining a first text data set, obtaining a first text vector corresponding to each first text data in the first text data set, and obtaining a first text intent corresponding to each first text data in the first text data set;

[0016] Constructing a corpus database based on the first text data, the first text vector corresponding to the first text data, and the first text intent;

[0017] Obtaining second text data to be predicted, and classifying the second text data using the initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data, and a second probability distribution corresponding to the second text intent;

[0018] Determining, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data, and obtaining a first probability distribution of the first text intent corresponding to the target text data;

[0019] The probability distribution differences between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data are calculated respectively, and target sample data are filtered out from the second text data according to the probability distribution differences.

[0020] In a third aspect, the present application provides a semantic recognition model training device, comprising:

[0021] A first processing module is configured to obtain a first text data set, obtain a first text vector corresponding to each first text data in the first text data set, and obtain a first text intent corresponding to each first text data in the first text data set;

[0022] A database module, configured to construct a corpus database based on the first text data, the first text vector corresponding to the first text data, and the first text intent;

[0023] a second processing module, configured to obtain second text data to be predicted, and classify the second text data using the initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data, and a second probability distribution corresponding to the second text intent;

[0024] a text matching module, configured to determine, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data, and obtain a first probability distribution of the first text intent corresponding to the target text data;

[0025] a sample screening module, configured to respectively calculate a probability distribution difference between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data, and screen target sample data from the second text data based on the probability distribution difference;

[0026] A model training module is used to train the initial semantic recognition model using at least the target sample data until a target semantic recognition model is obtained. In a fourth aspect, the present application provides a server including a processor and a memory;

[0027] The memory is used to store computer programs;

[0028] The processor is used to execute the computer program and implement the aforementioned semantic recognition model training method or sample data screening method when executing the computer program.

[0029] In a fourth aspect, the present application provides a computer-readable storage medium, which, when executed by one or more processors, enables the one or more processors to execute the steps of the aforementioned semantic recognition model training method or the steps of the sample data screening method.

[0030] The embodiment of the present application provides a method for screening sample data, a model training method, an apparatus, a device and a storage medium. The method for screening sample data is as follows: obtaining a first text data set, obtaining a first text vector corresponding to each first text data in the first text data set, and obtaining a first text intent corresponding to each first text data in the first text data set; constructing a corpus database according to the first text data, the first text vector corresponding to the first text data and the first text intent; obtaining second text data to be predicted, and classifying the second text data using an initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data and a second probability distribution corresponding to the second text intent; determining a plurality of first text data matching the second text data from the corpus database as target text data according to the second text vector, and obtaining a first probability distribution of the first text intent corresponding to the target text data; respectively calculating the probability distribution difference between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data, and screening target sample data from the second text data according to the probability distribution difference. Based on the difference in probability distribution, we can know the text data with lower model recognition accuracy in the second text data to be predicted, so as to effectively obtain more model recognition error samples as target sample data, so that the target sample data can be added to the training set for the algorithm model to be optimized to learn, so as to improve the performance of the algorithm model more quickly and effectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0032] Figure 1 A flowchart of the steps of a semantic recognition model training method provided in an embodiment of the present application;

[0033] Figure 2 A schematic diagram of the scenario structure for constructing a corpus database provided in an embodiment of the present application;

[0034] Figure 3This is a schematic diagram of the model structure of the twin network model for vector extraction from text data provided in the implementation of this application;

[0035] Figure 4 A structural diagram of a semantic recognition model training device is provided for an embodiment of the present application;

[0036] Figure 5 A schematic flow chart of the steps of a method for screening sample data provided in an embodiment of the present application;

[0037] Figure 6 A schematic diagram of the structure of a device for screening sample data provided in an embodiment of the present application;

[0038] Figure 7 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0040] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0041] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0042] Natural language understanding (NLU) is a key area of ​​artificial intelligence technology, primarily aimed at enabling computers to understand human language. NLU is commonly used in fields such as question-answering systems, intelligent customer service systems, and knowledge graphs. The core principle of NLU is to achieve semantic understanding by using semantic recognition algorithms and models to classify and predict input text. Therefore, the recognition accuracy of the algorithm model is a crucial evaluation metric.

[0043] Generally, when an algorithm model is just built, the natural language recognition accuracy is low, and iterative optimization is required to ensure that the recognition accuracy of the algorithm model meets the preset requirements, thereby meeting actual production needs.

[0044] The typical algorithm model optimization process can be summarized as raw data extraction, raw data cleaning, manual labeling, training set updates, model training, and testing. The key to this optimization process is to obtain as many examples of model recognition errors (also known as bad case samples) as possible, adding them to the training set for the algorithm model to learn, thereby improving the algorithm model's performance more quickly and effectively.

[0045] Currently, a common industry practice is to select business log data from a specific time period, randomly extract model input text and corresponding recognition results, and then manually annotate the text. If the manual annotation results disagree with the model recognition results, these are filtered out as bad case samples and then added to the training set to retrain the model. However, the proportion of bad case samples obtained through this method is unstable. In other words, the number of valid bad case samples obtained by annotating sample data from different dates or batches fluctuates greatly. A large amount of manpower is wasted on annotating invalid sample data, resulting in low output efficiency.

[0046] Based on this, the present application provides a sample data screening method, model training method, apparatus, device and storage medium. The sample data screening method and model training method can be executed by a computer device, which can be a terminal device or a server; the terminal device here can include but is not limited to: computers, smart phones, tablet computers, laptops, intelligent voice interaction devices, smart home appliances, vehicle terminals, smart wearable devices, etc.; the server here can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0047] The following embodiments of the present application are described in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0048] Please refer to Figure 1 , Figure 1 A flowchart illustrating the steps of a semantic recognition model training method provided in an embodiment of the present application.

[0049] like Figure 1As shown, the semantic recognition model training method includes steps S101 to S106.

[0050] Step S101: obtaining a first text data set, obtaining a first text vector corresponding to each first text data in the first text data set, and obtaining a first text intent corresponding to each first text data in the first text data set.

[0051] Exemplarily, the first text data set includes multiple first text data. The first text data can be obtained by extracting first business production log data, performing data cleaning on the first business production log data to eliminate erroneous data therein, the erroneous data including but not limited to blank data and garbled data, thereby obtaining cleaned text data, and annotating the cleaned text data, thereby using the annotated text data as the first text data. The multiple first text data can constitute the first text data set (also known as a training corpus). Alternatively, the first text data can also be obtained by manually collecting and organizing the multiple first text data.

[0052] After obtaining the first text data set, each first text data in the first text data set is analyzed using a preset semantic recognition model to obtain a first text vector corresponding to each first text data in the first text data set. Alternatively, each first text data in the first text data set is analyzed using a preset semantic recognition model to obtain a first text vector and a first text intent corresponding to each first text data in the first text data set.

[0053] That is, the semantic recognition model can be used to obtain the first text vector corresponding to each first text data in the first text data set, or the semantic recognition model can also be used to obtain the first text vector and first text intention corresponding to each first text data in the first text data set.

[0054] Optionally, the first text intention corresponding to each first text data in the first text data set may also be obtained by manually analyzing each first text data in the first text data set.

[0055] In some implementations, obtaining a first text vector corresponding to each first text data in the first text data set includes:

[0056] Each of the first text data in the first text data set is input as input data into a preset semantic recognition model to obtain a first text vector corresponding to the first text data, wherein the semantic recognition model has the same first network branch and second network branch, and at least some network structure parameters are shared in the first network branch and the second network branch.

[0057] Optionally, the first network branch includes a first text vector conversion network and a first pooling network connected to the output end of the first text vector conversion network; the second network branch includes a second text vector conversion network and a second pooling network connected to the output end of the second text vector conversion network; wherein, parameters are shared between the first text vector conversion network and the second text vector conversion network.

[0058] like Figure 2 As shown, data with the same label are selected from the first text data (also called training corpus) of the first text data set, and sample pairs are constructed in pairs, such as (text_A, text_A', 1). Data with different labels are selected, and dissimilar sample pairs are constructed in pairs, such as (text_A, text_B, 0). In this way, a similar sample set and a dissimilar sample set are obtained to form a semantic recognition model training set, that is, the two texts of each sample pair are used as two inputs of the preset semantic recognition model, thereby obtaining the first text vector corresponding to each first text data in the first text data set.

[0059] like Figure 3 As shown, the semantic recognition model is taken as a twin network model as an example. The twin network model has the same first network branch and second network branch, and at least part of the network structure parameters in the first network branch and the second network branch are shared.

[0060] Specifically, the first network branch includes a first text vector conversion network (Sentence-BERT model) and a first pooling network (pooling layer) connected to the output end of the first text vector conversion network. The second network branch includes a second text vector conversion network (Sentence-BERT model) and a second pooling network (pooling layer) connected to the output end of the second text vector conversion network, wherein the network structure parameters of the first pooling network and the second pooling network are shared, and / or the network structure parameters of the first text vector conversion network and the second text vector conversion network are shared.

[0061] In this embodiment, the twin network model is based on the Sentence-BERT model, and a pooling layer is added after the BERT output result to obtain a text vector of fixed dimensionality (i.e., text encoding). The twin network model is used to obtain the first text vector corresponding to each first text data in the first text data set, where u and v represent two text vectors, and |uv| represents the absolute value of the two vectors.

[0062] Optionally, a semantic recognition model is used to perform model prediction on the first text data from the first text data set to obtain the corresponding semantic probability distribution result, that is, the first text intent corresponding to each first text data, and the probability distribution result of the first text intent (also called the first probability distribution corresponding to the first text intent).

[0063] Step S102: constructing a corpus database according to the first text data, the first text vector corresponding to the first text data, and the first text intent.

[0064] like Figure 2 As shown, exemplarily, after obtaining the first text data, the first text vector corresponding to the first text data, and the first text intent, the corresponding first text intent is manually calibrated, and the first text data, the first text vector corresponding to the first text data, and the first text intent are sorted and associated, and the data is stored to construct a corpus database.

[0065] Step S103: Obtain the second text data to be predicted, and classify the second text data using the initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data, and a second probability distribution corresponding to the second text intent.

[0066] Exemplarily, by extracting the second business production log data, the second business production log data is cleaned to eliminate erroneous data therein, which includes but is not limited to blank data and garbled data, thereby obtaining cleaned text data, and the cleaned text data is labeled, so that the labeled text data is used as the second text data, and the second text data is classified using a preset initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data, and a second probability distribution corresponding to the second text intent.

[0067] Optionally, the initial semantic recognition model and the semantic recognition model used to obtain the first text vector corresponding to each first text data in the first text data set are the same recognition model.

[0068] Step S104: determining, from the corpus database according to the second text vector, a plurality of first text data matching the second text data as target text data, and obtaining a first probability distribution of the first text intent corresponding to the target text data.

[0069] Optionally, determining, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data includes:

[0070] Determining, from the first text vector corresponding to the first text data, a target text vector whose similarity with the second text vector exceeds a preset value;

[0071] The first text data corresponding to the target text vector is used as the target text data.

[0072] Exemplarily, a first text vector whose similarity to a second text vector exceeds a preset value is obtained from the corpus database, and the first text vector is used as the target text vector, and the first text data corresponding to the target text vector is used as the target text data. In other words, the first text data that matches the second text data is obtained from the corpus database, that is, the first text data whose similarity to the second text data exceeds the preset value is obtained from the corpus database.

[0073] For example, the cosine similarity between each second text vector and each first text vector in the corpus database is calculated, and the first text vector corresponding to the second text vector whose cosine similarity exceeds a preset value or a maximum value is used as the target text vector, and the first text data corresponding to the target text vector is used as the target text data. The target text data can be text data that is considered to be similar to or identical to the first text data in the corpus database.

[0074] Step S105: respectively calculating the probability distribution differences between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data, and filtering out target sample data from the second text data according to the probability distribution differences.

[0075] In some implementations, filtering out target sample data from the second text data based on the probability distribution difference includes:

[0076] Calculating a probability distribution difference value corresponding to the probability distribution difference using a preset difference calculation function, wherein the difference calculation function is exponentially related to the first probability distribution and the second probability distribution;

[0077] The probability distribution difference values ​​are sorted in size, and the second text data corresponding to the first N probability distribution difference values ​​are selected from the sorted probability distribution difference value queue as target sample data.

[0078] For example, a model typically identifies the text intent corresponding to text data using probability, and typically uses the text intent with the highest probability as the final text intent of the text data. For example, there are N second text data, labeled A1 to An. The true semantics of the second text data A1 is "apple (mobile phone)", the true semantics of the second text data A2 is "apple (fruit)", and the true semantics of the second text data An is "apple (computer)".

[0079] After the initial semantic recognition model recognizes the second text data A1, it obtains that the probability of the second text intent A1 corresponding to the second text data A1 is 0.9, the probability of the corresponding second text intent B1 is 0.3, and the probability of the corresponding second text intent C1 is 0.2.

[0080] After the initial semantic recognition model recognizes the second text data A2, it obtains that the probability of the second text intent A2 corresponding to the second text data A2 is 0.3, the probability of the corresponding second text intent B2 is 0.7, and the probability of the corresponding second text intent C2 is 0.1.

[0081] After the initial semantic recognition model recognizes the second text data An, it is obtained that the probability of the second text intent An corresponding to the second text data An is 0.2, the probability of the corresponding second text intent Bn is 0.1, and the probability of the corresponding second text intent Cn is 0.8.

[0082] There are N first text data items, or target text data items, that match the second text data items, labeled X1 through Xn. The true meaning of the first text data item X1 is "apple (mobile phone)", the true meaning of the first text data item X2 is "apple (fruit)", and the true meaning of the first text data item Xn is "apple (computer)". That is, the first text data item X1 matches the second text data item A1, the first text data item X2 matches the second text data item A2, and the first text data item Xn matches the second text data item An.

[0083] In the corpus database, the probability of the first text intent X1 corresponding to the first text data X1 is 0.99, the probability of the corresponding first text intent B1 is 0.3, and the probability of the corresponding first text intent C1 is 0.2.

[0084] In the corpus database, the probability of the first text intent X2 corresponding to the first text data X2 is 0.98, the probability of the corresponding first text intent B2 is 0.3, and the probability of the corresponding first text intent C2 is 0.2.

[0085] In the corpus database, the probability of the first text intention Xn corresponding to the first text data Xn is 0.99, the probability of the corresponding first text intention Bn is 0.3, and the probability of the corresponding first text intention Cn is 0.2.

[0086] Through analysis, it can be seen that the probability distribution difference values ​​of the initial semantic recognition model for recognizing the second text data A2 and the second text data An are relatively large. Therefore, the initial semantic recognition model has great difficulty in recognizing the second text data A2 and the second text data An, and it is not easy to distinguish the second text data A2 and the second text data An. Therefore, the second text data A2 and the second text data An are used as target text data, also known as model recognition error samples, or badcase samples.

[0087] The first text data stored in the corpus database is usually standard sample data, that is, the intention recognition accuracy of the first text data is 100%, which can be used as reference sample data to screen out target text data that matches the corpus database from the second text data and is considered to be similar or identical text data.

[0088] In addition, the initial semantic recognition model may have deviations in the semantic recognition of the second text data. In order to optimize the initial semantic recognition model, by analyzing the model's recognition probability distribution of similar or identical text data, text data with poor model recognition accuracy can be screened out, that is, second text data with a large difference in probability distribution from the reference sample data, and the screened out second text data is used as target sample data, also called model recognition error sample (or badcase sample), that is, the current semantic recognition model has low recognition accuracy for this sample. Using this error sample as training data to conduct targeted training on the model can better improve the model's recognition accuracy and model training efficiency.

[0089] It is understood that the difference calculation function can be set as needed, for example, the difference calculation function KL is P(x) and Q(x) are the probability distributions on the random variable X. P(x) is the first probability distribution corresponding to the text semantics of the first text data stored in the database. Q(x) is the second probability distribution corresponding to the text semantics of the second text data obtained in time by the initial semantic recognition model. The probability distribution of the second text data is respectively compared with the first distribution probability of multiple target text data to calculate the KL score, and then the sum and average are taken to obtain the value score of the sample data to be evaluated, that is, the calculation Therefore, the top K second text data with higher scores can be screened out from the second text data according to the scores as target sample data.

[0090] Step S106: at least using the target sample data to train the initial semantic recognition model until a target semantic recognition model is obtained.

[0091] In some embodiments, the training of the initial semantic recognition model using at least the target sample data until a target semantic recognition model is obtained includes:

[0092] Performing data labeling on the target sample data, and updating the first text dataset according to the labeled target sample data to obtain an updated text dataset;

[0093] The initial semantic recognition model is trained using the updated text data set until the loss function of the initial semantic recognition model converges to a preset value or the initial semantic recognition model is trained a preset number of times, thereby obtaining a target semantic recognition model.

[0094] Exemplarily, after obtaining the target sample data, the target sample data is labeled and then added to the first text data set to obtain an updated text data set. The updated text data set is then used to train the initial semantic recognition model until the loss function of the initial semantic recognition model converges to a preset value or the initial semantic recognition model is trained a preset number of times, and the target semantic recognition model can be obtained.

[0095] See also Figure 4 , Figure 4 A schematic block diagram of the structure of the semantic recognition model training device provided in an embodiment of the present application.

[0096] like Figure 4As shown, the semantic recognition model training device 200 includes a first processing module 201, a database module 202, a second processing module 203, a text matching module 204, a sample screening module 205, and a model training module 206. The first processing module 201 is used to obtain a first text data set, obtain a first text vector corresponding to each first text data in the first text data set, and obtain a first text intent corresponding to each first text data in the first text data set. The database module 202 is used to construct a corpus database based on the first text data, the first text vector corresponding to the first text data, and the first text intent. The second processing module 203 is used to obtain second text data to be predicted and classify the second text data using the initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data, and a second probability distribution corresponding to the second text intent. The text matching module 204 is used to determine multiple first text data that match the second text data from the corpus database as target text data based on the second text vector, and obtain a first probability distribution of the first text intent corresponding to the target text data. The sample screening module 205 is configured to calculate the probability distribution difference between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data, and to screen target sample data from the second text data based on the probability distribution difference. The model training module 206 is configured to train the initial semantic recognition model using at least the target sample data until a target semantic recognition model is obtained.

[0097] In some implementations, obtaining a first text vector corresponding to each first text data in the first text data set includes:

[0098] Each of the first text data in the first text data set is input as input data into a preset semantic recognition model to obtain a first text vector corresponding to the first text data, wherein the semantic recognition model has the same first network branch and second network branch, and at least some network structure parameters are shared in the first network branch and the second network branch.

[0099] In some embodiments, the first network branch includes a first text vector conversion network and a first pooling network connected to the output end of the first text vector conversion network; the second network branch includes a second text vector conversion network and a second pooling network connected to the output end of the second text vector conversion network; wherein parameters are shared between the first text vector conversion network and the second text vector conversion network.

[0100] In some implementations, determining, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data includes:

[0101] Determining, from the first text vector corresponding to the first text data, a target text vector whose similarity with the second text vector exceeds a preset value;

[0102] The first text data corresponding to the target text vector is used as the target text data.

[0103] In some implementations, filtering out target sample data from the second text data based on the probability distribution difference includes:

[0104] Calculating a probability distribution difference value corresponding to the probability distribution difference using a preset difference calculation function, wherein the difference calculation function is exponentially related to the first probability distribution and the second probability distribution;

[0105] The probability distribution difference values ​​are sorted in size, and the second text data corresponding to the first N probability distribution difference values ​​are selected from the sorted probability distribution difference value queue as target sample data.

[0106] In some embodiments, the training of the initial semantic recognition model using at least the target sample data until a target semantic recognition model is obtained includes:

[0107] Performing data labeling on the target sample data, and updating the first text dataset according to the labeled target sample data to obtain an updated text dataset;

[0108] The initial semantic recognition model is trained using the updated text data set until the loss function of the initial semantic recognition model converges to a preset value, thereby obtaining a target semantic recognition model.

[0109] It can be understood that the semantic recognition model training device 200 can be applied to a computer device and used to execute the steps of the semantic recognition model training method provided in any embodiment of the present application.

[0110] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the semantic recognition model training device 200 described above can refer to the corresponding process in the aforementioned semantic recognition model training method embodiment, and will not be repeated here.

[0111] See also Figure 5 , Figure 5 A schematic flow chart of the steps of a method for screening sample data provided in an embodiment of the present application.

[0112] like Figure 5 As shown, the method for screening sample data includes steps S201 to S205.

[0113] Step S201: obtaining a first text data set, obtaining a first text vector corresponding to each first text data in the first text data set, and obtaining a first text intent corresponding to each first text data in the first text data set;

[0114] Step S202: constructing a corpus database according to the first text data, the first text vector corresponding to the first text data, and the first text intent;

[0115] Step S203: obtaining second text data to be predicted, and classifying the second text data using the initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data, and a second probability distribution corresponding to the second text intent;

[0116] Step S204: determining, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data, and obtaining a first probability distribution of the first text intent corresponding to the target text data;

[0117] Step S205: respectively calculating the probability distribution differences between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data, and filtering out target sample data from the second text data according to the probability distribution differences.

[0118] It can be understood that step S201 and Figure 1 The same as step S101, step S202 and Figure 1 The same as step S102, step S203 and Figure 1 The same as step S103 in step S203. Figure 1 The same as step S103, step S204 and Figure 1 The same as step S104, step S205 is the same as Figure 1 The same as step S105, which will not be described here.

[0119] See also Figure 6 , Figure 6 This is a schematic block diagram of the structure of the sample data screening device provided in an embodiment of the present application.

[0120] like Figure 6As shown, the sample data screening device 400 includes a first processing module 401, a database module 402, a second processing module 403, a text matching module 404 and a sample screening module 405. The first processing module 401 is used to obtain a first text data set, and obtain the first text vector corresponding to each first text data in the first text data set, and obtain the first text intent corresponding to each first text data in the first text data set. The database module 402 is used to construct a corpus database based on the first text data, the first text vector corresponding to the first text data and the first text intent. The second processing module 403 is used to obtain the second text data to be predicted, and classify the second text data using the initial semantic recognition model to obtain the second text vector corresponding to the second text data, the second text intent corresponding to the second text data and the second probability distribution corresponding to the second text intent. The text matching module 404 is used to determine multiple first text data that match the second text data from the corpus database as target text data based on the second text vector, and obtain the first probability distribution of the first text intent corresponding to the target text data. The sample screening module 405 is configured to respectively calculate the probability distribution difference between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data, and screen target sample data from the second text data based on the probability distribution difference.

[0121] It can be understood that the sample data screening apparatus 400 can be applied to a computer device and used to execute the steps of the sample data screening method provided in any embodiment of the present application.

[0122] It should be noted that, those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the sample data screening device 400 described above can refer to the corresponding process in the aforementioned semantic recognition model training method or sample data screening method embodiment, and will not be repeated here.

[0123] like Figure 7 As shown, the computer device 10 includes a processor 11 and a memory 12 , and the processor 11 and the memory 12 are connected via a bus 13 , such as an I 2 C (Inter-integrated Circuit) bus.

[0124] Specifically, the processor 11 is used to provide computing and control capabilities to support the operation of the entire computer device 10. The processor 11 can be a central processing unit (CPU), and the processor 11 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0125] Specifically, the memory 12 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0126] Those skilled in the art will understand that Figure 5 The structure shown in the figure is merely a block diagram of a portion of the structure related to the embodiment of the present application, and does not constitute a limitation on the computer device to which the embodiment of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0127] Among them, the processor 11 is used to run the computer program stored in the memory, and implement any one of the semantic recognition model training methods and / or sample data screening methods provided in the embodiments of the present application when executing the computer program.

[0128] In some embodiments, the processor 11 is configured to run a computer program stored in the memory and implement the following steps when executing the computer program:

[0129] Obtaining a first text data set, obtaining a first text vector corresponding to each first text data in the first text data set, and obtaining a first text intent corresponding to each first text data in the first text data set;

[0130] Constructing a corpus database based on the first text data, the first text vector corresponding to the first text data, and the first text intent;

[0131] Obtaining second text data to be predicted, and classifying the second text data using the initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data, and a second probability distribution corresponding to the second text intent;

[0132] Determining, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data, and obtaining a first probability distribution of the first text intent corresponding to the target text data;

[0133] respectively calculating the probability distribution differences between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data, and filtering out target sample data from the second text data according to the probability distribution differences;

[0134] The initial semantic recognition model is trained using at least the target sample data until a target semantic recognition model is obtained.

[0135] In some implementations, obtaining a first text vector corresponding to each first text data in the first text data set includes:

[0136] Each of the first text data in the first text data set is input as input data into a preset semantic recognition model to obtain a first text vector corresponding to the first text data, wherein the semantic recognition model has the same first network branch and second network branch, and at least some network structure parameters are shared in the first network branch and the second network branch.

[0137] In some embodiments, the first network branch includes a first text vector conversion network and a first pooling network connected to the output end of the first text vector conversion network; the second network branch includes a second text vector conversion network and a second pooling network connected to the output end of the second text vector conversion network; wherein parameters are shared between the first text vector conversion network and the second text vector conversion network.

[0138] In some implementations, determining, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data includes:

[0139] Determining, from the first text vector corresponding to the first text data, a target text vector whose similarity with the second text vector exceeds a preset value;

[0140] The first text data corresponding to the target text vector is used as the target text data.

[0141] In some implementations, filtering out target sample data from the second text data based on the probability distribution difference includes:

[0142] Calculating a probability distribution difference value corresponding to the probability distribution difference using a preset difference calculation function, wherein the difference calculation function is exponentially related to the first probability distribution and the second probability distribution;

[0143] The probability distribution difference values ​​are sorted in size, and the second text data corresponding to the first N probability distribution difference values ​​are selected from the sorted probability distribution difference value queue as target sample data.

[0144] In some embodiments, the training of the initial semantic recognition model using at least the target sample data until a target semantic recognition model is obtained includes:

[0145] Performing data labeling on the target sample data, and updating the first text dataset according to the labeled target sample data to obtain an updated text dataset;

[0146] The initial semantic recognition model is trained using the updated text data set until the loss function of the initial semantic recognition model converges to a preset value, thereby obtaining a target semantic recognition model.

[0147] In some embodiments, the processor 11 is configured to run a computer program stored in the memory and implement the following steps when executing the computer program:

[0148] Obtaining a first text data set, obtaining a first text vector corresponding to each first text data in the first text data set, and obtaining a first text intent corresponding to each first text data in the first text data set;

[0149] Constructing a corpus database based on the first text data, the first text vector corresponding to the first text data, and the first text intent;

[0150] Obtaining second text data to be predicted, and classifying the second text data using the initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data, and a second probability distribution corresponding to the second text intent;

[0151] Determining, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data, and obtaining a first probability distribution of the first text intent corresponding to the target text data;

[0152] The probability distribution differences between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data are calculated respectively, and target sample data are filtered out from the second text data according to the probability distribution differences.

[0153] An embodiment of the present application also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of a semantic recognition model training method and / or the steps of a sample data screening method provided in any embodiment of the present application specification.

[0154] The storage medium may be an internal storage unit of the computer device in the aforementioned embodiment, such as a hard disk or memory of the computer device. The storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device.

[0155] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0156] It should be understood that the term "and / or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0157] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A semantic recognition model training method, characterized in that: The method comprises: Obtaining a first text data set, obtaining a first text vector corresponding to each first text data in the first text data set, and obtaining a first text intent corresponding to each first text data in the first text data set; Constructing a corpus database based on the first text data, the first text vector corresponding to the first text data, and the first text intent; Obtaining second text data to be predicted, and classifying the second text data using the initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data, and a second probability distribution corresponding to the second text intent; Determining, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data, and obtaining a first probability distribution of the first text intent corresponding to the target text data; respectively calculating the probability distribution differences between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data, and filtering out target sample data from the second text data according to the probability distribution differences; At least using the target sample data to train the initial semantic recognition model until a target semantic recognition model is obtained; The method of filtering out target sample data from the second text data according to the probability distribution difference includes: using a preset difference calculation function to calculate the probability distribution difference value corresponding to the probability distribution difference, wherein the difference calculation function KL is , P(x) and Q(x) are probability distributions on the random variable X, P(x) is the first probability distribution corresponding to the text semantics of the first text data stored in the database, and Q(x) is the second probability distribution corresponding to the text semantics of the second text data obtained in a timely manner through the initial semantic recognition model; the probability distribution of the second text data is respectively calculated with the first distribution probability of multiple target text data. The KL score is calculated and averaged to obtain the value score of the sample data to be evaluated, which is the probability distribution difference value, the size of the probability distribution difference value is sorted, and the second text data corresponding to the first N probability distribution difference values ​​are selected from the sorted probability distribution difference value queue as the target sample data.

2. The method according to claim 1, characterized in that The obtaining of the first text vector corresponding to each first text data in the first text data set includes: Each of the first text data in the first text data set is input as input data into a preset semantic recognition model to obtain a first text vector corresponding to the first text data, wherein the semantic recognition model has the same first network branch and second network branch, and at least some network structure parameters are shared in the first network branch and the second network branch.

3. The method according to claim 2, characterized in that The first network branch includes a first text vector conversion network and a first pooling network connected to the output end of the first text vector conversion network; the second network branch includes a second text vector conversion network and a second pooling network connected to the output end of the second text vector conversion network; wherein, parameters are shared between the first text vector conversion network and the second text vector conversion network.

4. The method according to claim 1, wherein Determining, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data, includes: Determining, from the first text vector corresponding to the first text data, a target text vector whose similarity with the second text vector exceeds a preset value; The first text data corresponding to the target text vector is used as the target text data.

5. The method according to claim 1, wherein The step of training the initial semantic recognition model using at least the target sample data until a target semantic recognition model is obtained includes: Performing data labeling on the target sample data, and updating the first text dataset according to the labeled target sample data to obtain an updated text dataset; The initial semantic recognition model is trained using the updated text data set until the loss function of the initial semantic recognition model converges to a preset value, thereby obtaining a target semantic recognition model.

6. A method for screening sample data, characterized in that: include: Obtaining a first text data set, obtaining a first text vector corresponding to each first text data in the first text data set, and obtaining a first text intent corresponding to each first text data in the first text data set; Constructing a corpus database based on the first text data, the first text vector corresponding to the first text data, and the first text intent; Obtaining second text data to be predicted, and classifying the second text data using the initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data, and a second probability distribution corresponding to the second text intent; Determining, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data, and obtaining a first probability distribution of the first text intent corresponding to the target text data; respectively calculating the probability distribution differences between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data, and filtering out target sample data from the second text data according to the probability distribution differences; The method of filtering out target sample data from the second text data according to the probability distribution difference includes: using a preset difference calculation function to calculate the probability distribution difference value corresponding to the probability distribution difference, and the difference calculation function KL is: , P(x) and Q(x) are probability distributions on the random variable X, P(x) is the first probability distribution corresponding to the text semantics of the first text data stored in the database, and Q(x) is the second probability distribution corresponding to the text semantics of the second text data obtained in a timely manner through the initial semantic recognition model; the probability distribution of the second text data is respectively calculated with the first distribution probability of multiple target text data. The KL score is calculated and averaged to obtain the value score of the sample data to be evaluated, which is the probability distribution difference value, the size of the probability distribution difference value is sorted, and the second text data corresponding to the first N probability distribution difference values ​​are selected from the sorted probability distribution difference value queue as the target sample data.

7. A semantic recognition model training device, characterized in that: include: A first processing module is configured to obtain a first text data set, obtain a first text vector corresponding to each first text data in the first text data set, and obtain a first text intent corresponding to each first text data in the first text data set; A database module, configured to construct a corpus database based on the first text data, the first text vector corresponding to the first text data, and the first text intent; a second processing module, configured to obtain second text data to be predicted, and classify the second text data using the initial semantic recognition model to obtain a second text vector corresponding to the second text data, a second text intent corresponding to the second text data, and a second probability distribution corresponding to the second text intent; a text matching module, configured to determine, from the corpus database according to the second text vector, a plurality of first text data that match the second text data as target text data, and obtain a first probability distribution of the first text intent corresponding to the target text data; a sample screening module, configured to respectively calculate a probability distribution difference between the second probability distribution corresponding to each second text data and the first probability distribution of each target text data, and screen target sample data from the second text data based on the probability distribution difference; A model training module, configured to train the initial semantic recognition model using at least the target sample data until a target semantic recognition model is obtained; The method of filtering out target sample data from the second text data according to the probability distribution difference includes: using a preset difference calculation function to calculate the probability distribution difference value corresponding to the probability distribution difference, and the difference calculation function KL is: , P(x) and Q(x) are probability distributions on the random variable X, P(x) is the first probability distribution corresponding to the text semantics of the first text data stored in the database, and Q(x) is the second probability distribution corresponding to the text semantics of the second text data obtained in a timely manner through the initial semantic recognition model; the probability distribution of the second text data is respectively calculated with the first distribution probability of multiple target text data. The KL score is calculated and averaged to obtain the value score of the sample data to be evaluated, which is the probability distribution difference value, the size of the probability distribution difference value is sorted, and the second text data corresponding to the first N probability distribution difference values ​​are selected from the sorted probability distribution difference value queue as the target sample data.

8. A server, characterized in that: Including processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program and implement the semantic recognition model training method or sample data screening method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that When the computer-readable storage medium is executed by one or more processors, the one or more processors execute the steps of the semantic recognition model training method or the sample data screening method according to any one of claims 1 to 6.

Images