Sample data acquisition and model acquisition method, device and equipment, and storage medium

Abstract

The present disclosure provides sample data and model acquisition methods, devices, equipment and storage media, relating to the fields of artificial intelligence such as deep learning, natural language processing and knowledge graph. The sample data acquisition method can include: obtaining seed samples, including: seed corpus and corresponding labels, the seed corpus is the original text corpus selected from the original text corpus set, and the label is the manually annotated label; according to the seed sample, the target text corpus meeting the predetermined requirement is selected from the remaining original text corpus, and the initial expansion sample is generated according to the target text corpus; a knowledge network is constructed based on the original text corpus set, and the secondary expansion sample is generated according to the seed sample, the initial expansion sample and the knowledge network; the seed sample, the initial expansion sample and the secondary expansion sample are used to form a sample set, which is used to train the to-be-trained model by using the samples in the sample set. The present disclosure can reduce the implementation cost and improve the model training effect.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, in particular to a sample data acquisition and model acquisition method, device and equipment, and a storage medium in the fields of deep learning, natural language processing and knowledge graph. BACKGROUND

[0002] The purpose of text data production is to construct specific task data, i.e. sample data, suitable for model training. The traditional text data production method is usually as follows: a large amount of original text corpus is obtained, the original text corpus is standardized according to the specific form of the task, and the standardized text corpus is manually annotated to obtain sample data for model training. However, this method requires a large amount of manpower and time cost, and is low in efficiency. SUMMARY

[0003] The present disclosure provides a sample data acquisition and model acquisition method, device, equipment and storage medium.

[0004] A sample data acquisition method comprises:

[0005] Obtaining a seed sample, which comprises seed corpus and corresponding labels, the seed corpus is selected from the original text corpus set, and the labels are manually annotated labels;

[0006] According to the seed sample, target text corpus meeting predetermined requirements is selected from the remaining original text corpus in the original text corpus set, and initial expansion sample is generated according to the target text corpus;

[0007] Based on the original text corpus set, a knowledge network is constructed, and secondary expansion sample is generated according to the seed sample, the initial expansion sample and the knowledge network;

[0008] The seed sample, the initial expansion sample and the secondary expansion sample are used to form a sample set, which is used to train the to-be-trained model by using the samples in the sample set.

[0009] A model acquisition method comprises:

[0010] acquire a sample set, wherein the sample set comprises a seed sample, an initial expansion sample and a secondary expansion sample, the seed sample comprises a seed corpus and a corresponding label, the seed corpus is selected from an original text corpus, and the label is a manually annotated label, the initial expansion sample is generated according to a target text corpus, the target text corpus is selected from remaining original text corpora in the original text corpus according to the seed sample and meets predetermined requirements, and the secondary expansion sample is generated according to the seed sample, the initial expansion sample and a knowledge network, and the knowledge network is constructed according to the original text corpus set;

[0011] train a to-be-trained model by using samples in the sample set.

[0012] A sample data acquisition device comprises a first acquisition module, a first expansion module, a second expansion module and a sample processing module.

[0013] The first acquisition module is configured to acquire a seed sample, wherein the seed sample comprises a seed corpus and a corresponding label, the seed corpus is selected from an original text corpus, and the label is a manually annotated label.

[0014] The first expansion module is configured to select a target text corpus that meets predetermined requirements from remaining original text corpora in the original text corpus according to the seed sample, and generate an initial expansion sample according to the target text corpus.

[0015] The second expansion module is configured to construct a knowledge network based on the original text corpus set, and generate a secondary expansion sample according to the seed sample, the initial expansion sample and the knowledge network.

[0016] The sample processing module is configured to use the seed sample, the initial expansion sample and the secondary expansion sample to form a sample set, and train a to-be-trained model by using samples in the sample set.

[0017] A model acquisition device comprises a second acquisition module and a model training module.

[0018] The second obtaining module is configured to obtain a sample set, which includes a seed sample, an initial expansion sample, and a secondary expansion sample. The seed sample includes a seed corpus and a corresponding label. The seed corpus is selected from an original text corpus. The label is a manually annotated label. The initial expansion sample is generated according to a target text corpus. The target text corpus is selected from the original text corpus according to the seed sample and meets predetermined requirements. The secondary expansion sample is generated according to the seed sample, the initial expansion sample, and a knowledge network. The knowledge network is constructed according to the original text corpus set.

[0019] The model training module is configured to train a to-be-trained model by using the samples in the sample set.

[0020] An electronic device includes:

[0021] at least one processor; and

[0022] a memory communicatively connected to the at least one processor; wherein

[0023] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0024] A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method described above.

[0025] A computer program product includes computer programs / instructions that, when executed by a processor, implement the method described above.

[0026] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings serve to better understand the present scheme and do not limit the present disclosure. Among them:

[0028] Figure 1 A flowchart of the sample data obtaining method embodiment of the present disclosure;

[0029] Figure 2 A schematic diagram of the relationship among the seed sample, the initial expansion sample, and the secondary expansion sample of the present disclosure;

[0030] Figure 3 A flowchart of a model obtaining method embodiment of the present disclosure is shown.

[0031] Figure 4 A component structure diagram of a sample data obtaining apparatus embodiment 400 of the present disclosure is shown.

[0032] Figure 5 A component structure diagram of a model obtaining apparatus embodiment 500 of the present disclosure is shown.

[0033] Figure 6 A schematic block diagram of an electronic device 600 that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0034] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help in understanding them. These should be considered in their context only. Thus, those of ordinary skill in the art will recognize the various embodiments described herein can be accomplished by combining the teachings of the present disclosure with the appropriate known implementations and practices. Also, for the purpose of clarity and a concise description, descriptions of well-known functions and constructions are omitted from the following description.

[0035] In addition, it should be understood that the term "and / or" as used herein merely describes an associated relationship between associated objects, and can represent three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0036] Figure 1 A flowchart of a sample data obtaining method embodiment of the present disclosure is shown. As shown in Figure 1 the following detailed implementations are included.

[0037] In step 101, a seed sample is obtained, which includes a seed corpus and a corresponding label. The seed corpus is a raw text corpus selected from a raw text corpus set, and the label is a manually annotated label.

[0038] In step 102, according to the seed sample, a target text corpus that meets predetermined requirements is selected from the remaining raw text corpus in the raw text corpus set, and an initial augmented sample is generated according to the target text corpus.

[0039] In step 103, a knowledge network is constructed based on the raw text corpus set, and a secondary augmented sample is generated according to the seed sample, the initial augmented sample, and the knowledge network.

[0040] In step 104, the seed sample, the initial expansion sample, and the secondary expansion sample are used to form a sample set, and the samples in the sample set are used to train the to-be-trained model.

[0041] By using the above method embodiment, a large number of expansion samples can be automatically obtained for model training by only a small amount of manual annotation, thereby saving manpower and time costs, improving processing efficiency, and improving model training effect. In addition, two different expansion methods are used to generate expansion samples, thereby improving the quantity and diversity of expansion samples, and further improving the model training effect.

[0042] There is no limitation on how to obtain the original text corpus. For example, it can be obtained from daily precipitation in a specific industry or from the network at a low cost.

[0043] A large number of original text corpora can be used to form an original text corpus set. In addition, a part of the original text corpora in the original text corpus set can be manually selected, which can be randomly selected. The number of selected original text corpora can be much smaller than the number of original text corpora in the original text corpus set, and the selected original text corpora can be manually annotated, i.e., to generate their labels. Compared with full manual annotation, small-scale annotation can greatly reduce manual cost.

[0044] Based on the seed sample, target text corpora that meet predetermined requirements can be selected from the remaining original text corpora in the original text corpus set, and initial expansion samples can be generated according to the target text corpora.

[0045] Preferably, for each seed sample, the following processing can be performed: the plaintext similarity and the semantic similarity between the seed corpus in the seed sample and each remaining original text corpus are obtained, and in response to determining that any original text corpus meets the following requirements: at least one of the plaintext similarity and the semantic similarity between the seed corpus is greater than the corresponding threshold value, the original text corpus is used as a target text corpus.

[0046] The corresponding thresholds of the plaintext similarity and the semantic similarity can be the same or different, and the specific values can be determined according to actual needs.

[0047] Taking the seed corpus a and the original text corpus b as examples, the similarity of the plaintext before the seed corpus a and the original text corpus b can be obtained, and the semantic similarity between the seed corpus a and the original text corpus b can be obtained, and then the obtained plaintext similarity and semantic similarity can be compared with the corresponding threshold respectively, if the plaintext similarity is greater than the corresponding threshold, or if the semantic similarity is greater than the corresponding threshold, or if the plaintext similarity and the semantic similarity are greater than the corresponding threshold respectively, then the original text corpus b can be selected as the original text corpus, that is, as the target text corpus.

[0048] The plaintext similarity refers to the similarity between the plaintext information of two text corpora, and the semantic similarity refers to the similarity between the text semantic information of two text corpora. The calculation of the similarity is not limited, for example, a mature calculation method can be used.

[0049] Preferably, for any target text corpus, the following processing can be performed respectively: taking the label of the seed corpus corresponding to the target text corpus as the label of the target text corpus, and using the target text corpus and its label to form an initial expansion sample. The seed corpus corresponding to the target text corpus is the seed corpus used to select the target text corpus.

[0050] Still taking the seed corpus a and the original text corpus b (target text corpus) as examples, the seed corpus a is the seed corpus corresponding to the original text corpus b, and the label of the seed corpus a can be taken as the label of the original text corpus b, and then an initial expansion sample can be formed using the original text corpus b and its label.

[0051] Through the above processing, coarse-grained sample expansion can be achieved, that is, the target text corpus can be found from the original text corpus set by using the seed corpus through similarity matching, and the initial expansion sample can be generated according to the target text corpus and the corresponding label, thereby achieving preliminary sample expansion. In addition, different similarity matching methods such as plaintext similarity and semantic similarity can be used to improve the comprehensiveness of the found target text corpus.

[0052] In addition, the knowledge network can also be constructed using the original text corpus set, and the secondary expansion sample can be generated according to the seed sample, the initial expansion sample and the knowledge network. The original text corpus set described here refers to the original text corpus set including all original text corpora (including seed corpora and target text corpora).

[0053] Preferably, knowledge recognition can be performed on each original text corpus in the original text corpus set, that is, knowledge mining can be performed, and then the knowledge network can be constructed according to the recognized knowledge.

[0054] The identified knowledge can include entities and facets, which are core knowledge representing the content of the text corpus, i.e., entity recognition and facet recognition can be performed. For example, a certain movie is an entity, and attribute information of the movie is a facet.

[0055] According to the association relationship between the identified knowledge, a knowledge network, i.e., a knowledge network graph, can be constructed, and then the secondary expansion sample required can be generated according to the seed sample, the initial expansion sample, and the knowledge network.

[0056] Preferably, for any seed sample, the following processing can be performed respectively: determining similar knowledge of the knowledge identified from the seed corpus in the seed sample according to the knowledge network, replacing the corresponding knowledge in the seed corpus with the similar knowledge, taking the replacement result as the secondary expansion corpus, and generating the secondary expansion sample according to the secondary expansion corpus. The similar knowledge is the knowledge directly connected through the edge. Similarly, for any initial expansion sample, the following processing can be performed respectively: determining similar knowledge of the knowledge identified from the target text corpus in the initial expansion sample according to the knowledge network, replacing the corresponding knowledge in the target text corpus with the similar knowledge, taking the replacement result as the secondary expansion corpus, and generating the secondary expansion sample according to the secondary expansion corpus.

[0057] Taking the seed corpus a as an example, it is assumed that there are two entities in the seed corpus a, which are entity x and entity y. For each entity, similar knowledge thereof can be determined from the knowledge network, i.e., the knowledge directly connected with the entity x through the edge (assuming entity z) and the knowledge directly connected with the entity y through the edge (assuming entity w), and the entity z and the entity w can be used to replace the entity x and the entity y respectively, so as to obtain the secondary expansion corpus corresponding to the seed corpus a, which can include the secondary expansion corpus after the entity x is replaced by the entity z, the secondary expansion corpus after the entity y is replaced by the entity w, and the secondary expansion corpus after the entity x and the entity y are replaced by the entity z and the entity w respectively.

[0058] Preferably, for the seed sample, the secondary expansion sample can be generated according to the secondary expansion corpus in the following manner: taking the label of the seed corpus as the label of the secondary expansion corpus, and using the secondary expansion corpus and the label thereof to form the secondary expansion sample. Similarly, for the initial expansion sample, the secondary expansion sample can be generated according to the secondary expansion corpus in the following manner: taking the label of the target text corpus as the label of the secondary expansion corpus, and using the secondary expansion corpus and the label thereof to form the secondary expansion sample. That is, the label of the text corpus before replacement can be taken as the label of the secondary expansion corpus obtained after replacement.

[0059] Through the above processing, fine-grained sample expansion can be realized, that is, fine-grained related knowledge in the text corpus is fully utilized, a relatively complete knowledge network is constructed based on the association relationship between the knowledge, and knowledge searching and replacing are performed according to the similarity between the knowledge in the knowledge network, so that further sample expansion is realized.

[0060] In combination with the above description, Figure 2 A schematic diagram of the relationship between the seed sample, the initial expansion sample, and the secondary expansion sample is shown in the figure, and specific implementation can be referred to the foregoing related description.

[0061] Further, the seed sample, the initial expansion sample, and the secondary expansion sample can be used to form a sample set, and the samples in the sample set can be used to train the to-be-trained model.

[0062] Correspondingly, Figure 3 A flowchart of the model obtaining method embodiment of the present disclosure is shown in FIG. 3. Figure 3 As shown in FIG. 3, the following specific implementation is included.

[0063] In step 301, a sample set is obtained, which includes a seed sample, an initial expansion sample, and a secondary expansion sample. The seed sample includes seed corpus and corresponding labels. The seed corpus is selected from the original text corpus set, and the labels are manually annotated. The initial expansion sample is generated according to a target text corpus. The target text corpus is selected from the remaining original text corpus in the original text corpus set according to the seed sample and meets predetermined requirements. The secondary expansion sample is generated according to the seed sample, the initial expansion sample, and a knowledge network. The knowledge network is constructed according to the original text corpus set.

[0064] In step 302, the samples in the sample set are used to train the to-be-trained model.

[0065] By using the above method embodiment, a large number of expansion samples can be automatically obtained by only a small amount of manual annotation, which can be used for model training, thereby saving manpower and time cost, improving processing efficiency, and improving model training effect. In addition, two different expansion methods are used to generate expansion samples, thereby improving the quantity and diversity of the expansion samples, and further improving the model training effect.

[0066] The generation method of each sample in the sample set can be referred to the foregoing related description, and will not be described herein. Further, the samples in the sample set can be used to train the to-be-trained model.

[0067] Preferably, before training the to-be-trained model by using the samples in the sample set, the labels of the samples in the sample set can also be predicted by the seed classifier respectively. In response to determining that the label of any sample is inconsistent with the predicted label, the sample can be determined as a strong negative example sample, and the seed classifier is obtained by training the seed sample.

[0068] Through the above processing, strong negative example samples can be determined, and strong negative example samples can enable the model to mine deeper internal relationships during training, prompting the model to fit the current scene sample more carefully, thereby improving the model training effect.

[0069] In addition, since the samples obtained according to the method of the present disclosure are mainly automatically constructed, there is a lack of human participation in auditing and screening, so there may be some noise samples. In order to further construct more pure samples, the following optimization processing method can be used.

[0070] Preferably, before training the to-be-trained model by using the samples in the sample set, the labels of the samples in the sample set can also be predicted by the seed classifier respectively. In response to determining that the label of any sample is inconsistent with the predicted label, the sample can be determined as a strong negative example sample, and the seed classifier is obtained by training the seed sample.

[0071] For example, for any sample in the sample set, the sample can be scored by using a pre-trained scoring model respectively. In response to determining that the score is less than a predetermined threshold, the sample can be determined as a noise sample. The specific value of the threshold can be determined according to actual needs.

[0072] The scoring model can be pre-trained, and the scoring model can be used to score the sample to evaluate the possibility of being a noise sample.

[0073] Through the above method, coarse filtering of noise samples can be realized, so that some obvious noise samples can be removed, and the model training effect can be further improved.

[0074] In addition, preferably, when training the to-be-trained model, a first loss and a second loss can be generated in each round of training process respectively. The first loss is generated according to the sample input into the to-be-trained model, and the sample input into the to-be-trained model is the sample in the sample set. The second loss is generated according to the constructed interpolation sample, and the interpolation sample is obtained by linearly interpolating the sample input into the to-be-trained model. The to-be-trained model can be updated by combining the first loss and the second loss.

[0075] Through linear interpolation processing, a batch of sample data between normal samples and noise samples can be constructed to alleviate the influence of data on noise.

[0076] For example, the text corpus of a sample is xp , label p is 1, and the text corpus of another sample is x q , label q is 0 (assuming that the label is wrong, and the correct label should be 1), and the text corpus x after linear interpolation is:

[0077] x = σx p + (1-σ)x q ; (1)

[0078] wherein σ can be randomly sampled from a beta distribution, and can be in the range of [0, 1].

[0079] The label can also be processed in the same way. Assuming that σ is 0.5, then the label after linear interpolation is 0.5*1+(1-0.5)*0 = 0.5, so that an interpolated sample can be obtained, whose text corpus is x obtained according to formula (1), and the label is 0.5.

[0080] Through linear interpolation, the influence of noise samples on model training is neutralized, which indirectly plays a role in sample denoising, and further improves the model training effect.

[0081] For example, assuming that the samples input into the model to be trained are sample 1 to sample 30, these samples belong to the same batch, then a first loss can be generated according to the processing result of the model to be trained on these samples, and the model can perform linear interpolation on these samples two by two (linearly interpolate each sample with each of the other samples except itself), thereby obtaining a plurality of interpolated samples, and a second loss can be generated according to the processing result of the model to be trained on these interpolated samples, and then the first loss and the second loss can be combined to update the parameters of the model to be trained, i.e., gradient update.

[0082] Preferably, the second loss can be a cross-entropy loss, and the first loss can be a loss optimized according to the principle of improving the correction ability of the model to noise samples.

[0083] wherein the second loss, i.e., the traditional cross-entropy loss l(θ), can be as follows:

[0084]

[0085] wherein N represents the number of samples in the same batch, h i represents the model prediction value (in the form of probability value), y i represents the label of the sample.

[0086] The traditional cross-entropy loss is not suitable for the case where there are many noisy samples, and the loss will cause serious overfitting to the noisy samples in the later training stage.

[0087] Therefore, the traditional cross-entropy loss is optimized in the scheme of the present disclosure, so that the first loss lb shown below can be obtained:

[0088]

[0089] where w i is a newly introduced parameter, which can be an empirical value, z i represents the model prediction value (non-probability value, such as a specific classification result).

[0090] By introducing w i , the original label y i is abandoned to some extent, and the guidance of the model is abandoned, and instead, the prediction value z i of the model is used as the guidance for subsequent updates. This processing has no effect on normal samples, but can alleviate the negative impact of incorrect labels on noisy samples.

[0091] As can be seen, in the above processing method, multiple denoising methods are used, that is, a large model score, linear interpolation, and loss optimization are used to filter out more pure training samples, thereby alleviating the problem of noisy samples generated by automatic generation of samples, that is, the influence of noisy samples on model training is removed as much as possible, thereby improving the model training effect.

[0092] It should be noted that for the foregoing method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the present disclosure is not limited by the order of the described actions, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present disclosure. In addition, the parts not described in detail in a certain embodiment can be referred to the related description in other embodiments.

[0093] In summary, by using the scheme of the method embodiments of the present disclosure, only a small amount of human participation is required to label seed samples, and then a large number of expanded samples can be automatically obtained through various expansion methods for model training, thereby saving human and time costs, improving processing efficiency, and improving model training effect. Moreover, multiple denoising methods can be used to improve the purity of the samples, thereby further improving the model training effect, etc.

[0094] In addition, the scheme of the method embodiment of the present disclosure can be applied to different scenarios, such as text classification and text search scenarios, and has wide applicability.

[0095] Taking a text sentiment classification scenario in text classification as an example, the to-be-trained model can be a text sentiment classification model. The classification can be binary classification or multi-classification (such as three-classification, etc.). Correspondingly, the labels of the text corpus refer to classification results. For example, the labels of binary classification can include positive sentiment and negative sentiment, which can be represented by 1 and 0 respectively. For example, the labels of three-classification can include happy, sad, and calm.

[0096] The above is an introduction to the method embodiment. The following describes the scheme of the present disclosure through a device embodiment.

[0097] Figure 4 FIG. 4 is a schematic diagram of the composition structure of the sample data acquisition device embodiment 400 of the present disclosure. As shown in FIG. 4, the device includes a first acquisition module 401, a first expansion module 402, a second expansion module 403, and a sample processing module 404. Figure 4

[0098] The first acquisition module 401 is configured to acquire a seed sample, which includes seed corpus and corresponding labels. The seed corpus is selected from the original text corpus set, and the labels are manually annotated labels.

[0099] The first expansion module 402 is configured to select target text corpus that meets predetermined requirements from the remaining original text corpus in the original text corpus set according to the seed sample, and generate an initial expansion sample according to the target text corpus.

[0100] The second expansion module 403 is configured to construct a knowledge network based on the original text corpus set, and generate a secondary expansion sample according to the seed sample, the initial expansion sample, and the knowledge network.

[0101] The sample processing module 404 is configured to use the seed sample, the initial expansion sample, and the secondary expansion sample to form a sample set, and use the samples in the sample set to train the to-be-trained model.

[0102] The scheme of the device embodiment described above can automatically obtain a large number of expansion samples for model training by only a small amount of manual annotation, thereby saving manpower and time costs, improving processing efficiency, and improving model training effect. In addition, two different expansion methods are used to generate expansion samples, thereby improving the number and diversity of expansion samples, and further improving the model training effect.

[0103] ​The part of the original text corpus can be selected from the original text corpus manually, and the selection can be random selection. The number of the selected original text corpus can be much smaller than the number of the original text corpus in the original text corpus set. The selected original text corpus can be manually labeled, i.e., the label thereof is generated.

[0104] Based on the seed sample, the first expansion module 402 can select target text corpus meeting predetermined requirements from the remaining original text corpus in the original text corpus set, and can generate an initial expansion sample according to the target text corpus.

[0105] Preferably, for each seed sample, the first expansion module 402 can respectively perform the following processing: obtaining the plaintext similarity and the semantic similarity between the seed corpus in the seed sample and each remaining original text corpus, and determining any original text corpus as a target text corpus in response to determining that the original text corpus meets the following requirement: at least one of the plaintext similarity and the semantic similarity between the original text corpus and the seed corpus is greater than a corresponding threshold.

[0106] Preferably, for any target text corpus, the first expansion module 402 can respectively perform the following processing: taking the label of the seed corpus corresponding to the target text corpus as the label of the target text corpus, and using the target text corpus and the label thereof to form an initial expansion sample. The seed corpus corresponding to the target text corpus is the seed corpus used to select the target text corpus.

[0107] In addition, the second expansion module 403 can construct a knowledge network based on the original text corpus set, and can generate a secondary expansion sample according to the seed sample, the initial expansion sample, and the knowledge network.

[0108] Preferably, the second expansion module 403 can respectively perform knowledge recognition, i.e., knowledge mining, on each original text corpus in the original text corpus set, and can construct a knowledge network according to the recognized knowledge.

[0109] Correspondingly, preferably, for any seed sample, the second expansion module 403 can respectively perform the following processing: determining similar knowledge of the knowledge recognized from the seed corpus in the seed sample according to the knowledge network, replacing the corresponding knowledge in the seed corpus with the similar knowledge, taking the replacement result as a secondary expansion corpus, and generating a secondary expansion sample according to the secondary expansion corpus. The similar knowledge is knowledge directly connected by an edge. Similarly, for any initial expansion sample, the second expansion module 403 can respectively perform the following processing: determining similar knowledge of the knowledge recognized from the target text corpus in the initial expansion sample according to the knowledge network, replacing the corresponding knowledge in the target text corpus with the similar knowledge, taking the replacement result as a secondary expansion corpus, and generating a secondary expansion sample according to the secondary expansion corpus.

[0110] Preferably, for the seed sample, the second expansion module 403 can generate the secondary expansion sample according to the secondary expansion corpus in the following manner: taking the label of the seed corpus as the label of the secondary expansion corpus, and using the secondary expansion corpus and the label thereof to form the secondary expansion sample. Similarly, for the initial expansion sample, the second expansion module 403 can generate the secondary expansion sample according to the secondary expansion corpus in the following manner: taking the label of the target text corpus as the label of the secondary expansion corpus, and using the secondary expansion corpus and the label thereof to form the secondary expansion sample. That is, the label of the text corpus before replacement can be taken as the label of the secondary expansion corpus obtained after replacement.

[0111] Figure 5 A constituent structure schematic diagram of the model obtaining device embodiment 500 of the present disclosure is shown in FIG. 5. As shown in FIG. 5, the model obtaining device embodiment 500 of the present disclosure comprises a second obtaining module 501 and a model training module 502. Figure 5

[0112] The second obtaining module 501 is configured to obtain a sample set, which comprises a seed sample, an initial expansion sample, and a secondary expansion sample. The seed sample comprises a seed corpus and a corresponding label. The seed corpus is an original text corpus selected from an original text corpus set. The label is a manually annotated label. The initial expansion sample is a sample generated according to a target text corpus. The target text corpus is an original text corpus that meets predetermined requirements and is selected from the original text corpus set according to the seed sample. The secondary expansion sample is a sample generated according to the seed sample, the initial expansion sample, and a knowledge network. The knowledge network is a knowledge network constructed according to the original text corpus set.

[0113] The model training module 502 is configured to train a to-be-trained model using the samples in the sample set.

[0114] By using the above-mentioned device embodiment and scheme, a large number of expansion samples can be automatically obtained with only a small amount of manual annotation, which can be used for model training, thereby saving manpower and time costs, improving processing efficiency, and improving model training effect. Moreover, two different expansion methods are used to generate expansion samples, thereby improving the quantity and diversity of expansion samples, and further improving the model training effect, etc.

[0115] Preferably, before training the to-be-trained model using the samples in the sample set, the model training module 502 can further predict the label of each sample in the sample set using a seed classifier. In response to determining that the label of any sample is inconsistent with the predicted label, the sample can be determined as a strong negative example sample. The seed classifier is trained using the seed sample.

[0116] ​Preferably, the model training module 502 can further identify each sample in the sample set as a noise sample before training the to-be-trained model with the samples in the sample set, and filter the samples identified as noise samples from the sample set.

[0117] For example, for any sample in the sample set, the pre-trained scoring model can be used to score the sample, and the sample can be determined as a noise sample in response to determining that the score is less than a predetermined threshold.

[0118] In addition, preferably, the model training module 502 can generate a first loss and a second loss in each round of training when training the to-be-trained model, the first loss is generated according to the samples input to the to-be-trained model, the samples input to the to-be-trained model are the samples in the sample set, and the second loss is generated according to the interpolated samples, the interpolated samples are the samples obtained by linearly interpolating the samples input to the to-be-trained model two by two, and then the to-be-trained model can be updated in combination with the first loss and the second loss.

[0119] Preferably, the second loss can be a cross-entropy loss, and the first loss can be a loss obtained by optimizing the cross-entropy loss according to the principle of improving the correction ability of the boosting model for noise samples.

[0120] Figure 4 and Figure 5 The specific working process of the device embodiment shown can refer to the related description in the foregoing method embodiment, and will not be described here.

[0121] In summary, by using the scheme of the device embodiment of the present disclosure, only a small amount of manual participation is required to label seed samples, and subsequently a large number of expanded samples can be automatically obtained by various expansion methods for model training, thereby saving manpower and time cost, improving processing efficiency and model training effect, and further improving the model training effect by using multiple denoising methods to improve the purity of the samples. In addition, it can be applied to different scenarios such as text classification and text search, and has wide applicability.

[0122] The scheme of the present disclosure can be applied to the field of artificial intelligence, and particularly relates to the fields of deep learning, natural language processing, and knowledge graph. Artificial intelligence is a discipline that studies enabling computers to simulate some thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.) of humans, and includes both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0123] The text corpus in the embodiments of the present disclosure is not for a specific user and cannot reflect the personal information of a specific user. In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solutions comply with relevant laws and regulations and do not violate public order and good customs.

[0124] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0125] Figure 6 A schematic block diagram of an electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, servers, blades, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0126] As shown in Figure 6 The device 600 includes a computing unit 601 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the device 600 can also be stored in the RAM 603. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0127] Various components in the device 600 are connected to the I / O interface 605, including an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0128] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, and the like. The computing unit 601 performs various methods and processes described above, such as the methods described in the present disclosure. For example, in some embodiments, the methods described in the present disclosure can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded onto the RAM 603 and executed by the computing unit 601, one or more steps of the methods described in the present disclosure can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the methods described in the present disclosure by any other suitable means, such as by means of firmware.

[0129] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0130] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0131] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0133] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0134] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0135] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technology disclosed in the present disclosure are achieved, which is not limited herein.

[0136] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for acquiring sample data, comprising: acquiring a seed sample, wherein the seed sample comprises seed corpora and corresponding labels, the seed corpora are selected from original text corpora, and the labels are manually annotated labels; selecting target text corpora that meet predetermined requirements from the remaining original text corpora in the original text corpus set according to the seed sample, and generating an initial augmented sample according to the target text corpora; performing knowledge recognition on each original text corpus in the original text corpus set respectively, and constructing a knowledge network according to the recognized knowledge; for any seed sample, the following processing is performed: determining similar knowledge of the knowledge recognized from the seed corpora in the seed sample according to the knowledge network, replacing the corresponding knowledge in the seed corpora with the similar knowledge, taking the replacement result as a secondary augmented corpus, and generating a secondary augmented sample according to the secondary augmented corpus; and for any initial augmented sample, the following processing is performed: determining similar knowledge of the knowledge recognized from the target text corpora in the initial augmented sample according to the knowledge network, replacing the corresponding knowledge in the target text corpora with the similar knowledge, taking the replacement result as the secondary augmented corpus, and generating the secondary augmented sample according to the secondary augmented corpus; using the seed sample, the initial augmented sample, and the secondary augmented sample to form a sample set, and using the samples in the sample set to train a to-be-trained model.

2. The method of claim 1, wherein, The selecting of the target text corpora that meet predetermined requirements from the remaining original text corpora in the original text corpus set according to the seed sample comprises: for each seed sample, the following processing is performed: obtaining the plaintext similarity and the semantic similarity between the seed corpora in the seed sample and each remaining original text corpus respectively; in response to determining that any original text corpus meets the following requirement: at least one of the plaintext similarity and the semantic similarity between the original text corpus and the seed corpora is greater than a corresponding threshold, taking the original text corpus as the target text corpus.

3. The method of claim 2, wherein, The generating of the initial augmented sample according to the target text corpora comprises: for any target text corpus, the following processing is performed: taking the label of the seed corpus corresponding to the target text corpus as the label of the target text corpus, and using the target text corpus and its label to form the initial augmented sample, wherein the seed corpus corresponding to the target text corpus is the seed corpus used to select the target text corpus. 4.The method of claim 1, wherein for the seed sample, the generating of the secondary augmented sample according to the secondary augmented corpus comprises: taking the label of the seed corpus as the label of the secondary augmented corpus, and using the secondary augmented corpus and its label to form the secondary augmented sample. According to the initial expansion sample, the generating the secondary expansion sample according to the secondary expansion corpus comprises: taking the label of the target text corpus as the label of the secondary expansion corpus, and using the secondary expansion corpus and the label to form the secondary expansion sample.

5. A model obtaining method, comprising: obtaining a sample set, the sample set being obtained by the method according to any one of claims 1-4; training a to-be-trained model using samples in the sample set.

6. The method of claim 5, further comprising: before the training of the to-be-trained model using samples in the sample set, predicting labels of each sample in the sample set using a seed classifier, respectively, and in response to determining that the label of any sample is inconsistent with the predicted label, determining the sample as a strong negative example sample, the seed classifier being trained using the seed sample.

7. The method of claim 5, further comprising: before the training of the to-be-trained model using samples in the sample set, respectively, performing noise identification on each sample in the sample set, and filtering out samples identified as noise samples from the sample set.

8. The method of any one of claims 5-7, wherein, the training of the to-be-trained model using samples in the sample set comprises: in each round of training, respectively generating a first loss and a second loss, the first loss being generated according to samples input to the to-be-trained model, the samples input to the to-be-trained model being samples in the sample set, the second loss being generated according to constructed interpolation samples, the interpolation samples being samples obtained by linear interpolation of two-by-two of the samples input to the to-be-trained model; updating parameters of the to-be-trained model in combination with the first loss and the second loss.

9. The method of claim 8, wherein: the second loss is a cross-entropy loss, and the first loss is a loss obtained by optimizing the cross-entropy loss according to a principle of correcting ability of a boosting model for noise samples.

10. A sample data acquisition apparatus comprising: a first obtaining module, a first expansion module, a second expansion module, and a sample processing module; the first obtaining module is configured to obtain a seed sample, wherein the seed sample includes a seed corpus and a corresponding label, the seed corpus is an original text corpus selected from an original text corpus set, and the label is a manually annotated label; the first expansion module is configured to select a target text corpus meeting a predetermined requirement from the original text corpora remaining in the original text corpus set according to the seed sample, and generate an initial expansion sample according to the target text corpus; the second expansion module is configured to generate a secondary expansion sample according to a secondary expansion corpus, the secondary expansion corpus being generated by expanding the initial expansion sample. The second expansion module is configured to perform the following processing on each original text corpus in the original text corpus set: performing knowledge recognition on each original text corpus, and constructing a knowledge network according to the recognized knowledge; for each sub-sample, performing the following processing: determining similar knowledge of the knowledge recognized from the seed corpus in the seed sample according to the knowledge network, replacing corresponding knowledge in the seed corpus by using the similar knowledge, taking the replacement result as a secondary expansion corpus, generating a secondary expansion sample according to the secondary expansion corpus, and the similar knowledge is knowledge directly connected by an edge; and for each initial expansion sample, performing the following processing: determining similar knowledge of the knowledge recognized from the target text corpus in the initial expansion sample according to the knowledge network, replacing corresponding knowledge in the target text corpus by using the similar knowledge, taking the replacement result as the secondary expansion corpus, and generating the secondary expansion sample according to the secondary expansion corpus. The sample processing module is configured to use the seed sample, the initial expansion sample and the secondary expansion sample to form a sample set, and use the samples in the sample set to train a to-be-trained model.

11. The apparatus of claim 10, wherein, The first expansion module is configured to perform the following processing on each sub-sample: acquiring a plaintext similarity and a semantic similarity between the seed corpus in the seed sample and each original text corpus, and in response to determining that any original text corpus meets the following requirement: at least one of the plaintext similarity and the semantic similarity between the original text corpus and the seed corpus is greater than a corresponding threshold value, taking the original text corpus as the target text corpus.

12. The apparatus of claim 11, wherein, The first expansion module is configured to perform the following processing on each target text corpus: taking a label of the target text corpus corresponding seed corpus as a label of the target text corpus, and using the target text corpus and the label to form an initial expansion sample, and the target text corpus corresponding seed corpus is a seed corpus used to select the target text corpus.

13. The apparatus of claim 10, wherein, For the seed sample, the second expansion module takes a label of the seed corpus as a label of the secondary expansion corpus, and uses the secondary expansion corpus and the label to form the secondary expansion sample; and for the initial expansion sample, the second expansion module takes a label of the target text corpus as a label of the secondary expansion corpus, and uses the secondary expansion corpus and the label to form the secondary expansion sample. A second acquisition module and a model training module; 14. A model acquisition apparatus comprising: The second acquisition module is configured to acquire a sample set, and the sample set is acquired by the method in any one of claims 1-4. The model training module is configured to train a to-be-trained model by using samples in the sample set.

15. The apparatus of claim 14, wherein, ​ The model training module is further configured to, before training the to-be-trained model using the samples in the sample set, predict labels of the samples in the sample set respectively using a seed classifier trained using the seed samples, and determine any sample as a strong negative example sample in response to determining that the label of the sample is inconsistent with the predicted label. 16.The apparatus of claim 14, wherein, The model training module is further configured to, before training the to-be-trained model using the samples in the sample set, perform noise identification on the samples in the sample set respectively, and filter out the samples identified as noise samples from the sample set. 17.The apparatus of any one of claims 14-16, wherein, The model training module generates a first loss and a second loss respectively in each round of training, the first loss is generated according to the samples input to the to-be-trained model, the samples input to the to-be-trained model are the samples in the sample set, the second loss is generated according to the interpolation samples, the interpolation samples are the samples obtained by linear interpolation of the samples input to the to-be-trained model two by two, and the to-be-trained model is updated in combination with the first loss and the second loss. 18.The apparatus of claim 17, wherein, The second loss is a cross-entropy loss, and the first loss is a loss obtained by optimizing the cross-entropy loss according to the principle of the correction ability of the boosting model for noise samples. 19.An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein, 20. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9. The computer instructions are used to enable the computer to perform the method of any one of claims 1-9. 21.A computer program product comprising computer programs / instructions, which, when executed by a processor, implement the method of any one of claims 1-9.

Images