Noise sample identification method, device, electronic device and storage medium

Abstract

The present disclosure provides a noise sample recognition method, device, electronic device and storage medium, which relates to the fields of artificial intelligence technology such as natural language processing and deep learning. The specific implementation scheme is: obtaining an original sample set; using a preset adjustment rule to add a mask to the original training corpus in the original sample set to obtain a target sample set; using a pre-trained language model to perform mask prediction on multiple target training corpora in the target sample set to obtain the first mask prediction character corresponding to each target training corpus; matching the first mask prediction character corresponding to each target training corpus with a preset condition; according to the target training corpus whose first mask prediction character in the target sample set does not match the preset condition, determining that the corresponding original training corpus in the original sample set is a noise sample. It can efficiently and accurately identify noise samples from the original sample set.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the field of artificial intelligence technologies such as natural language processing and deep learning, and more particularly to a method, device, electronic device, and storage medium for identifying noise samples. Background Art

[0002] In natural language processing applications, the quality of training samples plays a crucial role. Large-scale, high-quality training samples often lead to the development of effective application models, such as deep learning models like neural machine translation. However, in practice, high-quality training samples are generally difficult to obtain, creating significant obstacles in the practical application of these methods. Summary of the Invention

[0003] The present disclosure provides a noise sample recognition method, device, electronic device, and storage medium.

[0004] According to one aspect of the present disclosure, a method for identifying noise samples is provided, comprising: obtaining an original sample set; using a preset adjustment rule to add a mask to an original training corpus in the original sample set to obtain a target sample set; using a pre-trained language model to perform mask prediction on multiple target training corpora in the target sample set to obtain a first masked predicted character corresponding to each target training corpus; matching the first masked predicted character corresponding to each target training corpus with a preset condition; and determining, based on a target training corpus in the target sample set whose first masked predicted character does not match the preset condition, that the corresponding original training corpus in the original sample set is a noise sample.

[0005] According to another aspect of the present disclosure, a model training method for noise sample recognition is provided, comprising: obtaining an original sample set, wherein each original training corpus in the original sample set has a corresponding annotation; using a preset adjustment rule to add a mask to the original training corpus in the original sample set to obtain a target sample set; using an initial language model to perform mask prediction on multiple target training corpora in the target sample set to obtain a second masked predicted character corresponding to each target training corpus; adjusting model parameters of the initial language model based on the difference between the second masked predicted character corresponding to each target training corpus and the annotation corresponding to each original training corpus; and determining the language model obtained after adjustment as the language model for noise sample recognition.

[0006] According to another aspect of the present disclosure, a noise sample recognition device is provided, comprising: a first acquisition module for acquiring an original sample set; a first processing module for adding a mask to an original training corpus in the original sample set using a preset adjustment rule to obtain a target sample set; a first prediction module for performing mask prediction on a plurality of target training corpora in the target sample set using a pre-trained language model to obtain a first masked predicted character corresponding to each of the target training corpora; a matching module for matching the first masked predicted character corresponding to each of the target training corpora with a preset condition; and a first determination module for determining that the corresponding original training corpus in the original sample set is a noise sample based on a target training corpus whose first masked predicted character in the target sample set does not match the preset condition.

[0007] According to another aspect of the present disclosure, a model training device for noise sample recognition is provided, comprising: a second acquisition module for acquiring an original sample set, wherein each original training corpus in the original sample set has a corresponding annotation; a second processing module for adding a mask to the original training corpus in the original sample set using a preset adjustment rule to obtain a target sample set; a second prediction module for performing mask prediction on multiple target training corpora in the target sample set using an initial language model to obtain a second masked predicted character corresponding to each target training corpus; an adjustment module for adjusting model parameters of the initial language model based on the difference between the second masked predicted character corresponding to each target training corpus and the annotation corresponding to each original training corpus; and a fifth determination module for determining the language model obtained after adjustment as the language model for noise sample recognition.

[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the noise sample recognition method described in the first aspect above, or to execute the model training method for noise sample recognition described in the second aspect above.

[0009] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the noise sample recognition method described in the first aspect above, or to execute the model training method for noise sample recognition described in the second aspect above.

[0010] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program. When the computer program is executed by a processor, the computer program implements the steps of the noise sample identification method described in the first aspect above, or implements the steps of the model training method for noise sample identification described in the second aspect above.

[0011] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0013] Figure 1 is a flowchart of a noise sample recognition method according to the first embodiment of the present disclosure;

[0014] Figure 2 is a flowchart of a noise sample recognition method according to the second embodiment of the present disclosure;

[0015] Figure 3 is a flowchart of a noise sample recognition method according to the third embodiment of the present disclosure;

[0016] Figure 4 is a flowchart of a model training method for noise sample recognition according to the fourth embodiment of the present disclosure;

[0017] Figure 5 is a flowchart of a model training method for noise sample recognition according to the fifth embodiment of the present disclosure;

[0018] Figure 6 is a structural diagram of a noise sample recognition device according to a sixth embodiment of the present disclosure;

[0019] Figure 7 is a structural diagram of a noise sample recognition device according to a seventh embodiment of the present disclosure;

[0020] Figure 8 is a structural diagram of a model training device for noise sample recognition according to an eighth embodiment of the present disclosure;

[0021] Figure 9 is a structural diagram of a model training device for noise sample recognition according to a ninth embodiment of the present disclosure;

[0022] Figure 10 It is a block diagram of an electronic device used to implement the noise sample recognition method or the model training method for noise sample recognition according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0024] The present disclosure relates to the field of computer technology, and in particular to the field of artificial intelligence technologies such as natural language processing and deep learning.

[0025] The following is a brief description of the technical field involved in the solution of the present disclosure:

[0026] Artificial Intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0027] NLP (Natural Language Processing) is the technology used to communicate with computers using natural language. Because the key to processing natural language is to make computers "understand" it, natural language processing is also called natural language understanding or computational linguistics. It is both a branch of language information processing and a core area of ​​artificial intelligence.

[0028] Deep learning (DL) involves learning the inherent patterns and representational hierarchies of sample data. The information gained during this learning process is highly helpful in interpreting data such as text, images, and sound. Its ultimate goal is to enable machines to acquire human-like analytical learning capabilities and recognize data such as text, images, and sound. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition that far surpass previous technologies. Deep learning has achieved significant results in search technology, data mining, machine learning, machine translation, natural language processing, multimedia learning, speech, recommendation and personalization technologies, and other related fields. Deep learning enables machines to mimic human activities such as seeing, hearing, and thinking, solving many complex pattern recognition challenges and significantly advancing AI-related technologies.

[0029] Currently, the quality of training samples plays a crucial role in natural language processing applications. Large-scale, high-quality training samples often lead to the development of effective application models, such as deep learning models like neural machine translation. However, in practice, high-quality training samples are generally difficult to obtain, creating significant obstacles to the practical application of these methods. Therefore, efficiently and accurately identifying noise samples from training samples and obtaining high-quality training samples is crucial for developing effective deep learning models using these training samples.

[0030] The present disclosure proposes a method for efficiently and accurately identifying noise samples. The method first obtains an original sample set, then uses a preset adjustment rule to add a mask to the original training corpus in the original sample set to obtain a target sample set, and then uses a pre-trained language model to perform mask prediction on multiple target training corpora in the target sample set to obtain the first masked predicted character corresponding to each target training corpus, and then matches the first masked predicted character corresponding to each target training corpus with a preset condition. According to the target training corpus whose first masked predicted character in the target sample set does not match the preset condition, the corresponding original training corpus in the original sample set is determined to be a noise sample. In this way, noise samples can be efficiently and accurately identified from the original sample set, and human resources are saved.

[0031] The following describes a noise sample recognition method, a model training method for noise sample recognition, an apparatus, an electronic device, a non-transitory computer-readable storage medium, and a computer program product according to embodiments of the present disclosure with reference to the accompanying drawings.

[0032] First combine Figure 1 , the noise sample recognition method provided by the present disclosure is described in detail.

[0033] Figure 1 This is a flow chart of a noise sample identification method according to the first embodiment of the present disclosure. It should be noted that the noise sample identification method provided in this embodiment of the present disclosure is performed by a noise sample identification device. The noise sample identification device can be an electronic device or software configured in an electronic device, etc., to efficiently and accurately identify noise samples in an original sample set. This embodiment of the present disclosure uses the example of a noise sample identification device configured in an electronic device as an example.

[0034] Among them, the electronic device can be any stationary or mobile computing device capable of data processing, such as mobile computing devices such as laptops, smart phones, wearable devices, or stationary computing devices such as desktop computers, or servers, or other types of computing devices, etc., and this disclosure does not limit this.

[0035] like Figure 1 As shown, the noise sample recognition method may include the following steps:

[0036] Step 101: Obtain an original sample set.

[0037] The original sample set is a training sample set from which noise samples need to be identified, and includes a plurality of original training corpora, each of which has a corresponding annotation.

[0038] Among them, noise samples can be understood as original training corpus with inconsistent annotations, such as the actual category is "biological virus" but is labeled as "terminology", or there is sample confusion, such as the model's training task is a classification task but is labeled as the corresponding translation result, or original training corpus with other abnormal situations, and this disclosure does not limit this.

[0039] The annotations corresponding to each original training corpus are related to the task type of the training task when the model is trained based on the original sample set. For example, if the training task is a noun phrase classification task, the annotations corresponding to the original training corpus can be the category annotations of the noun phrases in the original training corpus; if the training task is a translation task, the annotations corresponding to the original training corpus can be the translation result annotations corresponding to the original training corpus, etc. This disclosure does not impose any restrictions on this.

[0040] Step 102 : Using a preset adjustment rule, a mask is added to the original training corpus in the original sample set to obtain a target sample set.

[0041] Step 103 : Using the pre-trained language model, perform mask prediction on multiple target training corpora in the target sample set to obtain a first mask prediction character corresponding to each target training corpus.

[0042] In an exemplary embodiment, the pre-trained language model can be a neural network model of any type or structure. The pre-trained language model can be trained using multiple target training corpora and corresponding annotations from a target sample set, or can be trained using multiple training corpora obtained by masking training corpora from other sample sets and the corresponding annotations, without limitation in this disclosure.

[0043] The preset adjustment rules refer to the rules used to adjust the original training corpus in the original sample set to obtain the target sample set. The preset adjustment rules are determined according to the task form of the training task of the pre-trained language model. The task form of the training task of the pre-trained language model can be understood as the style of the input corpus of the language model during the process of training the pre-trained language model. The target training corpus in the target sample set obtained by adding a mask to the original training corpus in the original sample set using the preset adjustment rules has the same style as the input corpus of the language model during the process of training the pre-trained language model.

[0044] In an exemplary embodiment, the training task of the pre-trained language model can be a prompt learning task. Accordingly, the task form of the training task can be that the input corpus contains a [mask] mask, and the training goal of the training task is to predict the position corresponding to the [mask] mask. It should be noted that when predicting the position corresponding to the mask, the prediction result is related to the task type of the training task of the pre-trained language model. For example, when the task type of the training task of the pre-trained language model is a text classification task, predicting the position corresponding to the mask is specifically predicting the position corresponding to the mask as the correct category; when the task type of the training task of the pre-trained language model is a translation task, predicting the position corresponding to the mask is specifically predicting the position corresponding to the mask as the correct translation result.

[0045] Accordingly, in embodiments of the present disclosure, a preset adjustment rule can be used to add a mask to the original training corpus in the original sample set to obtain a target sample set, which contains multiple target training corpora. The multiple target training corpora in the target sample set are then fed into a pre-trained language model to perform mask prediction, i.e., predict the positions corresponding to the masks in the multiple target training corpora to obtain the first masked predicted character corresponding to each target training corpus. The first masked predicted character is the prediction result for the position corresponding to the mask.

[0046] For example, assuming the original training corpus is "ABC Science and Technology Park", the training task of the pre-trained language model is a prompt learning task, the task type is a noun phrase classification task, and the adjustment rule is to concatenate the original training corpus and the category label mask through the conjunction "is" to obtain the target training corpus, where the category label mask indicates that the corresponding position is predicted as the category corresponding to the original training corpus. Then, using this adjustment rule, adding the [mask] mask to the original training corpus can obtain the target training corpus "ABC Science and Technology Park is [mask][mask]", and inputting the target training corpus into the pre-trained language model for mask prediction can obtain "ABC Science and Technology Park is a place", where "place" is the first mask prediction character corresponding to the target training corpus "ABC Science and Technology Park is [mask][mask]".

[0047] Step 104 : Match the first mask predicted characters corresponding to each target training corpus with preset conditions.

[0048] Step 105 : According to the target training corpus whose first mask predicted character in the target sample set does not match the preset condition, it is determined that the corresponding original training corpus in the original sample set is a noise sample.

[0049] In an exemplary embodiment, when the original training corpus in the original sample set is a noise sample, the conditions that the first mask prediction character corresponding to the corresponding target training corpus needs to meet can be pre-set. After obtaining the first mask prediction character corresponding to each target training corpus, the first mask prediction character corresponding to the target training corpus can be matched with the preset conditions, and based on the target training corpus whose first mask prediction character in the target sample set does not match the preset conditions, it is determined that the corresponding original training corpus in the original sample set is a noise sample.

[0050] Since the task form of the training task of the pre-trained language model (i.e., prediction by mask prediction) is more sensitive to noise samples, if the input corpus of the pre-trained language model is a noise sample, it will directly and significantly affect the performance of the language model. Therefore, by adopting a preset adjustment rule, a mask is added to the original training corpus in the original sample set to obtain the target training using the pre-trained language model, and the pre-trained language model is used to perform mask prediction on multiple target training corpora in the target sample set. Then, based on the uncontrollable results during mask prediction, the noise samples in the original sample set are identified, which can achieve accurate recognition of noise samples. In addition, since it can replace manual noise sample recognition for a large amount of original training corpus, the recognition efficiency of noise samples is improved, saving human resources.

[0051] In summary, the noise sample identification method provided by the embodiment of the present disclosure obtains an original sample set, adopts a preset adjustment rule, adds a mask to the original training corpus in the original sample set to obtain a target sample set, and uses a pre-trained language model to perform mask prediction on multiple target training corpora in the target sample set to obtain the first masked predicted character corresponding to each target training corpus, matches the first masked predicted character corresponding to each target training corpus with a preset condition, and determines that the corresponding original training corpus in the original sample set is a noise sample based on the target training corpus whose first masked predicted character in the target sample set does not match the preset condition, thereby being able to efficiently and accurately identify noise samples from the sample set and saving human resources.

[0052] Through the above analysis, it can be seen that in the embodiment of the present disclosure, a pre-trained language model can be used to perform mask prediction on multiple target training corpora in the target sample set, and then the noise samples in the original sample set can be identified based on the first mask prediction characters corresponding to each target training corpus. In one possible implementation form, the pre-trained language model can be obtained by training a masked pre-trained language model using multiple target training corpora in the target sample set. Figure 2 , further illustrating the noise sample recognition method provided by the present disclosure.

[0053] Figure 2 FIG. 1 is a flow chart of a noise sample recognition method according to the second embodiment of the present disclosure. Figure 2 As shown, the noise sample recognition method may include the following steps:

[0054] Step 201: Obtain an original sample set.

[0055] The original sample set is a training sample set from which noise samples need to be identified, and includes a plurality of original training corpora, each of which has a corresponding annotation.

[0056] The annotations corresponding to each original training corpus are related to the task type of the training task when the model is trained based on the original sample set. For example, if the training task is a noun phrase classification task, the annotations corresponding to the original training corpus can be the category annotations of the noun phrases in the original training corpus; if the training task is a translation task, the annotations corresponding to the original training corpus can be the translation result annotations corresponding to the original training corpus, etc. This disclosure does not impose any restrictions on this.

[0057] Step 202 : determining an adjustment rule based on the training tasks performed by the masked pre-trained language model during the pre-training process.

[0058] Among them, the masked pre-trained language model refers to the training task (i.e., pre-training task) in the pre-training process, which is a pre-trained language model that uses masked prediction to perform predictions. Among them, the pre-trained language model here refers to a language model obtained by training with large-scale unsupervised corpus. Among them, the masked pre-trained language model can be a pre-trained language model that uses masked prediction to perform predictions in a pre-training process of any structure or type. For example, the masked pre-trained language model can be a BERT (Bidirectional Encoder Representations from Transformers, bidirectional encoding representations based on transformers), RoBERTa (A Robustly Optimized BERT, strongly optimized BERT), ERNIE (BERT-CHINESE, WWM-BERT-CHINESE, Chinese pre-training model) and other models, and this disclosure does not limit this.

[0059] It is understood that when a masked pre-trained language model is trained using multiple target training corpora from a target sample set to obtain a pre-trained language model, the training tasks performed by the masked pre-trained language model during the pre-training process are the training tasks performed during the process of training the masked pre-trained language model to obtain the pre-trained language model. Accordingly, adjustment rules can be determined based on the training tasks performed by the masked pre-trained language model during the pre-training process.

[0060] Step 203 : adding masks to the multiple original training corpora in the original sample set according to the adjustment rule to obtain target training corpora corresponding to the training task.

[0061] Step 204: Generate a target sample set corresponding to the training task based on each target training corpus corresponding to the training task.

[0062] In an exemplary embodiment, after determining an adjustment rule based on the training task performed by the masked pretrained language model during the pretraining process, the adjustment rule can be used to add masks to multiple original training corpora in the original sample set to obtain target training corpora corresponding to the training task. Furthermore, based on the target training corpora corresponding to the training task, a target sample set corresponding to the training task is generated, wherein the target sample set includes each target training corpus. This achieves the addition of masks to the original training corpora in the original sample set to obtain a target sample set corresponding to the training task performed by the masked pretrained language model during the pretraining process.

[0063] In an exemplary embodiment, when the training task performed by the masked pre-trained language model during the pre-training process is a text classification task, the adjustment rules may include: for each original training corpus, splicing the original training corpus and the first category label mask through the first associated word to obtain the target training corpus corresponding to the training task; wherein, the first category label mask indicates that the corresponding position is predicted as the category corresponding to the original training corpus; or, for each original training corpus, splicing the original training corpus with the spliced ​​text to obtain the target training corpus corresponding to the training task; wherein, the spliced ​​text is obtained by splicing the text segment in the original training corpus with the second category label mask through the second associated word; wherein, the second category label mask indicates that the corresponding position is predicted as the category corresponding to the text segment.

[0064] In an exemplary embodiment, when the text classification task is specifically a noun phrase classification task, the adjustment rule may be: for each original training corpus, the original training corpus and the first category label mask are concatenated using a first associated word to obtain a target training corpus corresponding to the training task. The first associated word can be set as needed.

[0065] The following describes the adjustment rule and the process of adding masks to the original training corpus according to the adjustment rule with reference to a specific example.

[0066] For example, let's assume the original training corpus is "ABC Science Park," where "ABC" represents the name of a science park. The masked pre-trained language model performs the noun phrase classification task during pre-training. The adjustment rule is to concatenate the original training corpus and the first category label mask using the first associated word. Since "is" is a key associated factor in sentences linking noun phrases to categories, according to the natural distribution of text, the first associated word can be set to "is."

[0067] For the original training corpus "ABC Science and Technology Park", using this adjustment rule, "ABC Science and Technology Park" and the first category label mask "[mask][mask]" can be spliced ​​through the first associated word "is", thereby obtaining the target training corpus "ABC Science and Technology Park is [mask][mask]" corresponding to the original training corpus "ABC Science and Technology Park". The style of this target training corpus is the same as the style of the input corpus of the masked pre-training language model during the pre-training process.

[0068] In an exemplary embodiment, when the text classification task is specifically a named entity recognition task, the adjustment rule may be: for each original training corpus, the original training corpus is concatenated with the concatenated text to obtain the target training corpus corresponding to the training task; wherein the concatenated text is obtained by concatenating a text segment in the original training corpus with a second category label mask using a second associated word. The text segment is the segment in the original training corpus for which named entity recognition is to be performed, and the second associated word can be set as needed.

[0069] The following describes the adjustment rule and the process of adding masks to the original training corpus according to the adjustment rule with reference to a specific example.

[0070] For example, suppose the original training corpus is "Spokesperson D confirmed the news on the same day," where the text segment that needs to be recognized by named entity is "D," and "D" represents the name of an organization. The training task performed by the masked pre-trained language model during the pre-training process is the named entity recognition task. The adjustment rule is to concatenate the original training corpus with the concatenated text to obtain the target training corpus corresponding to the training task. The concatenated text is obtained by concatenating the text segment in the original training corpus with the second category label mask through the second associated word. Since "is" is an important associated factor in the sentence pattern that associates entities with categories according to the natural distribution of the text, the second associated word can be set to "is."

[0071] For the original training corpus "D spokesperson confirmed the news on the same day", using this adjustment rule, the text segment "D" and the second category label mask "[mask][mask][mask][mask]" can be spliced ​​with the second associated word "is" to obtain a spliced ​​text, and then the original training corpus and the spliced ​​text are spliced ​​to obtain the target training corpus "D spokesperson confirmed the news on the same day" corresponding to the original training corpus "D spokesperson confirmed the news on the same day, D is [mask][mask][mask][mask]". The style of this target training corpus is the same as the style of the input corpus of the masked pre-training language model during the pre-training process. It should be noted that punctuation marks can be added to the target training corpus as needed, such as "," in the target training corpus of this example.

[0072] When the training task performed by the masked pre-trained language model during the pre-training process is a text classification task, the above-mentioned adjustment rules are determined so that a mask can be added to the original training corpus in the original sample set according to the above-mentioned adjustment rules to obtain a target sample set corresponding to the text classification task.

[0073] Step 205 : Using the pre-trained language model, perform mask prediction on multiple target training corpora in the target sample set to obtain a first mask prediction character corresponding to each target training corpus.

[0074] The pre-trained language model is obtained by training a masked pre-trained language model using multiple target training corpora in the target sample set. During the training process, the target training corpora in the target sample set are used as the input of the masked pre-trained language model, and the annotations of the original training corpora corresponding to the target training corpora are used as the output of the masked pre-trained language model. The masked pre-trained language model is then subjected to iterative supervised training to obtain the pre-trained language model. The specific process of obtaining the pre-trained language model will be described in the following embodiments and will not be elaborated on here.

[0075] In an exemplary embodiment, multiple target training corpora in the target sample set can be respectively input into the pre-trained language model to perform mask prediction, that is, the positions corresponding to the masks in the multiple target training corpora are predicted to obtain the first mask prediction characters corresponding to each target training corpus. Among them, the first mask prediction character is the prediction result of the position corresponding to the mask. It should be noted that when predicting the position corresponding to the mask, the prediction result is related to the task type of the training task of the pre-trained language model. For example, when the task type of the training task of the pre-trained language model is a text classification task, predicting the position corresponding to the mask is specifically predicting the position corresponding to the mask as the correct category; when the task type of the training task of the pre-trained language model is a translation task, predicting the position corresponding to the mask is specifically predicting the position corresponding to the mask as the correct translation result.

[0076] For example, taking the text recognition task as an example, continuing with the above example, when the training task performed by the masked pre-trained language model during the pre-training process and the training task of the pre-trained language model during the training process is specifically a noun phrase classification task, for the original training corpus is "ABC Science and Technology Park", the target training corpus obtained is "ABC Science and Technology Park is [mask][mask]", and the target training corpus is input into the pre-trained language model for mask prediction, and "ABC Science and Technology Park is a place" can be obtained, where "place" is the first mask prediction character corresponding to the target training corpus "ABC Science and Technology Park is [mask][mask]".

[0077] When the training tasks performed by the masked pre-trained language model during the pre-training process and the training tasks of the pre-trained language model during the training process are specifically named entity recognition tasks, for the original training corpus "D spokesperson confirmed the news on the same day", the target training corpus obtained is "D spokesperson confirmed the news on the same day, D is [mask][mask][mask][mask]", and the target training corpus is input into the pre-trained language model for mask prediction, and the result is "D spokesperson confirmed the news on the same day, D is an organization", where "organization" is the first mask prediction character corresponding to the target training corpus "D spokesperson confirmed the news on the same day, D is [mask][mask][mask][mask]".

[0078] By determining an adjustment rule based on the training tasks performed by the masked pre-trained language model during the pre-training process, and using the adjustment rule to add a mask to the original training corpus in the original sample set to obtain a target sample set, the task form of converting the original training corpus in the original sample set into the training task performed by the masked pre-trained language model during the pre-training process is achieved. Furthermore, because the pre-trained language model is obtained by training the masked pre-trained language model using multiple target training corpora in the target sample set, the pre-trained language model can fully utilize the statistical co-occurrence knowledge learned by the masked pre-trained language model during the pre-training process to perform masked prediction. Since the pre-training corpus of the masked pre-trained language model during the pre-training process directly covers most of the task data, the pre-trained language model is relatively sensitive to data distribution. If the original sample set contains inconsistent annotations or noise samples such as sample confusion, it will directly and significantly affect the performance of the language model, making it easier to obtain the distribution of noise samples in the original sample set and more convenient to identify noise samples in the original sample set.

[0079] Step 206: Match the first mask predicted characters corresponding to each target training corpus with preset conditions.

[0080] Step 207 : According to the target training corpus whose first mask predicted character in the target sample set does not match the preset condition, it is determined that the corresponding original training corpus in the original sample set is a noise sample.

[0081] The specific implementation process and principle of the above steps 206-207 can be referred to the description in the above embodiment and will not be repeated here.

[0082] In summary, the noise sample identification method of the embodiment of the present invention obtains the original sample set, determines the adjustment rule according to the training task performed by the mask pre-trained language model in the pre-training process, adds a mask to the multiple original training corpora in the original sample set according to the adjustment rule to obtain the target training corpora corresponding to the training task, generates the target sample set corresponding to the training task according to the target training corpora corresponding to the training task, uses the pre-trained language model to perform mask prediction on the multiple target training corpora in the target sample set to obtain the first mask prediction character corresponding to each target training corpus, matches the first mask prediction character corresponding to each target training corpus with the preset condition, and determines that the corresponding original training corpus in the original sample set is a noise sample according to the target training corpus whose first mask prediction character in the target sample set does not match the preset condition. Noise samples can be identified from the original sample set efficiently and accurately, and human resources are saved.

[0083] Through the above analysis, it can be seen that in the embodiment of the present disclosure, after obtaining the first mask prediction character corresponding to each target training corpus, the first mask prediction character can be matched with the preset condition, and according to the target training corpus whose first mask prediction character in the target sample set does not match the preset condition, it is determined that the corresponding original training corpus in the original sample set is a noise sample. Figure 3 Taking the first mask prediction character indicating the prediction category corresponding to the target training corpus as an example, the process of determining the noise samples in the original sample set according to the first mask prediction character corresponding to each target training corpus in the noise sample recognition method provided by the present invention is further explained.

[0084] Figure 3 FIG. 1 is a flow chart of a noise sample recognition method according to the third embodiment of the present disclosure. Figure 3 As shown, the noise sample recognition method may include the following steps:

[0085] Step 301: Obtain an original sample set; wherein each original training corpus in the original sample set has a corresponding annotation.

[0086] Step 302: Using a preset adjustment rule, a mask is added to the original training corpus in the original sample set to obtain a target sample set.

[0087] Step 303: Use the pre-trained language model to perform mask prediction on multiple target training corpora in the target sample set to obtain the first mask prediction character corresponding to each target training corpus, wherein the pre-trained language model has learned the mapping relationship between the input target training corpus and the annotation of the corresponding original training corpus, and the first mask prediction character indicates the prediction category corresponding to the target training corpus.

[0088] In an exemplary embodiment, a pretrained language model is obtained by iteratively supervising the initial language model using the target training corpus in the target sample set as the input of the initial language model during training, and using the annotations of the original training corpus corresponding to the target training corpus as the output of the initial language model. The pretrained language model has learned the mapping relationship between the input target training corpus and the annotations of the corresponding original training corpus. The specific process of obtaining the pretrained language model will be described in the following embodiments and will not be repeated here.

[0089] In an exemplary embodiment, the pre-trained language model may be obtained by training a masked pre-trained language model using multiple target training corpora in the target sample set. Accordingly, the preset adjustment rules may be determined based on the training tasks performed by the masked pre-trained language model during the pre-training process. That is, before step 302, the process may further include determining the adjustment rules based on the training tasks performed by the masked pre-trained language model during the pre-training process.

[0090] It should be noted that the masked pre-trained language model in the embodiments of the present disclosure can be an autoencoder pre-trained language model, such as BERT, RoBERTa, ERNIE, etc., or a non-autoencoder pre-trained language model, such as an autoregressive pre-trained language model, and the present disclosure does not impose any restrictions on this. Specifically, when the masked pre-trained language model is an autoencoder pre-trained language model, since the autoencoder pre-trained language model calculates the association between the position corresponding to the [mask] mask and all other positions, the rationality of the sentence of the prompt learning task can be ignored.

[0091] The specific implementation process and principles of steps 301-303 can be referred to the description of the above embodiment and will not be repeated here.

[0092] Step 304 : Match the first mask predicted characters corresponding to each target training corpus with a preset condition, wherein the preset condition includes that the predicted category belongs to a set of predicted categories.

[0093] Step 305 : When the second prediction category corresponding to the second target training corpus does not belong to the set of prediction categories, it is determined that the first masked predicted character corresponding to the second target training corpus does not match a preset condition.

[0094] The set of prediction categories can be pre-determined based on the prediction categories that the target training corpus may correspond to when the target training corpus is a non-noise sample. The second target training corpus is the target training corpus in the target sample set, and the second prediction category is the prediction category corresponding to the target training corpus indicated by the first mask prediction character corresponding to the second target training corpus.

[0095] In an exemplary embodiment, when the first masked predicted character indicates the predicted category corresponding to the target training corpus and the preset condition includes that the predicted category belongs to a set of predicted categories, matching the first masked predicted character corresponding to each target training corpus with the preset condition can be understood as determining whether the predicted category corresponding to each target training corpus belongs to the set of predicted categories. In an exemplary embodiment, if the second predicted category corresponding to the second target training corpus in the target sample set does not belong to the set of predicted categories, it can be determined that the first masked predicted character corresponding to the second target training corpus does not match the preset condition; if the first predicted category corresponding to the first target training corpus belongs to the set of predicted categories, it can be determined that the first masked predicted character corresponding to the first target training corpus matches the preset condition. Wherein, the first target training corpus is the target training corpus in the target sample set, and the first predicted category is the predicted category corresponding to the target training corpus indicated by the first masked predicted character corresponding to the first target training corpus.

[0096] Step 306 : According to the second target training corpus in which the first mask predicted character in the target sample set does not match the preset condition, it is determined that the corresponding original training corpus in the original sample set is a noise sample.

[0097] In an exemplary embodiment, when the first mask predicted character corresponding to the second target training corpus in the target sample set does not match the preset condition, that is, when the predicted category corresponding to the second target training corpus does not belong to the set of predicted categories, it means that the pre-trained language model is confused on the predicted categories in the set of predicted categories. The specific manifestation may be that the original training corpus corresponding to the second target training corpus is not labeled as the category to which it belongs but is labeled as other categories, resulting in inconsistency in the labeling of the original training corpora, so that it can be determined that the original training corpus corresponding to the second target training corpus in the original sample set is a noise sample.

[0098] For example, taking the case where the pre-trained language model is trained by using multiple target training corpora in the target sample set to train the masked pre-trained language model, and the training tasks performed by the masked pre-trained language model during the pre-training process and the training tasks of the pre-trained language model during the training process are noun phrase classification tasks. Suppose the set of predicted categories includes "biological virus" and "term" categories, and when using the pre-trained language model to perform masked prediction on a certain target training corpus in the target sample set, the first masked prediction character corresponding to the target training corpus is "shu wu bing", that is, the target training corpus belongs to the "shu wu bing" category. Since the "shu wu bing" category does not belong to the set of predicted categories, it indicates that the pre-trained language model is confused in the prediction of the two categories of "biological virus" and "term". The specific manifestation may be that the original training corpus belonging to the "biological virus" category is labeled as the "term" category, resulting in inconsistent labeling of each original training corpus. Thus, the original training corpus corresponding to the target training corpus with the first masked prediction character of "shu wu bing" can be determined as a noise sample.

[0099] Since the pre-trained language model has learned the mapping relationship between the input target training corpus and the annotation of the corresponding original training corpus, fully utilized the text information in the target training corpus during the learning process, and the task form of the training task of the pre-trained language model (i.e., prediction by means of masked prediction) is sensitive to noise samples. If the input corpus of the pre-trained language model is a noise sample, it will directly and significantly affect the performance of the language model. Therefore, by using the pre-trained language model to perform masked prediction on multiple target training corpora in the target sample set, it is possible to accurately identify noise samples according to the prediction results. And by using the first masked prediction character to indicate the predicted category corresponding to the target training corpus, and determining whether the corresponding predicted category belongs to the set of predicted categories, it is further determined whether the first masked prediction character corresponding to the target training corpus matches the preset condition. Then, the original training corpus corresponding to the target training corpus whose first masked prediction character does not match the preset condition is determined as a noise sample, realizing the use of the pre-trained language model to predict the predicted category corresponding to the target training corpus, and accurately identifying the noise samples in the original sample set according to the uncontrollable results during the masked prediction of the pre-trained language model.

[0100] Step 307, correct the noise samples.

[0101] Step 308, update the original sample set with the corrected noise samples.

[0102] In an exemplary embodiment, after determining the noise samples in the original sample set, the noise samples can be corrected, and the original sample set can be updated with the corrected noise samples. For example Figure 3As shown, after the original sample set is updated, the preset adjustment rules can be used to add a mask to the original training corpus in the updated original sample set to obtain an updated target sample set. The updated target sample set can continue to be used for learning the language model and identifying noise samples in the original sample set. After identifying the noise samples, the original sample set can be further corrected and updated.

[0103] Taking the example of a pre-trained language model obtained by training a masked pre-trained language model using multiple target training corpora in a target sample set, after obtaining the original sample set, an adjustment rule can be determined based on the training tasks performed by the masked pre-trained language model during the pre-training process. Based on the adjustment rule, a mask is added to the original training corpora in the original sample set to obtain a target sample set. The masked pre-trained language model is then trained using multiple target training corpora in the target sample set to obtain a pre-trained language model. The pre-trained language model is then used to perform masked prediction on the multiple target training corpora in the target sample set to obtain a first masked predicted character corresponding to each target training corpus, where the first masked predicted character indicates the predicted category corresponding to the target training corpus. The first masked predicted character corresponding to each target training corpus is matched with a preset condition, and based on a second target training corpus in the target sample set whose first masked predicted character does not match the preset condition, it is determined that the original training corpus corresponding to the original sample set is a noise sample. The noise sample can then be corrected, and the original sample set can be updated using the corrected noise sample. The updated original sample set can continue to be used to train the pre-trained language model, and then the further trained language model can be used to further identify noise samples in the original sample set. After identifying the noise samples, the original sample set can be further corrected and updated.

[0104] In this way, the language model can be iteratively trained and continuously used to identify noise samples until a language model that meets the requirements is obtained. This language model can then be used in downstream applications, and the original sample set now contains very few noise samples, thereby improving the quality of the original sample set. The requirements that the language model meets can be determined based on the needs of the downstream application, and this disclosure does not impose any restrictions on this.

[0105] In summary, the noise sample recognition method of the embodiment of the present disclosure obtains an original sample set, wherein each original training corpus in the original sample set has a corresponding annotation, adopts a preset adjustment rule to add a mask to the original training corpus in the original sample set to obtain a target sample set, and uses a pre-trained language model to perform mask prediction on multiple target training corpora in the target sample set to obtain a first mask prediction character corresponding to each target training corpus, wherein the pre-trained language model has learned the mapping relationship between the input target training corpus and the annotation of the corresponding original training corpus, the first mask prediction character indicates the prediction category corresponding to the target training corpus, and each target training corpus is masked. The first mask prediction character corresponding to the target training corpus is matched with a preset condition, wherein the preset condition includes that the prediction category belongs to a set of prediction categories. When the second prediction category corresponding to the second target training corpus does not belong to the set of prediction categories, it is determined that the first mask prediction character corresponding to the second target training corpus does not match the preset condition. According to the target training corpus whose first mask prediction character in the target sample set does not match the preset condition, it is determined that the corresponding original training corpus in the original sample set is a noise sample, the noise sample is corrected, and the original sample set is updated with the corrected noise sample. The noise sample can be efficiently and accurately identified from the original sample set, and human resources are saved.

[0106] According to an embodiment of the present disclosure, the present disclosure also provides a model training method for noise sample recognition. Figure 4 , a detailed description is given of the model training method for noise sample recognition provided by the present disclosure.

[0107] Figure 4 This is a flow chart of a model training method for noise sample recognition according to the fourth embodiment of the present disclosure. It should be noted that the model training method for noise sample recognition provided in this embodiment of the present disclosure is executed by a model training device for noise sample recognition. The model training device for noise sample recognition can specifically be an electronic device, or software configured in an electronic device, to train a language model for noise sample recognition. This embodiment of the present disclosure uses the example of a model training device for noise sample recognition configured in an electronic device as an example.

[0108] like Figure 4 As shown, the model training method for noise sample recognition may include the following steps:

[0109] Step 401: Acquire an original sample set, wherein each original training corpus in the original sample set has a corresponding annotation.

[0110] Step 402 : Using a preset adjustment rule, a mask is added to the original training corpus in the original sample set to obtain a target sample set.

[0111] The specific implementation process and principle of steps 401 - 402 may refer to the description in the embodiment of the noise sample recognition method, which will not be repeated here.

[0112] Step 403 : Using the initial language model, perform mask prediction on multiple target training corpora in the target sample set to obtain second mask prediction characters corresponding to each target training corpus.

[0113] The preset adjustment rules refer to the rules used to adjust the original training corpus in the original sample set to obtain the target sample set. The preset adjustment rules are determined based on the task form of the initial language model training task. The task form of the initial language model training task can be understood as the style of the input corpus of the language model during the process of training the initial language model. The target training corpus in the target sample set obtained by masking the original training corpus in the original sample set using the preset adjustment rules has the same style as the input corpus of the language model during the process of training the initial language model.

[0114] The initial language model may be a neural network model of any type or structure, and this disclosure does not impose any restrictions on this.

[0115] In an exemplary embodiment, the training task of the initial language model can be a prompt learning task. Accordingly, the task form of the training task can be that the input corpus contains a [mask] mask, and the training goal of the training task is to predict the position corresponding to the [mask] mask. It should be noted that when predicting the position corresponding to the mask, the prediction result is related to the task type of the training task of the initial language model. For example, when the task type of the training task of the initial language model is a text classification task, predicting the position corresponding to the mask is specifically predicting the position corresponding to the mask as the correct category; when the task type of the training task of the initial language model is a translation task, predicting the position corresponding to the mask is specifically predicting the position corresponding to the mask as the correct translation result.

[0116] Accordingly, in the disclosed embodiments, a preset adjustment rule can be used to mask the original training corpus in the original sample set to obtain a target sample set, which includes multiple target training corpora. The multiple target training corpora in the target sample set are then fed into the initial language model for mask prediction, i.e., prediction is performed at the positions corresponding to the masks in the multiple target training corpora to obtain the second masked predicted characters corresponding to each target training corpus. The second masked predicted characters are the prediction results for the positions corresponding to the masks.

[0117] For example, assuming the original training corpus is "ABC Science and Technology Park", the initial language model training task is a prompt learning task, the task type is a noun phrase classification task, and the adjustment rule is to concatenate the original training corpus and the category label mask through the conjunction "is" to obtain the target training corpus, where the category label mask indicates that the corresponding position is predicted as the category corresponding to the original training corpus. Then, using this adjustment rule, adding the [mask] mask to the original training corpus can obtain the target training corpus "ABC Science and Technology Park is [mask][mask]", and inputting this target training corpus into the initial language model for mask prediction can obtain "ABC Science and Technology Park is a place", where "place" is the second mask prediction character corresponding to the target training corpus "ABC Science and Technology Park is [mask][mask]".

[0118] Step 404 : adjusting the model parameters of the initial language model according to the differences between the second mask predicted characters corresponding to each target training corpus and the annotations corresponding to each original training corpus.

[0119] Step 405: determine the language model obtained after adjustment as the language model for noise sample recognition.

[0120] In an exemplary embodiment, the initial language model can be trained by deep learning to obtain a language model for noise sample recognition. Compared with other machine learning methods, deep learning performs better on large data sets.

[0121] When training an initial language model through deep learning, one or more target training corpora in the target sample set can be used as input and fed into the initial language model to perform masked prediction on the target training corpora to obtain a second masked predicted character corresponding to the target training corpora. Furthermore, the difference between the second masked predicted character corresponding to the target training corpora and the annotation of the corresponding original training corpora can be obtained by combining the annotation of the target training corpora with the annotation of the corresponding original training corpora. The model parameters of the initial language model can be adjusted based on the difference to obtain an adjusted language model. Then, one or more target training corpora in the target sample set can be used as input and fed into the initial language model to perform masked prediction on the target training corpora to obtain a second masked predicted character corresponding to the target training corpora. Furthermore, the difference between the second masked predicted character corresponding to the target training corpora and the annotation of the corresponding original training corpora can be obtained by combining the annotation of the target training corpora with the annotation of the corresponding original training corpora. The model parameters of the adjusted language model can be adjusted based on the difference to obtain a further adjusted language model. Therefore, the initial language model is iteratively trained by continuously adjusting the model parameters of the initial language model until the accuracy of the second mask predicted characters corresponding to the target training corpus meets the preset threshold. The training is completed and the final adjusted language model is obtained. The adjusted language model has learned the mapping relationship between the target training corpus and the annotations of the corresponding original training corpus.

[0122] The language model obtained after adjustment can be used as a language model for noise sample recognition, and mask prediction can be performed on multiple target training corpora in the target sample set to obtain the first mask prediction character corresponding to each target training corpus. The first mask prediction character corresponding to each target training corpus can then be matched with the preset conditions. According to the target training corpus whose first mask prediction character in the target sample set does not match the preset conditions, the corresponding original training corpus in the original sample set is determined to be a noise sample.

[0123] Thus, a language model for noise sample recognition is trained based on an initial language model and a target sample set having the same task form as the training task of the initial language model. Since the language model learns the mapping relationship between the target training corpus and the annotations of the corresponding original training corpus during the training process, the text information in the target training corpus is fully utilized during the learning process, and the task form of the training task of the language model (i.e., prediction by mask prediction) is relatively sensitive to noise samples. If the input corpus of the language model for noise sample recognition is a noise sample, it will directly and significantly affect the performance of the language model, so that the trained language model can be used to accurately recognize noise samples in the original sample set. Moreover, since the language model for noise sample recognition can replace manual noise sample recognition for a large amount of original training corpus, it can improve the recognition efficiency of noise samples and save human resources.

[0124] In summary, the model training method for noise sample recognition provided by the embodiments of the present disclosure obtains an original sample set, wherein each original training corpus in the original sample set has a corresponding annotation, and uses a preset adjustment rule to add a mask to the original training corpus in the original sample set to obtain a target sample set. The initial language model is used to perform mask prediction on multiple target training corpora in the target sample set to obtain a second masked predicted character corresponding to each target training corpus. According to the difference between the second masked predicted character corresponding to each target training corpus and the annotation corresponding to each original training corpus, the model parameters of the initial language model are adjusted, and the language model obtained after adjustment is determined as the language model for noise sample recognition, thereby realizing the training of the initial language model and obtaining a language model that can efficiently and accurately recognize noise samples and save human resources.

[0125] Through the above analysis, it can be seen that in the embodiment of the present disclosure, the initial language model can be trained to obtain a language model for noise sample recognition. In one possible implementation form, the initial language model can be a mask pre-trained language model. Figure 5 , further illustrating the model training method for noise sample recognition provided by the present disclosure.

[0126] Figure 5 FIG. 5 is a flow chart of a model training method for noise sample recognition according to the fifth embodiment of the present disclosure. Figure 5 As shown, the model training method for noise sample recognition may include the following steps:

[0127] Step 501: Acquire an original sample set, wherein each original training corpus in the original sample set has a corresponding annotation.

[0128] Step 502 : determining an adjustment rule according to the training task performed by the masked pre-trained language model during the pre-training process.

[0129] The masked pre-trained language model refers to a training task (i.e., a pre-training task) in the pre-training process, which is a pre-trained language model that uses masked prediction to perform predictions. The pre-trained language model here refers to a language model trained using large-scale unsupervised corpus. The masked pre-trained language model can be a pre-trained language model that uses masked prediction to perform predictions during the pre-training process of any structure or type. For example, the masked pre-trained language model can be a model such as BERT, RoBERTa, or ERNIE, and this disclosure does not impose any restrictions on this.

[0130] It is understood that when the initial language model is a masked pre-trained language model, the training tasks performed by the masked pre-trained language model during pre-training are the training tasks performed during the process of training the masked pre-trained language model to obtain a language model for noise sample recognition. Accordingly, the adjustment rules can be determined based on the training tasks performed by the masked pre-trained language model during pre-training.

[0131] Step 503: Adopting the adjustment rule, masking is added to the original training corpus in the original sample set to obtain the target sample set.

[0132] In an exemplary embodiment, after determining the adjustment rules based on the training tasks performed by the masked pre-trained language model during the pre-training process, masks can be added to multiple original training corpora in the original sample set according to the adjustment rules to obtain the target training corpora corresponding to the training task, and then based on the target training corpora corresponding to the training task, a target sample set corresponding to the training task is generated, wherein the target sample set includes the target training corpora.

[0133] By determining an adjustment rule based on the training tasks performed by the masked pre-trained language model during the pre-training process, and using the adjustment rule to add a mask to the original training corpus in the original sample set to obtain a target sample set, the task form of converting the original training corpus in the original sample set into the training task performed by the masked pre-trained language model during the pre-training process is achieved.

[0134] In an exemplary embodiment, when the training task performed by the masked pre-trained language model during the pre-training process is a text classification task, the adjustment rules may include: for each original training corpus, the original training corpus is spliced ​​with the first category label mask through the first associated word to obtain the target training corpus corresponding to the training task; wherein the first category label mask indicates that the corresponding position is predicted as the category corresponding to the original training corpus; or, for each original training corpus, the original training corpus is spliced ​​with the spliced ​​text to obtain the target training corpus corresponding to the training task; wherein the spliced ​​text is obtained by splicing the text segment in the original training corpus with the second category label mask through the second associated word; wherein the second category label mask indicates that the corresponding position is predicted as the category corresponding to the text segment. For a specific description of the adjustment rule, please refer to the description in the embodiment of the above-mentioned noise sample recognition method, which will not be repeated here.

[0135] Step 504 : Using the masked pre-trained language model, perform masked prediction on multiple target training corpora in the target sample set to obtain second masked predicted characters corresponding to the target training corpora.

[0136] Step 505 : Adjust the model parameters of the masked pre-trained language model according to the difference between the second masked predicted characters corresponding to the target training corpus and the annotations corresponding to the original training corpus.

[0137] Step 506: Determine the language model obtained after adjustment as the language model for noise sample recognition.

[0138] The specific implementation process and principles of steps 504-506 can be referred to the description of the above embodiment and will not be repeated here.

[0139] In an exemplary embodiment, after the adjusted language model is determined as the language model for noise sample identification, the language model can be used to perform mask prediction on multiple target training corpora in the target sample set to obtain the first mask prediction character corresponding to each target training corpus, and then the first mask prediction character corresponding to each target training corpus is matched with a preset condition. According to the target training corpus whose first mask prediction character in the target sample set does not match the preset condition, the corresponding original training corpus in the original sample set is determined to be a noise sample. In addition, after identifying the noise sample in the original sample set, the noise sample can also be corrected, and the corrected noise sample can be used to update the original sample set, and then the updated original sample set can be used to continue iteratively train the language model, and the iteratively trained language model can be continuously used to identify noise samples until a language model that meets the requirements is obtained.

[0140] In summary, the model training method for noise sample recognition provided by the embodiments of the present disclosure obtains an original sample set, wherein each original training corpus in the original sample set has a corresponding annotation, determines an adjustment rule according to the training task performed by the masked pre-trained language model in the pre-training process, and uses the adjustment rule to add a mask to the original training corpus in the original sample set to obtain a target sample set. The masked pre-trained language model is used to perform mask prediction on multiple target training corpora in the target sample set to obtain a second masked predicted character corresponding to each target training corpus. According to the difference between the second masked predicted character corresponding to each target training corpus and the annotation corresponding to each original training corpus, the model parameters of the masked pre-trained language model are adjusted, and the language model obtained after the adjustment is determined as the language model for noise sample recognition, thereby realizing the training of the masked pre-trained language model and obtaining a language model that can efficiently and accurately recognize noise samples and save human resources.

[0141] The following combination Figure 6 , the noise sample recognition device provided by the present disclosure is described.

[0142] Figure 6 4 is a schematic structural diagram of a noise sample recognition device according to a sixth embodiment of the present disclosure.

[0143] like Figure 6 As shown, the noise sample recognition device 600 provided by the present disclosure includes: a first acquisition module 601 , a first processing module 602 , a first prediction module 603 , a matching module 604 and a first determination module 605 .

[0144] The first acquisition module 601 is used to acquire an original sample set;

[0145] A first processing module 602 is configured to add a mask to the original training corpus in the original sample set using a preset adjustment rule to obtain a target sample set;

[0146] The first prediction module 603 is configured to perform mask prediction on a plurality of target training corpora in the target sample set using a pre-trained language model to obtain a first mask prediction character corresponding to each target training corpus;

[0147] A matching module 604 is configured to match the first mask predicted characters corresponding to each target training corpus with a preset condition;

[0148] The first determining module 605 is configured to determine that the corresponding original training corpus in the original sample set is a noise sample based on the target training corpus whose characters predicted by the first mask in the target sample set do not match the preset condition.

[0149] It should be noted that the noise sample identification device provided in this embodiment can perform the noise sample identification method of the aforementioned embodiment. The noise sample identification device can be an electronic device or software configured on the electronic device to efficiently and accurately identify noise samples in the original sample set.

[0150] Among them, the electronic device can be any stationary or mobile computing device capable of data processing, such as mobile computing devices such as laptops, smart phones, wearable devices, or stationary computing devices such as desktop computers, or servers, or other types of computing devices, etc., and this disclosure does not limit this.

[0151] It should be noted that the description of the embodiment of the noise sample recognition method mentioned above is also applicable to the noise sample recognition device provided in the present disclosure, and will not be repeated here.

[0152] The noise sample recognition device provided by the embodiment of the present disclosure obtains an original sample set, adopts a preset adjustment rule, adds a mask to the original training corpus in the original sample set to obtain a target sample set, uses a pre-trained language model to perform mask prediction on multiple target training corpora in the target sample set to obtain a first masked predicted character corresponding to each target training corpus, matches the first masked predicted character corresponding to each target training corpus with a preset condition, and determines that the corresponding original training corpus in the original sample set is a noise sample based on the target training corpus whose first masked predicted character in the target sample set does not match the preset condition. This makes it possible to efficiently and accurately identify noise samples from the sample set and save human resources.

[0153] The following combination Figure 7 , further illustrating the noise sample recognition device provided by the present disclosure.

[0154] Figure 7 4 is a schematic structural diagram of a noise sample recognition device according to the seventh embodiment of the present disclosure.

[0155] like Figure 7 As shown, the noise sample recognition device 700 may specifically include: a first acquisition module 701, a first processing module 702, a first prediction module 703, a matching module 704 and a first determination module 705. Figure 7 The first acquisition module 701, the first processing module 702, the first prediction module 703, the matching module 704 and the first determination module 705 are Figure 6 The first acquisition module 601 , the first processing module 602 , the first prediction module 603 , the matching module 604 and the first determination module 605 have the same function and structure.

[0156] In an exemplary embodiment, the pre-trained language model is obtained by training a masked pre-trained language model using a plurality of target training corpora in a target sample set.

[0157] In an exemplary embodiment, the noise sample identification apparatus 700 further includes:

[0158] The second determining module 706 is configured to determine an adjustment rule according to the training tasks performed by the masked pre-trained language model during the pre-training process.

[0159] In an exemplary embodiment, the training task is a text classification task; accordingly, the adjustment rules include:

[0160] For each original training corpus, the original training corpus and the first category label mask are concatenated through the first associated word to obtain the target training corpus corresponding to the training task; wherein the first category label mask indicates that the corresponding position is predicted as the category corresponding to the original training corpus;

[0161] Alternatively, for each original training corpus, the original training corpus is spliced ​​with the spliced ​​text to obtain the target training corpus corresponding to the training task; wherein the spliced ​​text is obtained by splicing the text segment in the original training corpus and the second category label mask through the second associated word; wherein the second category label mask indicates that the corresponding position is predicted as the category corresponding to the text segment.

[0162] In an exemplary embodiment, the first processing module 702 includes:

[0163] A processing unit, configured to add masks to the plurality of original training corpora in the original sample set according to the adjustment rule, so as to obtain target training corpora corresponding to the training task;

[0164] The generation unit is used to generate a target sample set corresponding to the training task based on each target training corpus corresponding to the training task.

[0165] In an exemplary embodiment, each original training corpus in the original sample set has a corresponding annotation; the pre-trained language model has learned a mapping relationship between the input target training corpus and the annotation of the corresponding original training corpus.

[0166] In an exemplary embodiment, the first mask predicted character indicates a predicted category corresponding to the target training corpus, and the preset condition includes that the predicted category belongs to a set of predicted categories; the noise sample recognition device 700 further includes:

[0167] A third determining module 707 is configured to determine whether a first masked predicted character corresponding to the first target training corpus matches a preset condition when the first predicted category corresponding to the first target training corpus belongs to a set of predicted categories;

[0168] The fourth determining module 708 is configured to determine that the first masked predicted character corresponding to the second target training corpus does not match a preset condition when the second predicted category corresponding to the second target training corpus does not belong to the set of predicted categories.

[0169] In an exemplary embodiment, the noise sample identification apparatus 700 further includes:

[0170] Correction module 709, used to correct the noise sample;

[0171] The updating module 710 is configured to update the original sample set using the corrected noise sample.

[0172] It should be noted that the description of the embodiment of the noise sample recognition method mentioned above is also applicable to the noise sample recognition device provided in the present disclosure, and will not be repeated here.

[0173] The noise sample recognition device provided by the embodiment of the present disclosure obtains an original sample set, adopts a preset adjustment rule, adds a mask to the original training corpus in the original sample set to obtain a target sample set, uses a pre-trained language model to perform mask prediction on multiple target training corpora in the target sample set to obtain a first masked predicted character corresponding to each target training corpus, matches the first masked predicted character corresponding to each target training corpus with a preset condition, and determines that the corresponding original training corpus in the original sample set is a noise sample based on the target training corpus whose first masked predicted character in the target sample set does not match the preset condition. This makes it possible to efficiently and accurately identify noise samples from the sample set and save human resources.

[0174] The following combination Figure 8 , the model training device for noise sample recognition provided by the present disclosure is described.

[0175] Figure 8 4 is a structural diagram of a model training device for noise sample recognition according to the eighth embodiment of the present disclosure.

[0176] like Figure 8 As shown, the model training device 800 for noise sample recognition provided by the present disclosure includes: a second acquisition module 801, a second processing module 802, a second prediction module 803, an adjustment module 804 and a fifth determination module 805.

[0177] The second acquisition module 801 is used to acquire an original sample set, wherein each original training corpus in the original sample set has a corresponding annotation;

[0178] The second processing module 802 is configured to add a mask to the original training corpus in the original sample set using a preset adjustment rule to obtain a target sample set;

[0179] The second prediction module 803 is used to perform mask prediction on multiple target training corpora in the target sample set using the initial language model to obtain second mask prediction characters corresponding to each target training corpus;

[0180] An adjustment module 804 is configured to adjust model parameters of the initial language model based on the difference between the second masked predicted characters corresponding to each target training corpus and the annotations corresponding to each original training corpus;

[0181] The fifth determining module 805 is configured to determine the language model obtained after adjustment as the language model for noise sample recognition.

[0182] It should be noted that the model training device for noise sample recognition provided in this embodiment can execute the model training method for noise sample recognition in the aforementioned embodiment. The model training device for noise sample recognition can be an electronic device or software configured on the electronic device to train a language model for noise sample recognition.

[0183] Among them, the electronic device can be any stationary or mobile computing device capable of data processing, such as mobile computing devices such as laptops, smart phones, wearable devices, or stationary computing devices such as desktop computers, or servers, or other types of computing devices, etc., and this disclosure does not limit this.

[0184] It should be noted that the above description of the embodiment of the model training method for noise sample recognition is also applicable to the model training device for noise sample recognition provided by the present disclosure, and will not be repeated here.

[0185] The model training device for noise sample recognition provided by the embodiments of the present disclosure obtains an original sample set, wherein each original training corpus in the original sample set has a corresponding annotation, and uses a preset adjustment rule to add a mask to the original training corpus in the original sample set to obtain a target sample set. The initial language model is used to perform mask prediction on multiple target training corpora in the target sample set to obtain a second masked predicted character corresponding to each target training corpus. According to the difference between the second masked predicted character corresponding to each target training corpus and the annotation corresponding to each original training corpus, the model parameters of the initial language model are adjusted, and the language model obtained after adjustment is determined as the language model for noise sample recognition, thereby achieving training of the initial language model and obtaining a language model that can efficiently and accurately recognize noise samples and save human resources.

[0186] The following combination Figure 9 , the model training device for noise sample recognition provided by the present disclosure is described.

[0187] Figure 9 4 is a structural diagram of a model training device for noise sample recognition according to the ninth embodiment of the present disclosure.

[0188] like Figure 9 As shown, the model training device 900 for noise sample recognition may specifically include: a second acquisition module 901, a second processing module 902, a second prediction module 903, an adjustment module 904 and a fifth determination module 905. Figure 9 The second acquisition module 901, the second processing module 902, the second prediction module 903, the adjustment module 904 and the fifth determination module 905 are Figure 8 The second acquisition module 801 , the second processing module 802 , the second prediction module 803 , the adjustment module 804 and the fifth determination module 805 have the same function and structure.

[0189] In an exemplary embodiment, the initial language model is a masked pre-trained language model;

[0190] The model training device 900 for noise sample recognition further includes:

[0191] The sixth determining module 906 is configured to determine an adjustment rule according to the training tasks performed by the masked pre-trained language model during the pre-training process.

[0192] It should be noted that the above description of the embodiment of the model training method for noise sample recognition is also applicable to the model training device for noise sample recognition provided by the present disclosure, and will not be repeated here.

[0193] The model training device for noise sample recognition provided by the embodiments of the present disclosure obtains an original sample set, wherein each original training corpus in the original sample set has a corresponding annotation, and uses a preset adjustment rule to add a mask to the original training corpus in the original sample set to obtain a target sample set. The initial language model is used to perform mask prediction on multiple target training corpora in the target sample set to obtain a second masked predicted character corresponding to each target training corpus. According to the difference between the second masked predicted character corresponding to each target training corpus and the annotation corresponding to each original training corpus, the model parameters of the initial language model are adjusted, and the language model obtained after adjustment is determined as the language model for noise sample recognition, thereby achieving training of the initial language model and obtaining a language model that can efficiently and accurately recognize noise samples and save human resources.

[0194] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0195] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0196] like Figure 10 As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the device 1000 can also be stored in the RAM 1003. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0197] Various components in device 1000 are connected to I / O interface 1005, including an input unit 1006, such as a keyboard, mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a magnetic disk, optical disk, etc.; and a communication unit 1009, such as a network card, modem, wireless communication transceiver, etc. The communication unit 1009 allows device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0198] The computing unit 1001 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as the noise sample identification method or the model training method for noise sample identification. For example, in some embodiments, the noise sample identification method or the model training method for noise sample identification can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the noise sample identification method or the model training method for noise sample identification described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured in any other appropriate manner (eg, by means of firmware) to execute the noise sample recognition method or the model training method for noise sample recognition.

[0199] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0200] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0201] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0202] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0203] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0204] A computer system can include clients and servers. The client is a cloud server, also known as a cloud computing server or cloud host. This is a host product within the cloud computing service system that addresses the management difficulties and poor scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server can also be a distributed system server or a server integrated with blockchain.

[0205] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0206] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for identifying noise samples, comprising: Get the original sample set; Using a preset adjustment rule, masking is added to the original training corpus in the original sample set to obtain a target sample set; wherein the adjustment rule includes: concatenating each of the original training corpus and the first category label mask through a first associated word to obtain a target training corpus corresponding to the training task; the first category label mask indicates that the corresponding position is predicted as the category corresponding to the original training corpus; Using a pre-trained language model, performing mask prediction on a plurality of target training corpora in the target sample set to obtain a first mask prediction character corresponding to each of the target training corpora; Matching the first mask predicted characters corresponding to each target training corpus with a preset condition; According to the target training corpus in which the first mask-predicted character in the target sample set does not match the preset condition, it is determined that the corresponding original training corpus in the original sample set is a noise sample.

2. The method according to claim 1, wherein The pre-trained language model is obtained by training a masked pre-trained language model using multiple target training corpora in the target sample set.

3. The method according to claim 2, wherein: Before the preset adjustment rule is adopted to add a mask to the original training corpus in the original sample set to obtain the target sample set, the method further includes: The adjustment rule is determined according to the training tasks performed by the masked pre-trained language model during the pre-training process.

4. The method according to claim 3, wherein: When the training task is a text classification task; The adjustment rules also include: Each of the original training corpora is spliced ​​with the spliced ​​text to obtain the target training corpus corresponding to the training task; wherein the spliced ​​text is obtained by splicing the text fragments in the original training corpus with the second category label mask through the second associated words; wherein the second category label mask indicates that the corresponding position is predicted as the category corresponding to the text fragment.

5. The method according to claim 3, wherein: The step of using a preset adjustment rule to add a mask to the original training corpus in the original sample set to obtain a target sample set includes: According to the adjustment rule, masking is added to the plurality of original training corpora in the original sample set to obtain the target training corpora corresponding to the training task; According to each of the target training corpora corresponding to the training tasks, a target sample set corresponding to the training tasks is generated.

6. The method according to any one of claims 1 to 5, wherein: Each original training corpus in the original sample set has a corresponding annotation; the pre-trained language model has learned the mapping relationship between the input target training corpus and the annotation of the corresponding original training corpus.

7. The method according to any one of claims 1 to 5, wherein: The first masked predicted character indicates a predicted category corresponding to the target training corpus, and the preset condition includes that the predicted category belongs to a set of predicted categories; After matching the first mask predicted characters corresponding to each target training corpus with a preset condition, the method further includes: When the first prediction category corresponding to the first target training corpus belongs to the set of prediction categories, determining that the first masked predicted character corresponding to the first target training corpus matches the preset condition; When the second prediction category corresponding to the second target training corpus does not belong to the set of prediction categories, it is determined that the first masked predicted character corresponding to the second target training corpus does not match the preset condition.

8. The method according to any one of claims 1 to 5, wherein: After determining that the corresponding original training corpus in the original sample set is a noise sample based on the target training corpus whose characters predicted by the first mask in the target sample set do not match the preset condition, the method further includes: Correcting the noise sample; The original sample set is updated using the corrected noise sample.

9. A model training method for noise sample recognition, comprising: Obtaining an original sample set, wherein each original training corpus in the original sample set has a corresponding annotation; Using a preset adjustment rule, masking is added to the original training corpus in the original sample set to obtain a target sample set; Using the initial language model, performing mask prediction on a plurality of target training corpora in the target sample set to obtain a second mask prediction character corresponding to each of the target training corpora; Adjusting the model parameters of the initial language model according to the difference between the second mask predicted characters corresponding to each target training corpus and the annotations corresponding to each original training corpus; The language model obtained after adjustment is determined as a language model for noise sample recognition, and the language model is used to execute the method according to any one of claims 1 to 8.

10. The method according to claim 9, wherein: The initial language model is a masked pre-trained language model; Before the method adopts the preset adjustment rule to add a mask to the original training corpus in the original sample set to obtain the target sample set, the method further includes: The adjustment rule is determined according to the training tasks performed by the masked pre-trained language model during the pre-training process.

11. A noise sample recognition device, comprising: A first acquisition module is used to obtain an original sample set; A first processing module is configured to add a mask to the original training corpus in the original sample set using a preset adjustment rule to obtain a target sample set; wherein the adjustment rule includes: concatenating each of the original training corpus and the first category label mask using a first associated word to obtain a target training corpus corresponding to the training task; the first category label mask indicates that the corresponding position is predicted as the category corresponding to the original training corpus; A first prediction module is configured to perform mask prediction on a plurality of target training corpora in the target sample set using a pre-trained language model to obtain a first mask prediction character corresponding to each target training corpus; A matching module, configured to match the first mask predicted characters corresponding to each target training corpus with a preset condition; The first determining module is configured to determine that the corresponding original training corpus in the original sample set is a noise sample based on the target training corpus in which the first mask-predicted character in the target sample set does not match the preset condition.

12. The device according to claim 11, wherein The pre-trained language model is obtained by training a masked pre-trained language model using multiple target training corpora in the target sample set.

13. The device according to claim 12, wherein Also includes: The second determining module is configured to determine the adjustment rule according to the training tasks performed by the masked pre-trained language model during the pre-training process.

14. The device according to claim 13, wherein When the training task is a text classification task, the adjustment rule further includes: Each of the original training corpora is spliced ​​with the spliced ​​text to obtain the target training corpus corresponding to the training task; wherein the spliced ​​text is obtained by splicing the text fragments in the original training corpus with the second category label mask through the second associated words; wherein the second category label mask indicates that the corresponding position is predicted as the category corresponding to the text fragment.

15. The device according to claim 13, wherein The first processing module includes: a processing unit, configured to add masks to the plurality of original training corpora in the original sample set according to the adjustment rule, so as to obtain the target training corpora corresponding to the training task; A generating unit is used to generate a target sample set corresponding to the training task based on each target training corpus corresponding to the training task.

16. The device according to any one of claims 11 to 15, wherein: Each original training corpus in the original sample set has a corresponding annotation; the pre-trained language model has learned the mapping relationship between the input target training corpus and the annotation of the corresponding original training corpus.

17. The device according to any one of claims 11 to 15, wherein: The first masked predicted character indicates a predicted category corresponding to the target training corpus, and the preset condition includes that the predicted category belongs to a set of predicted categories; The device further comprises: a third determining module, configured to determine, when the first prediction category corresponding to the first target training corpus belongs to the set of prediction categories, whether the first masked predicted character corresponding to the first target training corpus matches the preset condition; The fourth determining module is configured to determine that the first masked predicted character corresponding to the second target training corpus does not match the preset condition when the second predicted category corresponding to the second target training corpus does not belong to the set of predicted categories.

18. The device according to any one of claims 11 to 15, wherein: Also includes: A correction module, used for correcting the noise sample; An updating module is configured to update the original sample set using the corrected noise sample.

19. A model training device for noise sample recognition, comprising: A second acquisition module is used to acquire an original sample set, wherein each original training corpus in the original sample set has a corresponding annotation; A second processing module is configured to add a mask to the original training corpus in the original sample set using a preset adjustment rule to obtain a target sample set; A second prediction module is configured to perform mask prediction on a plurality of target training corpora in the target sample set using an initial language model to obtain a second mask prediction character corresponding to each target training corpus; an adjustment module, configured to adjust model parameters of the initial language model according to differences between the second masked predicted characters corresponding to each target training corpus and the annotations corresponding to each original training corpus; A fifth determining module is configured to determine the language model obtained after adjustment as a language model for noise sample recognition, wherein the language model is used to execute the method according to any one of claims 1 to 8.

20. The device according to claim 19, wherein The initial language model is a masked pre-trained language model; Wherein, the device further includes: The sixth determination module is configured to determine the adjustment rule according to the training tasks performed by the masked pre-trained language model during the pre-training process.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8, or the method of any one of claims 9-10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 8, or to execute the method according to any one of claims 9 to 10.

23. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 8, or implements the steps of the method according to any one of claims 9 to 10.

Images