Noisy learning method for named entity recognition task

Abstract

The invention discloses a noisy learning method for a named entity recognition task, and aims to solve the technical problems of low recognition accuracy and insufficient generalization ability in the prior art. The method comprises the following steps: firstly, adopting two pre-training language models, and obtaining a low loss set and a high loss set according to classification loss division training; then, separating out-of-distribution noise from high-loss concentration; thirdly, exchanging a low-loss set and a high-loss set among the networks, further screening out a difficult sample and an in-distribution noise sample from the high-loss set, learning the in-distribution noise sample, and updating network parameters and label distribution estimation at the same time; and finally, designing a multi-constraint loss function to further reduce misleading of noise to the model. The method has the beneficial technical effects that the generalization ability of the model is improved by effectively utilizing the feature diversity of the noise tag, meanwhile, the misleading problem of the noise tag to the model is relieved, and the recognition accuracy of the model in a low-quality annotation data environment is remarkably enhanced.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and information security technology, specifically to a noisy learning method for named entity recognition tasks. Background Technology

[0002] Named entity recognition (NENT) in the cybersecurity field aims to extract various types of entities related to cybersecurity from security-related text. The results can be used for other complex tasks such as security incident extraction, IOC extraction, and cybersecurity knowledge graph construction. Currently, cybersecurity entity recognition often employs a research paradigm of "neural network model + high-quality data + strongly supervised learning," which relies on large-scale, high-quality labeled datasets. However, due to the privacy, sensitivity, and specialized nature of cybersecurity data, publicly available large-scale labeled datasets are insufficient. Furthermore, labeling cybersecurity data requires expert knowledge in the field, and due to the influence of background knowledge and subjective biases among different experts, sample labeling generates label noise. Using noisy labels as supervisory information can mislead model training, leading to overfitting to noisy samples and underfitting to clean samples. These problems significantly hinder the development and application of NENT in the field of information security. Summary of the Invention

[0003] This invention provides a noisy learning method for named entity recognition tasks to solve the technical problems of low recognition accuracy and insufficient generalization ability in existing technologies when faced with few samples or samples with a lot of noise.

[0004] Researchers have proposed various methods to address the label noise problem in NER tasks, which can be broadly categorized into two types: sample selection and label correction. The former treats low-loss data as clean data and retains it to update the neural network. However, this approach leads to a shift in label distribution. Furthermore, because the model tends to learn from simpler samples, complex samples (hard samples) and noisy samples may be mixed in high-loss data. Directly discarding high-loss data without differentiation could result in data waste and decreased model generalization performance, which is unacceptable in the information security field, where labeled data is scarce and updated rapidly. Label correction is further divided into estimating the noise transition matrix and prediction-based loss regularization. However, the noise transition matrix is ​​difficult to estimate, and prediction-based loss correction suffers from error accumulation. Both methods require a clean dataset as a prior aid.

[0005] Based on the above issues, the applicant conceived a simple idea: could clean datasets be separated first through sample selection, and then data waste reduced through label correction? To this end, the applicant selected 1000 samples from each of the five categories "TIME," "IDTY," "APT," "MAL," and "TOOL" from the APTNER dataset to form an experimental dataset. Data2 contains a mixture of noisy and clean samples, while Data3 contains only noisy samples. Specific details will be described in the experimental section. Then, using BilSTM+CRF as the backbone model, experiments were conducted using direct training (BilSTM+CRF), a sample selection strategy (BilSTM+CRF+selected), a loss correction strategy (BilSTM+CRF+correct), and a joint strategy of the two methods (BilSTM+CRF+cotraining). The results are as follows... Figure 3 As shown, the performance of the joint strategy is higher than that of a single sample selection strategy and a corrective loss strategy, but it is still far lower than the results of the model trained on high-quality labeled data. This indicates that the presence of label noise misleads the model. Furthermore, it can be seen that under the same training strategy, there is no significant difference in model performance between models trained with Dataset3 and those trained with Dataset2. This suggests that noisy samples also contain a large number of learnable features. Therefore, reducing the misleading nature of noise labels while learning their features becomes the main problem to be solved in this study.

[0006] To address the aforementioned issues, the applicant proposes a joint training model. Specifically, two networks are first trained. In each mini-batch, each network partitions the training samples based on its classification loss, resulting in a low-loss set and a high-loss set. Due to the complexity and specialization of the cybersecurity field, distribution data outside of defined labels may be incorrectly labeled as a category and used for training. This out-of-distribution noise, because its distribution does not belong to any known label, such as... Figure 4 For example, in this sequence, the context indicates that "UTF-16" is an encoding scheme and does not belong to any predefined category, yet it is incorrectly labeled as S-MAL. Training with such out-of-distribution noise may cause the model to learn an incorrect label distribution, leading to a decrease in the model's generalization ability. Therefore, the applicant introduces the idea of ​​kernel density estimation to separate out-of-distribution noise from the high-loss set, removing the misleading influence of out-of-distribution noise on the model. Then, the network exchanges low-loss and high-loss sets, further filtering out difficult samples and in-distribution noise samples from the high-loss set, learning the in-distribution noise samples in a label distribution learning manner, while updating the network parameters and label distribution estimates. Finally, a self-supervised learning module is introduced to further reduce the misleading influence of noise on the model.

[0007] Based on the above concept, in order to solve the technical problems existing in the prior art, the present invention adopts the following technical solution:

[0008] Design a noisy learning method for named entity recognition task, including the following steps:

[0009] Step S1: Obtain a noisy dataset D, with an input sequence X∈D, X={x1,x2,....,x n The corresponding sequence label is Y = {y1, y2, ..., y}. n}, for each label pair (x i ,y i ), x i It is the input token value, y i It belongs to category C = {c1, c2, ..., c} m One of them. Define a neural network model F(θ) with parameter θ, and let F(θ,X) be the probability distribution that transforms the input sequence X into the corresponding label sequence.

[0010] Step S2: First, train two models, θ1 and θ2. In each iteration B, based on the prediction results of θ1 and θ2, divide the dataset into a low-loss dataset B. low and high-loss dataset B high (B high It is further divided into B obtained from model θ1. high1 B obtained from model θ2 high2 ),

[0011] Step S3: Based on the sample selection and partitioning in step S2, kernel density estimation is used to select samples from B. high Samples that are outside the categories are selected as out-of-distribution noise samples B. out These samples do not belong to any predefined category and are discarded, thereby reducing their negative impact on model training.

[0012] Step S4: Based on the remaining sample B obtained in step S3 low The entity included is of category c. i The prototype c pi Intra-distribution noise label correction is performed by comparing the similarity between samples and prototypes of each category to correct the labels, thereby improving the robustness of the model.

[0013] Step S5: Based on the out-of-distribution noise sample B obtained in step S3 out From the high-loss set, difficult samples and in-distribution noise samples are screened out, and B′ is selected. high (B′ high =B high -B outThe samples in ) are divided into B hard and B′ noise B hard By B hard1 and B hard2 Composition, B′ noise For the noise sample after label correction in step S4, B′ noise By B' noise1 and B' noise2 Composition, B hard and B low Combined into a clean dataset B clean And based on the prediction results of models θ1 and θ2, B clean Divided into B clean1 and B clean2 Exchange B clean1 and B clean2 Update the two model parameters in θ2 and θ1 respectively;

[0014] Step S6: Finally, based on B obtained in step S5... clean and B′ noise Label distribution learning is performed. Auxiliary consistency loss is used to guide and regularize models θ1 and θ2.

[0015] Furthermore, the detailed steps of step S2 are as follows:

[0016] Step S2.1: Input the sequence X and the corresponding label sequence Y from each batch B into θ1 and θ2 respectively, and let x i ∈X, y i ∈Y. Given the predicted labels for θ1 and θ2, then the low-loss dataset B... low and high-loss dataset B high for:

[0017]

[0018] Step S2.2: This invention introduces self-supervised techniques into the model, obtaining two augmented sequences X1′ and X2′ of X through data augmentation methods, which are then used as inputs to θ1 and θ2, respectively. The self-supervised loss function can be expressed as:

[0019]

[0020] Here, project() refers to the intermediate layer representation of the sequence after encoding, and predict() represents the probability distribution of the predicted sequence.

[0021] Furthermore, the detailed steps of step S3 are as follows:

[0022] Step S3.1, from B highFirst, separate out-of-distribution noise from the dataset. Start with B... low The samples in the set are used as prototypes, assuming c i If there are n entities under a category, then c i The category prototype can be represented as:

[0023]

[0024] e j For c i Entities under a category. The label representation and entity representation are combined to obtain a new prototype representation c. i .

[0025] Step S3.2: When learning category prototypes, in addition to obtaining the mean of entity representations... In addition, each entity e should be... i Corresponding category tags To sum and take the mean, that is...

[0026]

[0027] That is, c i The category prototype can be represented as Projecting prototypes of each category into space R D Then calculate B high The distances between each type of entity and its prototype are used to distribute them into space R. D If a sample contains multiple label information, the label closest to the prototype label is selected.

[0028] Step S3.3: Next, kernel density estimation is used to calculate the surrounding density of the current sample. This can be expressed as:

[0029]

[0030] Here, x represents the current sample, x i This represents the other samples in the dataset, where h is the sliding window. This represents the density surrounding the current sample. This invention first removes the original labels from the sample that do not contain contextual features, retaining only the entity and model annotation labels. Then, sample s... i Represented as Samples s i Density is expressed as:

[0031]

[0032] Calculate sample s i After the nuclear density f(s) i According to f(s) iSort the samples in ascending order by size, and the top-n samples are the out-of-distribution noise sample set B. out The labels of the noisy samples are converted to "O" and remain in the training dataset.

[0033] Furthermore, the detailed steps of step S4 are as follows:

[0034] Step S4.1, B high The remaining samples constitute a new sample set B′ high From B′ high Get any sample Here, the symbol '[]' indicates that it is optional, and x is calculated separately. i Prototypes of different categories of entities The similarity between them and y i , With the prototype of each category label The similarity between samples b i With the cth j The similarity score of a class can be represented as

[0035]

[0036] Then, we obtain sample b. i Correction tags:

[0037]

[0038] according to B′ can be further high The samples in the dataset are divided into a hard sample set B. hard and B′ noise ,Right now

[0039]

[0040] Furthermore, the detailed steps of step S5 are as follows:

[0041] Step S5.1, B hard and B low Combined into a clean dataset B clean And based on the prediction results of models θ1 and θ2, B clean Divided into B clean1 and B clean2 Exchange B clean1 and B clean2 If we apply the classification to θ2 and θ1 respectively, then the classification losses of θ1 and θ2 on clean samples are:

[0042]

[0043] Finally, the detailed steps of step S6 are as follows:

[0044] Step S6.1, B clean Label distribution learning for the dataset. The label distribution for each sample s is obtained using the Fuzzy C-Means algorithm. i The probability distribution of belonging to each category, i.e.:

[0045]

[0046] Among them, c j This indicates that the j-th category uses its corresponding prototype. Let α represent the ambiguity, and w ij Coefficients:

[0047]

[0048] By initializing an n×m zero matrix A and updating A j :

[0049] A j =A j +w j

[0050] Then calculate s i The membership vector of this category is then normalized using softmax to generate its corresponding label distribution.

[0051]

[0052] Based on the KL divergence as B cleank Loss for (k=1,2):

[0053]

[0054] Step S6.2, B′ noise Label distribution learning. The SeqMix method is used to learn the label distribution of noisy samples. Specifically, from B′... noise Obtaining noise samples s i The sequence X containing ' i ′, from B clean Obtain clean samples s j Sequence X j , with a window of fixed length t from X i ′ and X j Obtaining subsequences and The t-th mixed word is represented as:

[0055]

[0056] Its category is represented as:

[0057]

[0058] here, Representing words Corrected label Representing words The tag.

[0059] The loss function for this part is set as follows:

[0060]

[0061] Step S6.3: To ensure consistency between the estimated label distribution on clean samples and its corresponding true labels, an auxiliary consistency loss is used to guide and regularize the model, which can be expressed as:

[0062]

[0063] Step S6.4: The final cost function of the model can be expressed as:

[0064]

[0065] The final cost loss function is derived from the self-supervised loss function. Sample exchange loss Clean Sample Label Distribution Loss Label correction loss and auxiliary consistency loss composition.

[0066] Furthermore, the training process of models θ1 and θ2 is divided into two stages. The first stage is the cold start stage, in which no label distribution learning is performed, and only high and low loss samples are divided based on self-supervision. α is set to 1, and β and γ are both set to 0. In the second stage, sample selection and label distribution learning are performed simultaneously. α is kept constant, and β and γ are gradually increased. In the testing stage, sample selection and label distribution learning no longer work. It is only necessary to select the best-performing model from θ1 and θ2 for entity recognition.

[0067] Compared with the prior art, the beneficial technical effects of the present invention are as follows:

[0068] 1. This invention proposes a noisy learning model for named entity recognition tasks in the field of information security. The model adopts a joint training strategy of sample selection and corrective label, which divides the training data into clean samples, in-distribution noisy samples and out-of-distribution noisy samples, removes the misleading influence of out-of-distribution noisy samples on the model, and realizes further utilization of in-distribution noisy samples through label distribution learning, effectively alleviating the problems of sample waste and noise label misleading.

[0069] 2. Based on the current status of labeled datasets in the field of information security, this invention proposes a sample selection strategy based on kernel density estimation to meticulously divide noisy datasets, separating difficult samples, in-distribution noise samples, and out-of-distribution noise samples from high-loss samples, thereby achieving full utilization of labeled data.

[0070] 3. This invention can provide efficient and accurate entity identification support for network security monitoring, threat intelligence extraction, risk assessment and response strategy formulation, enhance network security protection capabilities, and make an important contribution to building a safer network environment. Attached Figure Description

[0071] Figure 1 This is a schematic diagram of the overall model of the present invention.

[0072] Figure 2 This is a schematic diagram of the overall process of the present invention.

[0073] Figure 3 This is a schematic diagram of the training results for label correction in this invention.

[0074] Figure 4 This is an example of out-of-distribution noise in a noisy dataset. Detailed Implementation

[0075] The technical solution of the present invention will be described in detail below, but the scope of protection of the present invention is not limited to the embodiments described.

[0076] Example 1: A noisy learning method for named entity recognition task, see [link to example]. Figure 1 and Figure 2 This includes the following steps:

[0077] Step S1: For a noisy dataset D, there is an input sequence X∈D, X={x1,x2,....,x n The corresponding sequence label is Y = {y1, y2, ..., y}. n}, for each label pair (x i ,y i ), x i It is the input token value, y i It belongs to category C = {c1, c2, ..., c} m One of them. Define a neural network model F(θ) with parameter θ, and let F(θ,X) be the probability distribution that transforms the input sequence X into the corresponding label sequence.

[0078] Step S2: First, train two models, θ1 and θ2. In each iteration B, based on the prediction results of θ1 and θ2, divide the dataset into a low-loss dataset B. lowand high-loss dataset B high (B high It is further divided into B obtained from model θ1. high1 B obtained from model θ2 high2 ,

[0079] Step S2.1: Input the sequence X and the corresponding label sequence Y from each batch B into θ1 and θ2 respectively, and let x i ∈X, y i ∈Y. Given the predicted labels for θ1 and θ2, then the low-loss dataset B... low and high-loss dataset B high for:

[0080]

[0081] Step S2.2: This invention introduces self-supervised techniques into the model, obtaining two augmented sequences X1′ and X2′ of X through data augmentation methods, which are then used as inputs to θ1 and θ2, respectively. The self-supervised loss function can be expressed as:

[0082]

[0083] Here, `project()` refers to the intermediate representation of the encoded sequence, and `predict()` represents the predicted probability distribution of the sequence.

[0084] Step S3: Based on the sample selection and partitioning in step S2, kernel density estimation is used to select samples from B. high Samples that are outside the categories are selected as out-of-distribution noise samples B. out These samples do not belong to any predefined category and are discarded, thereby reducing their negative impact on model training.

[0085] Step S3.1, from B high First, separate out-of-distribution noise from the dataset. Start with B... low The samples in the set are used as prototypes, assuming c i If there are n entities under a category, then c i The category prototype can be represented as:

[0086]

[0087] e j For c i Entities are categorized. However, as a sequence labeling task, entity features need to be closely related to the context. Therefore, the label representation and entity representation are fused together to obtain a new prototype representation c. iFor example, merging the tag representation and entity representation of the original annotation statement "APT-C-05Organization is an overseas organization targeting attacks against XXX" yields "type:B-ORG APT-C-05type:E-ORG Organizationtype:O is type:O an type:O overseas type:O organization type:O targeting type:O attacks type:O against type:S-LOC XXX".

[0088] Step S3.2: When learning category prototypes, in addition to obtaining the mean of entity representations... In addition, each entity e should be... i Corresponding category tags To sum and take the mean, that is...

[0089]

[0090] That is, c i The category prototype can be represented as Projecting prototypes of each category into space R D Then calculate B high The distances between each type of entity and its prototype are used to distribute them into space R. D In the process, if a sample contains multiple label information, the label closest to the prototype label is selected.

[0091] Step S3.3: Next, kernel density estimation is used to calculate the surrounding density of the current sample. This can be expressed as:

[0092]

[0093] Here, x represents the current sample, x i This represents the other samples in the dataset, where h is the sliding window. This represents the density surrounding the current sample. This invention first removes the original labels from the sample that do not contain contextual features, retaining only the entity and model annotation labels. Then, sample s... i Represented as Samples s i Density is expressed as:

[0094]

[0095] Calculate sample s i After the nuclear density f(s) i According to f(s)i Sort the samples in ascending order by size, and the top-n samples are the out-of-distribution noise sample set B. out The labels of the noisy samples are converted to "O" and kept in the training dataset.

[0096] Step S4: Based on the remaining sample B obtained in step S3 low The entity included is of category c. i The prototype c pi Intra-distribution noise label correction is performed by comparing the similarity between samples and prototypes of each category to correct the labels, thereby improving the robustness of the model.

[0097] Step S4.1, B high The remaining samples constitute a new sample set B′ high From B′ high Get any sample Here, the symbol '[]' indicates that it is optional, and x is calculated separately. i Prototypes of different categories of entities The similarity between them and y i , With the prototype of each category label The similarity between samples b i With the cth j The similarity score of a class can be represented as

[0098]

[0099] Then, we obtain sample b. i Correction tags:

[0100]

[0101] according to B′ can be further high The samples in the dataset are divided into a hard sample set B. hard and B′ noise ,Right now

[0102]

[0103] Step S5: Based on the out-of-distribution noise sample B obtained in step S3 out From the high-loss set, difficult samples and in-distribution noise samples are screened out, and B′ is selected. high (B′ high =B high -B out The samples in ) are divided into B hard and B′ noise B hard By B hard1 and Bhard2 Composition, B′ noise For the noise sample after label correction in step S4, B′ noise By B' noise1 and B' noise2 Composition, B hard and B low Combined into a clean dataset B clean And based on the prediction results of models θ1 and θ2, B clean Divided into B clean1 and B clean2 Exchange B clean1 and B clean2 Update the two model parameters in θ2 and θ1 respectively.

[0104] Step S5.1, B hard and B low Combined into a clean dataset B clean And based on the prediction results of models θ1 and θ2, B clean Divided into B clean1 and B clean2 Exchange B clean1 and B clean2 If we apply the classification to θ2 and θ1 respectively, then the classification losses of θ1 and θ2 on clean samples are:

[0105]

[0106] Step S6: Finally, based on B obtained in step S5... clean and B′ noise Label distribution learning is performed. An auxiliary consistency loss is used to guide and regularize models θ1 and θ2.

[0107] Step S6.1, B clean Label distribution learning for the dataset. The label distribution for each sample s is obtained using the Fuzzy C-Means algorithm. i The probability distribution of belonging to each category, i.e.:

[0108]

[0109] Among them, c j This indicates that the j-th category uses its corresponding prototype. Let α represent the ambiguity, and w ij Coefficients:

[0110]

[0111] By initializing an n×m zero matrix A and updating A j :

[0112] A j=A j +w j

[0113] Then calculate s i The membership vector of this category is then normalized using softmax to generate its corresponding label distribution.

[0114]

[0115] Based on the KL divergence as B cleank Loss for (k=1,2):

[0116]

[0117] Step S6.2, B′ noise Label distribution learning. The SeqMix method is used to learn the label distribution of noisy samples. Specifically, from B′... noise Obtaining noise samples s i The sequence X containing ' i ′, from B clean Obtain clean samples s j Sequence X j , with a window of fixed length t from X i ′ and X j Obtaining subsequences and The t-th mixed word is represented as:

[0118]

[0119] Its category is represented as:

[0120]

[0121] here, Representing words Corrected label Representing words The tag.

[0122] The loss function for this part is set as follows:

[0123]

[0124] Step S6.3: To ensure consistency between the estimated label distribution on clean samples and its corresponding true labels, an auxiliary consistency loss is used to guide and regularize the model, which can be expressed as:

[0125]

[0126] Step S6.4: The final cost function of the model can be expressed as:

[0127]

[0128] The final cost loss function is derived from the self-supervised loss function. Sample exchange loss Clean Sample Label Distribution Loss Label correction loss and auxiliary consistency loss composition.

[0129] As can be seen from the model training process of this invention, the training process is divided into two main stages. First, in the cold start stage, the two pre-trained models, Albert and ELECTRA, use self-supervised learning and loss functions to divide the training samples into high and low loss categories. Then, in the second stage, out-of-distribution noise samples are separated from the high-loss samples using kernel density estimation technology to reduce their negative impact on model training. Furthermore, the remaining samples of the high-loss samples are treated as in-distribution noise, and the category prototypes of the low-loss samples are calculated. By comparing the similarity between the in-distribution noise and each category prototype, the labels of the in-distribution noise are corrected. Afterwards, through mixed label learning and the calculation of multi-level label correction loss, the self-supervised learning parameters are effectively updated, and the overall performance of the model is improved.

[0130] Existing named entity recognition (NAME) technologies often neglect domain-specific terminology and complex contextual information when processing text data, leading to insufficient recognition accuracy, especially in information security scenarios with noisy labels. The model in this invention effectively identifies and eliminates out-of-distribution noise from the training data, ensuring the accuracy of the learning process. Simultaneously, the model enhances the recognition of difficult samples by exchanging and filtering samples, and further optimizes the utilization of noisy samples through label distribution learning. Finally, the introduction of a self-supervised learning module improves the model's understanding of context, enabling it to move beyond purely relying on noise labeling and achieve efficient recognition of entities in the information security domain, ultimately improving the model's generalization ability and recognition accuracy.

Claims

1. A noisy learning method for a named entity recognition task, characterized in that, Comprising the following steps: Step S1, obtain a noisy dataset D, samples are s i wherein, the input sequence X ∈ D, X = {x1, x2,...., x n}, the corresponding sequence label is Y = {y1, y2,..., y n}, for each label pair (x i , y i ), x i is the input token value, y i is one of the categories C = {c1, c2,..., c m}, define a neural network model F(θ) with a parameter θ, and the probability distribution of the input sequence X converted into the corresponding label sequence is F(θ, X); Step S2: Train two pre-trained models F(θ1) and F(θ2) to model θ1 and θ2. In each iteration B, the dataset is divided into a low-loss dataset B based on the prediction results of θ1 and θ2. low and high-loss dataset B high B high It is further divided into B obtained from model θ1. high1 B obtained from model θ2 high2 A self-supervised learning module is introduced to update the two models θ1 and θ2; Step S3, based on the sample selection and division of step S2, through kernel density estimation, from B high The samples outside each category are screened out as outlier noise samples B out and discarded, reducing the negative impact on model training; Step S4, obtaining sample B based on sample A obtained in step S3 low entity included in the sample i prototype of the category c The distribution of noise label correction is carried out, and the label is corrected by comparing the similarity between the sample and the prototype of each category, so as to improve the robustness of the model. Step S5, based on the distribution of step S3 outside the noise sample B out , the difficult sample and the distribution of noise samples are screened out from the high loss set, and B' high high = B high -B out ) is divided into B hard and B' noise , B hard is composed of B hard1 and B hard2 , B' noise is the noise sample after correcting the label in step S4, B' noise is composed of B' noise1 and B' noise2 , B hard and B low are combined into a clean data set B clean , and B clean is divided into B clean1 and B clean2 according to the prediction results of the models θ1 and θ2, B clean1 and B clean2 are exchanged into θ2 and θ1 respectively, and the two model parameters are updated;​ Step S6, finally, based on the B clean and B' noise , label distribution learning is performed, and an auxiliary consistency loss is used to guide and regularize the models θ1 and θ2.

2. The noisy learning method of a named entity recognition task according to claim 1, characterized in that, The step S2 comprises: Step S2.1, input the sequence X and the corresponding label sequence Y in each batch B into θ1 and θ2 respectively, set x i ∈X, y i ∈Y, is the predicted label of θ1 and θ2, then the low-loss data set B low and the high-loss data set B high are: Step S2.2, introduce the self-supervised technique into the model F(θ), obtain two enhanced sequences X1' and X2' of X through a data enhancement method, then input them as θ1 and θ2 respectively, and the self-supervised loss function can be represented as: Wherein, project() refers to the intermediate layer representation of the sequence after encoding, and predict() represents the probability distribution of predicting the sequence. 3.The noisy learning method of a named entity recognition task according to claim 1, characterized in that, The step S3 comprises: Step S3.1, with B low The samples in the set as prototypes, assuming c i There are n entities in total under the category, then c i The category prototype can be represented as: e j For c i The entity under the category, the label representation and the entity representation are jointly fused to obtain a new prototype representation c i , Step S3.2, learning class prototypes, obtaining the mean of the entity representations and each entity e i the corresponding class label summing and averaging, i.e. c i The category prototypes are represented as Each category prototype is projected into the space R D and then the distances of each category entity in B high to the prototype are calculated and distributed into the space R D If the sample contains multiple label information, the label with the closest distance to the prototype label is selected, Step S3.3, the surrounding density of the current sample is calculated by using kernel density estimation, which can be represented as: where x represents the current sample, x i represents other samples in the dataset, H is the sliding window, the density around the current sample, and the original label of the sample which does not contain the context feature is removed, only the entity and the model annotation label are reserved, then the sample s i is expressed as The sample s i The density is expressed as: Calculate sample s i nuclear density f(s) i After that, according to f(s) i Sort the samples in ascending order by size, and the top-n samples are the out-of-distribution noise sample set B. out The labels of the noisy samples are converted to "O" and remain in the training dataset. 4.The noisy learning method of a named entity recognition task according to claim 1, characterized in that, The step S4 comprises: Step S4.1, B high The remaining samples in B high constitute a new sample set B' high Any sample b from B' i is obtained The symbol '[]' indicates optional, respectively calculating x i and the similarity between different categories of entity prototypes and the similarity between each category label prototype The similarity score of sample b i to the c j th category can be expressed as Then, the sample b i is obtained, and the correction label is According to The samples in B' can be further divided into a difficult sample set B high and B', i.e. hard and B', i.e. noise ​ 5.The noisy learning method for named entity recognition tasks of claim 1, wherein, The step S5 comprises: Step S5.1, combine B hard and B low into clean dataset B clean and split B clean into B clean1 and B clean2 according to the prediction results of models θ1 and θ2, exchange B clean1 and B clean2 into θ2 and θ1 respectively, then the classification loss of θ1 and θ2 on clean samples is: 6.The noisy learning method for named entity recognition tasks of claim 1, wherein, The step S6 comprises: Step S6.1, obtaining each sample s according to the Fuzzy C-Means algorithm i The probability distribution belonging to each class, i.e.: where c j represents the jth category with its corresponding prototype represents, a represents ambiguity, w ij represents the coefficient: By initializing an n x m zero matrix A and updating A j : A j = A j + w j Then calculate s i For the membership vector of this category, the corresponding label distribution is generated by softmax normalization: According to the KL divergence as B cleank loss of (k = 1, 2): Step S6.2, learn the label distribution of noisy samples using SeqMix method, from B′ noise acquire noisy samples s i from the sequence X i , from B clean acquire clean samples s j from the sequence X j , respectively acquire subsequences from X i and X j with a fixed length t window and The t-th mixed word is expressed as: The category thereof is represented as: wherein, the word corrected label, the word the label, The part of the loss function is set as: Step S6.3, the auxiliary consistency loss is used to guide and regularize the model, which can be represented as: Step S6.4, the final cost function of the model can be represented as: wherein the final cost loss function comprises a self-supervised loss function sample exchange loss clean sample label distribution loss label correction loss and auxiliary consistency loss α, β, γ are weight coefficients of the loss functions.

7. The noisy learning method of named entity recognition tasks according to claim 6, characterized in that, The training process of the model θ1 and θ2 is divided into two stages, the first stage is the cold start stage, in this stage, the label distribution learning is not carried out, only the high and low loss sample division is carried out based on the self-supervision, α is set to 1, β and γ are set to 0, in the second stage, sample selection and label distribution learning are carried out at the same time, α is kept unchanged, β and γ are gradually increased; in the test, sample selection and label distribution learning no longer work, only the model with the best performance needs to be selected from θ1 and θ2 for entity recognition.

Images