Cross-language named entity recognition method and device based on large interval representation learning and medium

Abstract

The invention discloses a cross-language named entity recognition method and device based on large interval representation learning and a medium, and the method comprises the steps: obtaining a source language annotated text sequence and a target language unannotated text sequence, and respectively extracting a first span and a second span; and training the source model by using the first span, and generating a pseudo tag corresponding to the second span. And according to the target language span feature and the similarity between the target language span feature and the category center to which the target language span feature belongs, evaluating the false label confidence, and dividing the false label confidence into high confidence and low confidence. And training a target model by using a joint loss function in combination with a source language annotation span and a high-confidence target language pseudo-tag span. And correcting the low-confidence false label to participate in subsequent iterative training of the target model so as to utilize more target language data. The trained target model is used for identifying the named entity in the target language text. The core of the method is to improve the cross-language named entity recognition performance by combining large interval representation learning with pseudo tag screening and dynamic correction.

Description

Technical Field

[0001] The present invention relates to the technical field of cross-language named entity recognition, and in particular to a cross-language named entity recognition method, device, and medium based on large-margin representation learning. Background Art

[0002] Named Entity Recognition (NER) is a fundamental task in the field of information extraction. Its goal is to identify entity fragments from a given raw text and classify them into predefined entity types. In recent years, deep learning models have achieved remarkable success in named entity recognition tasks, but the performance of these models is highly dependent on large-scale annotated training data. However, for low-resource languages, obtaining high-quality annotated data is extremely expensive or even unrealistic. This data scarcity problem severely limits the application of named entity recognition models in low-resource languages. To address this problem, cross-lingual named entity recognition technology has emerged. By leveraging annotated data from high-resource languages ​​(such as English), it can solve the named entity recognition task in a low-resource target language, thereby alleviating the dilemma of data scarcity.

[0003] To bridge the gap between high-resource source languages ​​and low-resource target languages, current mainstream approaches rely primarily on multilingual pre-trained language models combined with self-training frameworks. These approaches typically employ a teacher-student learning mechanism, which consists of two key steps: (1) training a teacher model on labeled data in the source language; and (2) training a student model on unlabeled data in the target language using pseudo-labels generated by the teacher model. To further enhance cross-lingual transferability, recent research has also introduced contrastive learning paradigms, which enhance the model's cross-lingual generalization capabilities by explicitly aligning the distribution of identical entity classes in the source and target languages ​​in a shared representation space.

[0004] From a theoretical perspective, the core goal of cross-lingual named entity recognition methods is to learn a language-independent feature distribution that accurately reflects the underlying distribution of each entity type and generalizes well to the target language. According to classical learning theory, the optimal generalization error is achieved under the maximum margin assumption, that is, each entity cluster should be closely distributed around its class center, while the maximum margin should be maintained between different clusters. However, the current state-of-the-art cross-lingual named entity recognition methods fail to achieve this goal in practical applications. Specifically, the high similarity between class centers in existing algorithms leads to dense interweaving of clusters. This distribution characteristic severely hinders the classifier's ability to distinguish unseen target language data that deviates from the class center in a cross-lingual setting. In contrast, ideally, the classifier should be able to achieve a clear separation between clusters in the feature space, thereby demonstrating strong generalization capabilities.

[0005] Through in-depth analysis of existing methods, we can summarize two key reasons why the algorithm fails to achieve the optimal interval:

[0006] (1) Distribution skew: Entities of different categories often have highly unbalanced distributions in the dataset. This extreme distribution skew causes minority entities to collapse into very small clusters in the feature space, making it difficult for them to occupy sufficient space, thus affecting the classifier’s discriminative ability.

[0007] (2) Pseudo-label bias: During cross-lingual adaptation, pseudo-label generation often introduces noise and confusion, especially for easily confused entity categories. This pseudo-label bias blurs the boundaries between different categories in the feature space, further exacerbating the overlap between clusters. In addition, the class imbalance problem makes minority class entities more likely to be misclassified as majority class during pseudo-label generation, further weakening the generalization performance of the model.

[0008] Therefore, how to design a method in the cross-lingual named entity recognition task that can effectively alleviate distribution skew, reduce pseudo-label bias, and achieve the maximum margin distribution between clusters, thereby improving the generalization ability of the model in low-resource target languages, has become a key scientific problem that needs to be solved urgently. Summary of the Invention

[0009] In view of the shortcomings of the existing technology, the present invention provides a cross-language named entity recognition method, device and medium based on large-margin representation learning.

[0010] In a first aspect, an embodiment of the present invention provides a cross-language named entity recognition method based on large-margin representation learning, the method comprising the following steps:

[0011] Obtain a plurality of annotated first text sequences corresponding to the source language and a plurality of unannotated second text sequences corresponding to the target language; obtain a first span corresponding to each first text sequence and a second span corresponding to each second text sequence; perform mapping conversion on the labels in the first text sequences to obtain a label corresponding to each first span;

[0012] Training a source model based on the first span; inputting the second span into the trained source model, and using the output of the source model as the pseudo label corresponding to the second span;

[0013] Determining the confidence of the pseudo label according to the similarity between the span feature corresponding to each second span and the center of the category to which it belongs, thereby classifying the second span into a second span with a pseudo label of high confidence and a second span with a pseudo label of low confidence;

[0014] training a target model based on the first span and the second span having pseudo labels with high confidence according to a joint loss function that is a weighted sum of an equiangular tight frame loss function and a classification loss function;

[0015] Correct the pseudo labels with low confidence so that the second span of the pseudo labels with low confidence participates in the iterative training of the target model;

[0016] The trained target model is used as the target named entity recognition model; the target language text sequence to be tested is input into the target named entity recognition model to obtain the cross-language named entity recognition result.

[0017] In a second aspect, an embodiment of the present invention provides an electronic device comprising a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned cross-language named entity recognition method based on large-interval representation learning.

[0018] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned cross-language named entity recognition method based on large-margin representation learning.

[0019] In a fourth aspect, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the above-mentioned cross-language named entity recognition method based on large-margin representation learning.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention discloses a cross-language named entity recognition method based on large-margin representation learning, which can effectively deal with the distribution skew and pseudo-label bias problems existing in the cross-language migration process. During the training process of the target model, by introducing the equiangular tight frame (ETF) loss, features of different categories are forced to form a maximum margin distribution in the feature space, making the features of the same entity more compact and the distinction between different categories significantly enhanced, thereby improving the discrimination ability of the target model on unseen target languages. At the same time, the present invention corrects the pseudo-labels with low confidence so that the second span of the pseudo-labels with low confidence participates in the iterative training of the target model. By combining the sample screening strategy and the dynamic pseudo-label correction mechanism, high-confidence pseudo-label samples can be screened out to participate in the training to avoid the error accumulation caused by noise labels, and the low-confidence pseudo-labels are dynamically updated through global similarity and local neighborhood consistency, gradually improving the quality and reliability of the pseudo-labels, further promoting the formation of the maximum margin distribution, and enhancing the recognition accuracy and generalization ability of the target model in a cross-language environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A flowchart of a cross-language named entity recognition method based on large-margin representation learning provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0026] It should be noted that, unless there is any conflict, the features in the following embodiments and implementations may be combined with each other.

[0027] like Figure 1 As shown, an embodiment of the present invention provides a cross-language named entity recognition method based on large-margin representation learning, the method comprising the following steps:

[0028] Step (1): Get the N corresponding to the source language s The first text sequence with annotations and the corresponding N in the target language t an unlabeled second text sequence; obtaining a first span corresponding to each first text sequence and a corresponding second span corresponding to each second text sequence; and performing mapping conversion on the labels in the first text sequence to obtain an entity label corresponding to each first span.

[0029] Step (1.1): For the source language, given N s The first annotated text sequence represents the i-th first text sequence, Represents the label sequence corresponding to the i-th first text sequence, i∈N s ; For the target language, given N t Unlabeled second text sequence Represents the i-th second text sequence.

[0030] It should be noted that in this example, the text sequence X is composed of multiple word units. For a text sequence containing L word units, it can be expressed as X = {x1, x2, ..., x L}, the label sequence Y contains the label corresponding to each word, that is, Y = {σ1,σ2,…,σ L}.

[0031] Step (1.2): Get the first span corresponding to each first text sequence to form a first span set And the second span corresponding to each second text sequence forms a second span set Mapping conversion is performed on the labels in the first text sequence to obtain entity labels corresponding to each first span.

[0032] Specifically, for each text sequence X, all possible spans s = {x j ,x j+1 ,…,x J}, where x j Indicates the starting word, i.e., the jth word, x J Indicates the terminal word, i.e. the Jth word;

[0033] For the source language, obtain the first span s1 corresponding to each first text sequence to form a first span set For the target language, the second span s2 corresponding to each second text sequence forms a second span set First span collection With the second span set Combined together to form a span set

[0034] Furthermore, the labels in the first text sequence are mapped and transformed to obtain each span s1 i The corresponding entity label y i ∈{0,1,…,c,…,C-1}, where 0 represents non-entity, 1~x–1 represent different entity types, and C is the number of entity types.

[0035] Step (2): Train a source model based on the first span; input the second span into the trained source model, and use the output of the source model as the pseudo label corresponding to the second span.

[0036] Specifically, in this example, the training process of the source model follows a self-training framework. The source model includes an encoder, a feature mapping layer, and an entity class classification layer configured based on the XLM-R multilingual pre-trained model. The first text sequence is input into the encoder to obtain a word vector representation, which is then combined with span information and passed through the feature mapping layer to obtain a normalized span representation. Finally, a predicted probability is obtained through an entity class classification layer, and the cross-entropy loss is optimized based on the predicted probability and the actual label of the span. After the source model training is completed, the trained source model is used to predict pseudo labels for the unlabeled data in the target language.

[0037] Furthermore, the step (2) specifically includes the following sub-steps:

[0038] Step (2.1): Construct a source model, which includes an encoder, a feature mapping layer, and an entity class classification layer configured based on the XLM-R multilingual pre-trained model.

[0039] Step (2.2): Transform the first text sequence D src Input into the source model constructed in step (2.1), first pass through the encoder E to obtain the word vector representation h of each word x, and then according to each first span s1={x j ,x j+1 ,…,x J}, get the first span representation corresponding to each first span Among them, h j is the starting word x j The corresponding word vector representation, h J is the terminal word x J The corresponding word vector representation is, l is the length information corresponding to the first span, m is the lexical information corresponding to the first span; then, the first span is represented Input feature map layer f map , mapping it to the normalized first span representation z1 of dimension q; finally, the normalized first span representation z1 is input into the entity classification layer f cls That is, the predicted probability p = softmax(f cls (z1)).

[0040] Step (2.3): Based on the predicted probability obtained in step (2.2) and the actual label y of the first span, the source model is trained using cross entropy loss. The loss function is The specific expression is as follows:

[0041]

[0042] Where, represents the cross entropy loss function adopted by the source model, Represents the first span set, s1i Indicates the first span of i, y i represents the actual label corresponding to the first span of i, Indicates that the source model predicts the first span label of i is y i probability.

[0043] After the training is completed, the source model M obtained by training is used src Predict pseudo labels for the unlabeled data in the target language, i.e. for each second span Input it into the trained source model M src Get the predicted probability (also known as soft false label) And the corresponding hard fake label represents the probability that the source model predicts that the second span label of i is j.

[0044] Step (3): The confidence of the pseudo label is determined based on the similarity between the span feature corresponding to each second span and the center of the category to which it belongs, thereby screening the second span.

[0045] Furthermore, the step (3) specifically includes the following sub-steps:

[0046] Step (3.1): Calculate the category center μ of each category c using momentum update c , the specific calculation formula is as follows:

[0047]

[0048] in, represents the category center after the tth update, β is the smoothing coefficient; Z c is the set of all sample features belonging to category c, including the first span and the second span; z c Represents the sample features belonging to category c. c∈{0,1,…,c,…,C-1}, c is the c-th category, and C is the number of entity types.

[0049] Step (3.2): For each second span, use cosine similarity to measure its normalized span representation z2 i Category center corresponding to its pseudo label The similarity between them is expressed as follows:

[0050]

[0051] Among them, ‖·‖ represents the L2 norm of the vector, Represents the hard pseudo label of the second span of i. For each category c, this example uses a two-component Gaussian mixture model to fit the distribution of cosine similarity, corresponding to reliable samples with high confidence pseudo labels and other unreliable samples. Due to the clustering effect, in the feature space, the sample representation is often more similar to the class center representation of the true category to which it belongs, which means that the similarity The higher the second span z2 i The higher the confidence of the pseudo-label, the higher the Indicates the second span s2 i The probability of belonging to the Gaussian component g with a larger mean, which also reflects the pseudo label The probability of being correct. Then, reliable pseudo-labeled samples are identified as:

[0052]

[0053] Where c represents the cth category.

[0054] Finally, the reliable samples are combined into The remaining samples constitute the unreliable sample subset

[0055] Step (4) trains the target model based on the first span and the second span with high-confidence pseudo-labels according to a joint loss function, wherein the joint loss function is a weighted sum of an equiangular tight frame loss function and a classification loss function.

[0056] Furthermore, the step (4) specifically includes the following sub-steps:

[0057] Step (4.1): Build a target model. The architecture of the target model is consistent with the source model, including an encoder, feature mapping layer, and entity classification layer configured based on the XLM-R multilingual pre-trained model.

[0058] Step (4.2): Combine the first set of labeled spans according to the joint loss function and the second set of spans with filtered pseudo labels with high confidence Training target model M tgt .

[0059] The joint loss function is the weighted sum of the equiangular tight frame loss function and the classification loss function. The sum of the weights corresponding to the equiangular tight frame loss function and the classification loss function is 1. The expression is as follows:

[0060]

[0061] Among them, α is a dynamically adjusted weight coefficient used to balance the contribution of representation learning and classifier learning; is the equiangular tight frame loss function, is the classification loss function.

[0062] It should be noted that in the early stages of training, the value of α is large, and the target model mainly focuses on optimizing the representation learning loss. Represented by the effective features of the learning data; as the training progresses, α gradually decreases, and the focus of the target model gradually shifts to the classifier learning loss to optimize the performance of the classifier.

[0063] On the one hand, by optimizing the ETF loss, large-margin representation learning is performed to achieve alignment of entity features of the same type in the source and target languages ​​and maximize the separation of entity features of different types. First, a predefined equiangular tight frame structure is introduced, that is, a matrix Where K is the number of categories and q is the feature dimension. Each vector e k Represents the target feature vector of a category, and these vectors satisfy the properties of the equiangular tight frame, that is:

[0064]

[0065] in, Is an indicator function, which is 1 when k1=k2, otherwise it is 0. Given a feature representation z2 of a target language sample i and its corresponding pseudo label ETF losses are defined as:

[0066]

[0067] Where τ is the temperature parameter, Representation category The target feature vector of . Similarly, the feature representation of the source language sample uses its true label y i , the overall ETF loss function is as follows:

[0068]

[0069] By minimizing this loss, the target model transforms the feature z1 of the first span sample into i and the features z2 of the second span samples i Zoom in to the target vector of its corresponding category The features of similar entities in the source and target languages ​​are aligned, while the features are kept away from target vectors of other categories, ultimately forming a uniform and maximally separated geometric structure in the feature space.

[0070] On the other hand, the classifier is trained by optimizing the classification loss, i.e., the cross entropy loss based on the target model prediction probability and the span label. Use its true label y i , for the second span corresponding to the target language Then use its pseudo label The overall classification loss function is as follows:

[0071]

[0072] Step (5): Modify the pseudo labels with low confidence so that the second span of the pseudo labels with low confidence participates in the iterative training of the target model.

[0073] Furthermore, the step (5) specifically includes the following sub-steps:

[0074] Step (5.1): In order to make full use of the clustering information to correct the second span with low confidence pseudo labels The pseudo-label of , needs to combine global information and local information at the same time. Among them, the global information is the similarity between the span feature corresponding to each second span and the category center corresponding to its pseudo-label Please refer to step (3.2) for the specific calculation method; local information is the sample k-nearest neighbors belonging to the category The number of samples is expressed as

[0075] Step (5.2): Determine the global correction direction d based on global information c (s2 i ,j), that is, a binary vector whose j-th element is:

[0076]

[0077] in, is the indicator function, is the average similarity of category j. If the second span s2 has a low confidence pseudo label i If the similarity with category j is higher than the average similarity of the category, then d c (s2 i ,j)=1, otherwise it is 0.

[0078] Step (5.3): Determine the local correction direction d based on local information n (s2 i ,j), which is also a binary vector, and its j-th element is:

[0079]

[0080] in, is the average number of neighbors for category j. If the second span s2 has a low confidence pseudo label iIf the number of samples of category j in the nearest neighbors of is higher than the average number of samples of the category, then d n (s2 i ,j)=1, otherwise it is 0.

[0081] Step (5.4): Integrate global information and local information to correct the global direction d c (s2 i ) and local correction direction d n (s2 i ) is weighted summed to obtain the final correction direction d(s2 i ):

[0082] d(s2 i )=Norm(d c (s2 i )+d n (s2 i ))

[0083] Wherein, Norm(·) represents the normalization operation.

[0084] Step (5.5): Use the final corrected direction d(s2 i ) For the second span sample s2 with low confidence pseudo label i Soft fake label To update:

[0085]

[0086] Among them, γ is the correction strength parameter, which controls the weight between the original pseudo label and the correction direction. The updated pseudo label It will be used in the next round of training to gradually improve the quality of pseudo labels, thereby helping more target data pass screening and participate in training.

[0087] Step (6): Through multiple rounds of iterative optimization, the trained target model is used as the target named entity recognition model M tgt , the target language text sequence X to be tested is processed by word segmentation and then input into the target named entity recognition model M tgt Perform named entity recognition and output label results

[0088] Example 1

[0089] In this example, we assume we have a labeled dataset whose source language is English and an unlabeled dataset whose target language is Spanish. Our goal is to use cross-lingual transfer learning to train a model using the labeled data in English and achieve high-quality named entity recognition in Spanish.

[0090] Step (1): Enumerate spans and map labels. For a text sequence in the source language (English), such as "Peter Sutherland was born in Hawaii.", we enumerate all possible spans, such as "Peter Sutherland" and "Hawaii", and map them to corresponding entity labels: Peter is the starting token, whose label is b-PER, and Sutherland is the ending token, whose label is e-PER, so that the span "Peter Sutherland" is labeled as "PER" (name) and "Hawaii" is labeled as "LOC" (place). For a text sequence in the target language (Spanish), such as "Francisco García juega en el FC Barcelona.", we also enumerate all possible spans, such as "Francisco García" and "FC Barcelona", but these spans are not labeled at this time.

[0091] Step (2): Train the source model. This example uses English labeled data to train the source model. The source model is built based on the XLM-R multilingual pre-trained model, including an encoder, a feature mapping layer, and an entity classification layer. The English text "Peter Sutherland was born in Hawaii." is input into the encoder to obtain a word vector representation. Combined with span information (such as "Peter Sutherland" and "Hawaii"), the span representation is generated through the feature mapping layer, and finally the entity classification layer outputs the entity category prediction probability of each span. During the training process, by optimizing the cross entropy loss function, the source model can accurately predict "Peter Sutherland" as "PER" and "Hawaii" as "LOC".

[0092] Step (3): Generate pseudo labels. After training, we use the source model to generate pseudo labels for the unlabeled Spanish data. For example, for the Spanish sentence "Francisco García juega en el FC Barcelona.", the model predicts "Francisco García" as "PER" and "FC Barcelona" as "ORG". These predictions serve as pseudo labels for subsequent target model training.

[0093] Step (4): Filter high-confidence pseudo-labels. In the feature space, we calculate the similarity between the target language sample and the center of its category. For example, for "Francisco García" and "FC Barcelona", if their feature representations have a high similarity with the center of the "PER" and "ORG" categories, these pseudo-labels are retained; if the similarity is low (such as the prediction confidence of a certain span is lower than the threshold), they are discarded. In this way, we filter out high-confidence pseudo-label data, such as "Francisco García→PER" and "FC Barcelona→ORG".

[0094] Step (5): Train the target model. We combine the annotated data in English (e.g., “Peter Sutherland→PER” and “Hawaii→LOC”) with the filtered pseudo-labeled data in Spanish (e.g., “Francisco García→PER” and “FCBarcelona→ORG”) to train the target model. The architecture of the target model is consistent with the source model. During training, we optimize the ETF loss to align the features of similar entities in English and Spanish (e.g., “PER” and “PER”) in the feature space, while maximizing the feature separation of different entities (e.g., “PER” and “LOC”). In addition, we optimize the classification loss to improve the performance of the entity classifier.

[0095] Step (6): Dynamically correct pseudo-labels. During training, we use the clustering information obtained from representation learning to dynamically correct pseudo-labels with low confidence. For example, if the pseudo-label of a Spanish sample is initially predicted to be "ORG", but its feature representation has a low similarity to the center of the "ORG" category and is closer to the center of the "LOC" category, then its pseudo-label is corrected to "LOC". In this way, we gradually improve the quality of pseudo-labels and introduce more target language data to participate in the next round of training.

[0096] Step (7): Multiple iterations and final model. We repeat the above steps for multiple rounds of iterative optimization. Each round corrects the pseudo-labels and introduces more high-quality target language data. Ultimately, we obtain a high-performance target named entity recognition model. For example, on the Spanish test set, the model can accurately identify "Francisco García" as "PER", "FC Barcelona" as "ORG", and "Madrid" as "LOC".

[0097] In summary, the present invention discloses a cross-language named entity recognition method based on large-margin representation learning, which can effectively deal with the distribution skew and pseudo-label bias problems existing in the cross-language transfer process. During the training process of the target model, by introducing the equiangular tight frame (ETF) loss, features of different categories are forced to form a maximum margin distribution in the feature space, making the features of the same entity more compact, and the discrimination between different categories is significantly enhanced, thereby improving the discrimination ability of the target model in the unseen target language. At the same time, the present invention corrects the pseudo-labels with low confidence so that the second span of the pseudo-labels with low confidence participates in the iterative training of the target model. By combining the sample screening strategy and the dynamic pseudo-label correction mechanism, high-confidence pseudo-label samples can be screened out to participate in the training to avoid the error accumulation caused by noise labels, and the low-confidence pseudo-labels are dynamically updated through global similarity and local neighborhood consistency, gradually improving the quality and reliability of the pseudo-labels, further promoting the formation of the maximum margin distribution, and enhancing the recognition accuracy and generalization ability of the target model in a cross-language environment.

[0098] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned cross-language named entity recognition method based on large-interval representation learning. Figure 2 As shown in FIG, a hardware structure diagram of a device with data processing capability in which the cross-language named entity recognition method based on large interval representation learning provided by an embodiment of the present invention is located, except Figure 2 In addition to the processor, memory, and network interface shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0099] Accordingly, the present application also provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the cross-language named entity recognition method based on large-interval representation learning as described above. The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0100] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only.

[0101] It will be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A cross-language named entity recognition method based on large margin representation learning, characterized in that: The method comprises the following steps: Obtain a plurality of annotated first text sequences corresponding to the source language and a plurality of unannotated second text sequences corresponding to the target language; obtain a first span corresponding to each first text sequence and a second span corresponding to each second text sequence; perform mapping conversion on the labels in the first text sequences to obtain a label corresponding to each first span; Training a source model based on the first span; inputting the second span into the trained source model, and using the output of the source model as the pseudo label corresponding to the second span; Determining the confidence of the pseudo label according to the similarity between the span feature corresponding to each second span and the center of the category to which it belongs, thereby classifying the second span into a second span with a pseudo label of high confidence and a second span with a pseudo label of low confidence; training a target model based on the first span and the second span having pseudo labels with high confidence according to a joint loss function that is a weighted sum of an equiangular tight frame loss function and a classification loss function; Correct the pseudo labels with low confidence so that the second span of the pseudo labels with low confidence participates in the iterative training of the target model; The trained target model is used as the target named entity recognition model; the target language text sequence to be tested is input into the target named entity recognition model to obtain the cross-language named entity recognition result.

2. A cross-language named entity recognition method based on large margin representation learning according to claim 1, characterized in that: The process of training the source model based on the first span includes: Constructing a source model, wherein the source model includes an encoder, a feature mapping layer, and an entity classification layer connected in sequence; Input the first text sequence into the encoder of the source model to obtain the word vector representation corresponding to each word in the first text sequence, thereby obtaining the first span representation corresponding to each first span; Inputting the first span representation into the feature mapping layer to obtain a normalized first span representation; The normalized first span representation is input to the entity classification layer to obtain the predicted probability; The source model is trained by constructing a cross-entropy loss based on the predicted probabilities and the actual labels corresponding to the first span.

3. A cross-language named entity recognition method based on large margin representation learning according to claim 2, characterized in that: The first span representation includes a word vector representation corresponding to the starting word unit, a word vector representation corresponding to the ending word unit, length information corresponding to the first span, and lexical information corresponding to the first span.

4. A cross-language named entity recognition method based on large margin representation learning according to claim 1, characterized in that: The process of inputting the second span into the trained source model and using the output of the source model as the pseudo label corresponding to the second span includes: Each second span is input into the trained source model, the predicted probability output by the source model is used as the soft false label, and the maximum value of the predicted probability is used as the hard false label.

5. A cross-language named entity recognition method based on large margin representation learning according to claim 1, characterized in that: The confidence of the pseudo label is determined based on the similarity between the span feature corresponding to each second span and the center of the category to which it belongs, thereby screening the second spans. The process includes: Calculate the category center corresponding to each category; Inputting the second spans into the source model to obtain a normalized span representation corresponding to each second span; Calculate the similarity between the normalized span feature corresponding to each second span and the category center corresponding to its pseudo label; A two-component Gaussian mixture model is used to fit the distribution of similarity, where the first component corresponds to the second span of pseudo labels with high confidence, and the second component corresponds to the second span of pseudo labels with low confidence.

6. A cross-language named entity recognition method based on large margin representation learning according to claim 1, characterized in that: The process of training the target model based on the first span and the second span with high-confidence pseudo-labels according to the joint loss function includes: Constructing a target model, wherein the target model includes an encoder, a feature mapping layer, and an entity classifier layer connected in sequence; Constructing a joint loss function, where the joint loss function is a weighted sum of an equiangular tight frame loss function and a classification loss function, where the sum of the weights corresponding to the equiangular tight frame loss function and the classification loss function is 1; the equiangular tight frame loss function is used to perform large-margin representation learning to achieve alignment of entity features of the same type in the source and target languages ​​and maximize the separation of features of entities of different types; The target model is trained based on the first span and the second span with high-confidence pseudo-labels according to the joint loss function.

7. A cross-language named entity recognition method based on large margin representation learning according to claim 1, characterized in that: The process of correcting the pseudo-label with low confidence so that the second span of the pseudo-label with low confidence participates in the iterative training of the target model includes: The similarity between the span feature corresponding to each second span and the category center corresponding to its pseudo label is used as global information; The number of samples of the span feature corresponding to each second span that belongs to its pseudo label in the k-nearest neighbors is used as local information; Determine the global correction direction based on global information, and determine the local correction direction based on local information; and take the weighted sum of the global correction direction and the local correction direction as the final correction direction; The final correction direction is used to correct the pseudo labels with low confidence, so that the second span of the pseudo labels with low confidence participates in the iterative training of the target model.

8. An electronic device comprising a memory and a processor, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the cross-language named entity recognition method based on large-interval representation learning as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the cross-language named entity recognition method based on large-margin representation learning is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the cross-language named entity recognition method based on large-margin representation learning described in any one of claims 1 to 7 is implemented.

Images