A Chinese entity recognition method, device and equipment

Abstract

The application provides a Chinese entity recognition method, device and equipment, the method comprises: generating a language model based on a masked entity to process an original sample containing an entity label to generate an enhanced sample with a second entity; obtaining a domain expansion interpretation about the entity label; inputting the original sample, the enhanced sample and the domain expansion interpretation into a recognition model for training, and performing label semantic fusion on the input data based on a label semantic fusion module in the recognition model, guiding the recognition model to learn and generate a sample representation containing semantic information of the entity label; adjusting the training loss of the recognition model based on the sample representation and a preset adjustment factor, so that the recognition model has a self-adaptive tuning function for the loss weight of easily distinguishable entity types and difficult distinguishable entity types; and performing a Chinese entity recognition task based on the trained recognition model. The method of the application can enable the recognition model to support low-resource training and learn label semantic information, thereby improving recognition accuracy.

Description

Technical Field

[0001] This invention relates to the field of named entity recognition technology, and in particular to a Chinese entity recognition method, apparatus and device. Background Technology

[0002] For current mainstream deep learning-based named entity recognition models to achieve good performance, a large amount of high-quality labeled data from the domain is required for training. However, for some low-resource domains or languages, obtaining such high-quality labeled data is difficult. This not only relies on rich domain knowledge and expert guidance, but also involves high costs for manual annotation. These methods often experience significant performance degradation because they cannot fully learn the hidden feature representations. Data augmentation is one of the most direct and effective methods to solve the low-resource problem. Its core idea is to perform operations such as deletion, replacement, insertion, and back-translation on the given data to augment the samples.

[0003] In recent years, data augmentation methods have been successfully applied to many natural language processing tasks, such as sentence classification, machine translation, and dialogue generation. In these tasks, data augmentation operations such as deletion, replacement, insertion, and back-translation do not change the original semantic labels of the sentences. However, for named entity recognition (NER), it is necessary to predict the label of each character in the sequence samples. Traditional data augmentation operations may destroy character-level semantics, leading to inaccurate predictions by the model. Furthermore, back-translation or generative data augmentation methods often suffer from entity label misalignment when applied to character-level tasks like NER, failing to achieve ideal results. Some methods propose using the semantic information in the label names as additional signals and prior knowledge input to the model, while others propose introducing fine-grained type information from entity annotations as auxiliary augmentation, which helps the model understand text semantics. However, the vast majority of existing fine-grained annotation datasets are in English, and although there are a few related annotation datasets in other languages, their scale cannot compare to that of English datasets. Moreover, these methods often simply concatenate the label semantic embeddings and the original sample embeddings to effectively utilize the semantic information of the labels. While this approach is simple and direct, it doesn't effectively guide model learning and increases the length of training input samples. Furthermore, entity recognition samples in low-resource scenarios are more prone to uneven category distribution, i.e., a long-tail distribution problem. This prevents the model from accurately learning information from infrequently occurring samples, thus hindering its ability to accurately identify Chinese entity boundaries. Summary of the Invention

[0004] The technical problem actually solved by this invention is to provide a Chinese entity recognition method, apparatus and equipment that can effectively expand training data in a low-resource environment and has the ability to perform label semantic analysis in order to combine label semantic information for high-precision Chinese entity recognition.

[0005] To address the above problems, this invention provides a Chinese entity recognition method, comprising:

[0006] The original sample containing entity labels is processed based on the masked entity generation language model to generate an enhanced sample with a second entity, wherein the original sample contains a first entity and the second entity is different from the first entity;

[0007] Obtain extended domain definitions for the entity labels;

[0008] The original sample, the enhanced sample, and the domain-extended paraphrase are input into the recognition model for training. The input data is then subjected to label semantic fusion based on the label semantic fusion module in the recognition model, which guides the recognition model to learn and generate sample representations containing the semantic information of the entity labels.

[0009] The training loss of the recognition model is adjusted based on the sample representation and the preset adjustment factor, so that the recognition model has an adaptive optimization function for the loss weights of easily distinguishable entity types and difficult-to-distinguish entity types.

[0010] Based on the trained recognition model, Chinese entity recognition is performed on the test text.

[0011] As an optional embodiment, it also includes:

[0012] Linearly embed entity labels corresponding to the first entity into the original sample, and mask the first entity in the original sample;

[0013] The masked entity generation language model processes the original sample containing entity labels to generate enhanced samples with second entities, including:

[0014] The original sample containing the entity label and masking the first entity is input into the masked entity generation language model, so that the masked entity generation language model generates a second entity based on the input original sample, and embeds the second entity into the masked first entity in the original sample, thereby forming the enhanced sample.

[0015] As an optional embodiment, the step of enabling the masked entity generation language model to generate an enhanced sample containing the second entity based on the input original sample includes:

[0016] The masked entity generation language model analyzes the semantic information of the unmasked text data and linearly embedded entity labels in the original sample, and predicts and generates a second entity of the same category as the first entity based on the analysis results.

[0017] The masked entity generation language model generates the enhanced sample based on the second entity and the original input sample.

[0018] As an optional embodiment, the label semantic fusion module has a twin encoder with shared parameters, which processes the sample data and the extended label paraphrasing separately to extract the hidden layer features of the sample data and the extended label paraphrasing.

[0019] As an optional embodiment, the step of performing label semantic fusion on the input data based on the label semantic fusion module in the recognition model, guiding the recognition model to learn and generate sample representations containing semantic information of the entity labels, includes:

[0020] The input data is processed by the twin encoder in the label semantic fusion module to obtain hidden layer features corresponding to the sample data and the extended interpretation of the labels, respectively.

[0021] The hidden layer features of the sample data and the extended interpretation of the labels are processed using a cross-attention mechanism to generate a sample representation containing the semantic information of the entity labels.

[0022] As an optional embodiment, the hidden layer features of the sample data and the extended interpretation of the labels are processed using a cross-attention mechanism to generate a sample representation containing the semantic information of the entity labels, including:

[0023] Hidden layer features for extended interpretation using the aforementioned tags The query state serves as the cross-attention mechanism, and is represented by the hidden layer features H of the sample data. X As key and value states in the cross-attention mechanism, the weighted sample representation C' is generated by combining the following formula. j :

[0024]

[0025]

[0026] Where d is the model dimension, W Q W K W V All are learnable parameters;

[0027] The weighted sample representations are used as the key and value states of the cross-attention mechanism, H XTo query the status, the attention mechanism is executed again to obtain a sample representation containing the semantic information of the entity label.

[0028] As an optional embodiment, adjusting the training loss of the recognition model based on the sample representation and a preset adjustment factor includes:

[0029] Based on the sample representation and the preset adjustment factor, the training loss of the recognition model is adjusted by combining an uncertainty weighting strategy.

[0030] As an optional embodiment, adjusting the training loss of the recognition model based on the sample representation, a preset adjustment factor, and an uncertainty weighting strategy includes:

[0031] Based on the sample representation and adjustment factor, the training loss of the recognition model is adjusted using the following formula:

[0032]

[0033]

[0034] in, For each entity category, the model predicts the probability, C is the sample representation, W and b are both learnable parameters, L is the loss function of the recognition model, γ is a regulation factor with a value range of [0,5], t is the total number of entity categories, and α... j The hyperparameter represents the loss weights for different entity label categories. I(·) is an indicator function; when j = y, y is the entity label category, the value of I(j == y) is 1, otherwise it is 0. If the value approaches 1, then entity category j is an easily distinguishable type, and the corresponding loss weight decreases. If the value approaches 1, then the entity category j is a difficult-to-distinguish type, and the corresponding loss weight increases.

[0035] Another embodiment of the present invention also provides a Chinese entity recognition device, comprising:

[0036] The processing module is used to process the original sample containing entity labels through the masked entity generation language model to generate an enhanced sample with a second entity, wherein the original sample contains a first entity and the second entity is different from the first entity;

[0037] The module is used to obtain extended domain definitions for the entity labels;

[0038] The input module is used to input the original sample, the enhanced sample, and the domain extended interpretation into the recognition model for training, and to perform label semantic fusion on the input data based on the label semantic fusion module in the recognition model, so as to guide the recognition model to learn and generate sample representations containing the semantic information of the entity labels;

[0039] The adjustment module is used to adjust the training loss of the recognition model according to the sample representation and the preset adjustment factor, so that the recognition model has an adaptive optimization function for the loss weights of easily distinguishable entity types and difficult-to-distinguish entity types.

[0040] The recognition module is used to perform Chinese entity recognition on the test text using the trained recognition model.

[0041] Another embodiment of the present invention also provides an electronic device, comprising:

[0042] At least one processor; and,

[0043] A memory communicatively connected to the at least one processor; wherein,

[0044] The memory stores instructions that can be executed by the at least one processor to implement the Chinese entity recognition method as described in any of the embodiments above.

[0045] Based on the disclosure of the above embodiments, it can be understood that the beneficial effects of the embodiments of the present invention include: processing the original samples to generate enhanced samples through a masked entity language generation model can not only effectively expand the training data, but also ensure the alignment of text entities with the original labels. Secondly, by setting a label semantic fusion module, the label semantic information of entity labels in the original samples is fully recognized and learned, thereby improving the accuracy of Chinese entity recognition. In addition, for the long-tail distribution problem in low-resource scenarios, the method of this embodiment optimizes the training loss function, enabling the recognition model to adaptively assign higher loss weights to difficult-to-distinguish entity label categories in the sample data, further improving the recognition accuracy of the recognition model.

[0046] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0047] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0048] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0049] Figure 1 This is a flowchart of the Chinese entity recognition method in an embodiment of the present invention.

[0050] Figure 2 This is a diagram illustrating the application process of the Chinese entity recognition method in this embodiment of the invention.

[0051] Figure 3 This is a diagram illustrating the process of generating enhanced samples based on original samples in an embodiment of the present invention.

[0052] Figure 4 This is a structural block diagram of the Chinese entity recognition device in an embodiment of the present invention. Detailed Implementation

[0053] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but these are not intended to limit the scope of the invention.

[0054] It should be understood that various modifications can be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this disclosure will be apparent to those skilled in the art.

[0055] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present disclosure and, together with the general description of the disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the disclosure.

[0056] These and other features of the invention will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0057] It should also be understood that although the invention has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of the invention, which have the features described in the claims and are therefore all within the scope of protection defined herein.

[0058] The above and other aspects, features and advantages of this disclosure will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0059] Specific embodiments of the present disclosure are described thereafter with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure and can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the present disclosure. Therefore, the specific structural and functional details disclosed herein are not intended to be limiting, but merely to serve as the basis and representative basis for the claims to teach those skilled in the art to use the present disclosure in a variety of substantially any suitable detailed structures.

[0060] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in still another embodiment,” all of which may refer to one or more of the same or different embodiments according to this disclosure.

[0061] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0062] like Figure 1 As shown, this embodiment of the invention provides a Chinese entity recognition method, including:

[0063] S100: Based on the masked entity generation language model, the original sample containing entity labels is processed to generate an enhanced sample with a second entity. The original sample contains a first entity, and the second entity is different from the first entity.

[0064] S101: Obtain extended domain definitions for entity tags;

[0065] S102: Input the original samples, enhanced samples and domain-extended paraphrases into the recognition model for training, and perform label semantic fusion on the input data based on the label semantic fusion module in the recognition model to guide the recognition model to learn and generate sample representations containing semantic information of entity labels.

[0066] S103: The training loss of the recognition model is adjusted based on the sample representation and the preset adjustment factor, so that the recognition model has an adaptive tuning function for the loss weights of easily distinguishable entity types and difficult-to-distinguish entity types.

[0067] S104: Perform Chinese entity recognition on the test text based on the trained recognition model.

[0068] As can be seen from the above embodiments, the beneficial effects of the method in this embodiment include processing the original samples to generate enhanced samples through a masked entity language generation model, which not only effectively expands the training data but also ensures the alignment of text entities with the original labels. Secondly, by setting a label semantic fusion module, the label semantic information of entity labels in the original samples is fully recognized and learned, improving the accuracy of Chinese entity recognition. In addition, for the long-tail distribution problem in low-resource scenarios, the method in this embodiment optimizes the training loss function, enabling the recognition model to adaptively assign higher loss weights to difficult-to-distinguish entity label categories in the sample data, further improving the recognition accuracy of the recognition model. In practical applications, users can input the test data, such as Chinese sentences containing multiple different entities (characters, words), into the recognition model. The recognition model can then output a label sequence corresponding to the Chinese sentence, which contains the label of each entity. Even if the category to which the label of a certain entity in the sentence belongs is difficult to recognize due to very limited annotation sample resources, the recognition model in this embodiment can still be fully trained based on these limited resources, enabling it to accurately identify that type of entity and predict its label.

[0069] like Figure 2 As shown, the training process of the recognition model in this embodiment can be divided into two stages. First, the first stage is to augment the training data based on the masked entity generation language model, thereby expanding the training data. In the second stage, this embodiment introduces domain-extended paraphrasing of entity labels as additional domain prior knowledge, which, along with the augmented samples and the original samples, serves as input to the recognition model. In this stage, the label semantic fusion module performs label semantic fusion, guiding the model to learn sample representations containing label semantic information, thus enabling accurate named entity recognition of the test text. Furthermore, addressing the long-tail distribution problem of data in low-resource scenarios, this embodiment optimizes the model's training loss function, enabling the model to adaptively assign higher loss weights to samples of difficult-to-distinguish categories. This not only solves the long-tail distribution problem but also further improves the model's recognition accuracy.

[0070] Furthermore, data augmentation is one of the most direct and effective methods to solve the low-resource problem. However, since named entity recognition (NID) focuses on character-level semantics, traditional data augmentation methods often result in augmented samples that do not match the original labels when applied to this task. Therefore, this embodiment proposes to linearize and embed entity labels into the original samples. This not only constrains the newly generated samples during data augmentation but also enhances the boundaries of Chinese entities. The process includes:

[0071] S105: Linearly embed the entity label corresponding to the first entity in the original sample, and mask the first entity in the original sample;

[0072] A masked entity generation language model processes raw samples containing entity labels to generate augmented samples with second entities, including:

[0073] S106: Input the original sample containing entity labels and masking the first entity into the masked entity generation language model, so that the masked entity generation language model generates the second entity based on the input original sample, and embeds the second entity into the masked first entity in the original sample, thereby forming an enhanced sample.

[0074] The process of enabling the masked entity generation language model to generate enhanced samples containing a second entity based on the original input sample includes:

[0075] S107: The masked entity generation language model analyzes the semantic information of the unmasked text data and linearly embedded entity labels in the original samples, and predicts and generates a second entity of the same category as the first entity based on the analysis results.

[0076] S108: The masked entity generation language model generates enhanced samples based on the processing of the second entity and the original input samples.

[0077] For example, the above process can be defined as:

[0078]

[0079] In the formula, x2 is an entity, and its corresponding tag type is s2.

[0080] Combination Figure 3 As shown, the character sequence and corresponding label sequence of the original sample are first linearized. Then, the labels of entity types with semantic meaning (i.e., entity types other than type O in the figure) are embedded into the character sequence. For example, the corresponding entity labels are inserted before and after the two characters of the first entity "Paris" in the original sample. Next, the linearized sample X' is input into the masked entity generation language model, which is a sequence-to-sequence generation model. When the model receives the input data, it first uses the masking identifier [Mask] to randomly replace the characters in the entities other than type O. Then, the model maximizes the probability between the masked entity characters and the linearized sample sequence, so that the model can use the context of the unmasked original sample and the semantic information of the linearized embedded labels to predict the masked characters, thereby generating a new entity, i.e., the second entity, such as "Beijing" in the figure. This process can be described by the following formula:

[0081]

[0082] In the formula, θ represents the parameters of the masked entity generation language model. X represents the original sample, xi It is a character in X. k i For integers between 0 and 1, if k i =1 indicates that x i Replaced by the masking identifier [Mask]; if k i =1 indicates that the character remains unchanged. n represents the number of characters in X'.

[0083] Furthermore, since the masked entity language generation model is pre-trained on the original training corpus, the predicted masked entity characters are very likely to be entity characters in the original samples, meaning the first entity and the second entity are the same. To enable the model to generate augmented samples that are different from the original samples and are coherent and rich, this embodiment increases the threshold for candidate entity probabilities, expands the candidate entity character set, and uses a random strategy to select candidate entity characters from this set. Simultaneously, a Gaussian distribution is applied to sample the entity character masking rate to obtain a dynamically changing entity masking rate, thereby increasing the diversity of augmented samples and preventing predicted entities from being the same as the original entities.

[0084] Continue to combine Figure 3 As shown, after the prediction of the masked entity is completed, the generated new sample is also linearized. This method can ensure that the entity labels in the original sample and the augmented sample are aligned. In order to maintain consistency with the original training sample, it needs to be converted back to the format of the original sample. That is, the final generated augmented sample is in the same format as the original sample. After that, all sample data will be used as input for the subsequent label semantic fusion stage.

[0085] Currently, in named entity recognition tasks, i.e., Chinese entity recognition tasks, the labels are usually high-level summaries of entity categories. However, when users understand entities, they usually pre-determine the domain in which the entity belongs and assign more domain-related concrete meanings to the abstract entity labels. For example, the entity "fever" in "everyone shines in their respective positions" is generally not labeled as an entity in some domains such as news, but it is a very important symptom type entity in the medical field.

[0086] Furthermore, the same entity may have different specific meanings in different domains. For example, "apple" is labeled as a fruit in the food domain, but as a company or brand in the financial domain. To enable the model to simulate human thought processes as closely as possible, this embodiment introduces domain-extended interpretations of entity labels as additional prior knowledge. In practical applications, domain-extended interpretations of labels are easily obtained and changed, which can improve the model's domain generalization. Specific label extension interpretations can be found in the table below:

[0087]

[0088]

[0089] Furthermore, the label semantic fusion module in this embodiment is equipped with a Siamese encoder. This is because the corpus size of labels and their domain extended paraphrases is very small compared to the original training set, insufficient to train a good encoder. Therefore, this embodiment proposes to use a Siamese encoder in this module, as its Siamese form can effectively handle small-scale data and class imbalance. In this embodiment, the Siamese encoder includes a label extended paraphrase encoder and an encoder for the original training samples. The two encoders have similar structures and share model parameters, used to process the sample data and label extended paraphrases respectively to extract the hidden layer features of the sample data and label extended paraphrases.

[0090] For example, given the input sample sequence (i.e., the aforementioned sample data) and the label extended paraphrasing sequence, the hidden layer features are extracted using a Siamese encoder, as shown in the formula:

[0091]

[0092]

[0093] Wherein, PLM is the neural network model used by the encoder, which can be changed according to the task and dataset characteristics. In this embodiment, the RoBERTa model is used. In the formula, H X The hidden layer features of the sample sequence are represented by , n represents the length of the sample sequence X, and d is the vector dimension of the encoder. This represents the j-th label category s. j Hidden layer features of extended label paraphrasing; s j ∈S, where S is the set of entity label types, |S| is the number of elements in the entity label type set, i.e., the number of label types, m represents the length of the extended definition of the label, n is the text length of the input sample data, i.e., the number of characters, x n This represents the nth character. This represents the hidden layer feature of the nth character. This represents the j-th extended definition of this tag type. This represents the hidden layer feature of the j-th extended interpretation.

[0094] Furthermore, in this embodiment, when performing label semantic fusion on the input data based on the label semantic fusion module in the recognition model, and guiding the recognition model to learn and generate sample representations containing semantic information of entity labels, the process includes:

[0095] S109: Based on the twin encoder in the label semantic fusion module, the input data is processed to obtain the hidden layer features corresponding to the sample data and the extended interpretation of the labels respectively;

[0096] S110: The hidden layer features of the sample data and the extended interpretation of the labels are processed using a cross-attention mechanism to generate a sample representation containing semantic information of entity labels.

[0097] Specifically, the hidden layer features of the sample data and the extended interpretation of labels are processed using a cross-attention mechanism to generate sample representations containing semantic information of entity labels, including:

[0098] S111: Hidden layer features for expanding paraphrasing using labels The query state is used as the cross-attention mechanism, and the hidden layer features H of the sample data are used as the basis. X As key and value states in the cross-attention mechanism, the weighted sample representation C' is generated by combining the following formula. j :

[0099]

[0100]

[0101] Where d is the model dimension, W Q W K W V All are learnable parameters;

[0102] S112: The weighted sample representations are used as the key and value states of the cross-attention mechanism, H X To query the status, the attention mechanism is executed again to obtain a sample representation containing semantic information of entity labels.

[0103] For example, after obtaining the hidden layer features of the sample sequence and the label interpretation sequence, most existing methods use direct multiplication or concatenation. However, this approach does not fully learn the semantic information of the labels and also increases the length of the input sequence. To address this issue, this embodiment proposes a novel fusion strategy that utilizes a cross-attention mechanism to process the input sequence, allowing the model to focus more on the core parts of the sample data while ignoring redundant parts. It also integrates label semantic knowledge into the generated sample representation. Simultaneously, this fusion strategy maintains the fixed length and structure of the samples.

[0104] Specifically, in this embodiment, the hidden layer features of expanded meaning using tags are first used. The query state is used as the cross-attention mechanism, and the hidden layer features H of the sample character sequence are used. X The key and value states serve as the basis for the cross-attention mechanism. As shown in the formula:

[0105]

[0106]

[0107] Where d is the model dimension, W Q W K W V All are learnable parameters, Q, K, and V are the query value and key value in the attention mechanism, respectively, and T is the matrix transpose symbol with dimension 1. After obtaining the weighted sample representation C' j Then, it is used as the key and value state of the cross-attention mechanism, H X To query the state, the attention mechanism is executed again to obtain the sample representation C of the entity label type in the corresponding sample data. j The specific process is shown in the following formula:

[0108] C j =Attention(H X W Q ,C' j W K ,C' j W V ).

[0109] Furthermore, the problem of uneven sample class distribution is particularly prominent in low-resource scenarios, such as... Figure 3 In the sample data, entities of type "O" constitute the majority. Therefore, compared to other entity types with richer semantics, predicting the label of "type O" entity characters is relatively simple for the recognition model. However, previous recognition models did not differentiate between the difficulty of distinguishing entity categories, often causing the model to focus on the easily distinguishable "type O" throughout the training process, failing to fully learn other entity types in the sample. To solve this problem, this embodiment specifically introduces an adjustment factor γ, making the model more focused on the difficult-to-distinguish entity types, achieving an adaptive weight tuning effect.

[0110] Specifically, in this embodiment, when adjusting the training loss of the recognition model based on sample representation and a preset adjustment factor, the following is included:

[0111] S112: Based on sample representation and preset adjustment factors, the training loss of the recognition model is adjusted by combining uncertainty weighting strategy.

[0112] For example, based on the formula: First, add a linear layer to the sample representation C, and then calculate the model prediction probability for each entity category. In the formula, W and b are both learnable parameters. Then, based on the predicted probability... The loss function L of the calculation model is:

[0113]

[0114] Where γ is an adjustment factor, ranging from [0,5], with a preferred value of 2. When γ is zero, the above formula is the common cross-entropy loss function. t is the total number of entity categories, α... j is a hyperparameter representing the loss weights for different entity label categories. I(·) is an indicator function; when j = y, the value of I(j == y) is 1, otherwise it is 0. j represents the j-th entity category, and y represents the entity label category. In the formula... The probability of not belonging to the j-th entity category is used to reduce the loss contribution of easily distinguishable entity categories. Specifically, if If the value approaches 1, it indicates that the corresponding entity's category j is an easily distinguishable type. The value will approach 0, meaning the entity category contributes less to the loss, and the corresponding loss weight decreases. If the value approaches 1, it indicates that the entity category j is a difficult type to distinguish, so the corresponding loss weight increases. In other words, the model should pay more attention to learning the entity category and entity label.

[0115] like Figure 4 As shown, another embodiment of the present invention also provides a Chinese entity recognition device 100, comprising:

[0116] The processing module is used to process the original sample containing entity labels through the masked entity generation language model to generate an enhanced sample with a second entity, wherein the original sample contains a first entity and the second entity is different from the first entity;

[0117] The module is used to obtain extended domain definitions for the entity labels;

[0118] The input module is used to input the original sample, the enhanced sample, and the domain extended interpretation into the recognition model for training, and to perform label semantic fusion on the input data based on the label semantic fusion module in the recognition model, so as to guide the recognition model to learn and generate sample representations containing the semantic information of the entity labels;

[0119] The adjustment module is used to adjust the training loss of the recognition model according to the sample representation and the preset adjustment factor, so that the recognition model has an adaptive optimization function for the loss weights of easily distinguishable entity types and difficult-to-distinguish entity types.

[0120] The recognition module is used to perform Chinese entity recognition on the test text using the trained recognition model.

[0121] As an optional embodiment, it also includes:

[0122] An embedding module is used to linearly embed entity labels corresponding to a first entity into the original sample and to mask the first entity in the original sample.

[0123] The masked entity generation language model processes the original sample containing entity labels to generate enhanced samples with second entities, including:

[0124] The original sample containing the entity label and masking the first entity is input into the masked entity generation language model, so that the masked entity generation language model generates an enhanced sample containing the second entity based on the input original sample.

[0125] As an optional embodiment, the step of enabling the masked entity generation language model to generate an enhanced sample containing the second entity based on the input original sample includes:

[0126] The masked entity generation language model analyzes the semantic information of the unmasked text data and linearly embedded entity labels in the original sample, and predicts and generates a second entity of the same category as the first entity based on the analysis results.

[0127] The masked entity generation language model generates the enhanced sample based on the second entity and the original input sample.

[0128] As an optional embodiment, the label semantic fusion module has a twin encoder with shared parameters, which processes the sample data and the extended label paraphrasing separately to extract the hidden layer features of the sample data and the extended label paraphrasing.

[0129] As an optional embodiment, the step of performing label semantic fusion on the input data based on the label semantic fusion module in the recognition model, guiding the recognition model to learn and generate sample representations containing semantic information of the entity labels, includes:

[0130] The input data is processed by the twin encoder in the label semantic fusion module to obtain hidden layer features corresponding to the sample data and the extended interpretation of the labels, respectively.

[0131] The hidden layer features of the sample data and the extended interpretation of the labels are processed using a cross-attention mechanism to generate a sample representation containing the semantic information of the entity labels.

[0132] As an optional embodiment, the hidden layer features of the sample data and the extended interpretation of the labels are processed using a cross-attention mechanism to generate a sample representation containing the semantic information of the entity labels, including:

[0133] Hidden layer features for extended interpretation using the aforementioned tags The query state serves as the cross-attention mechanism, and is represented by the hidden layer features H of the sample data. X As key and value states in the cross-attention mechanism, the weighted sample representation C' is generated by combining the following formula. j :

[0134]

[0135]

[0136] Where d is the model dimension, W Q W K W V All are learnable parameters;

[0137] The weighted sample representations are used as the key and value states of the cross-attention mechanism, H X To query the status, the attention mechanism is executed again to obtain a sample representation containing the semantic information of the entity label.

[0138] As an optional embodiment, adjusting the training loss of the recognition model based on the sample representation and a preset adjustment factor includes:

[0139] Based on the sample representation and the preset adjustment factor, the training loss of the recognition model is adjusted by combining an uncertainty weighting strategy.

[0140] As an optional embodiment, adjusting the training loss of the recognition model based on the sample representation, a preset adjustment factor, and an uncertainty weighting strategy includes:

[0141] Based on the sample representation and adjustment factor, the training loss of the recognition model is adjusted using the following formula:

[0142]

[0143]

[0144] in, Let C be the model prediction probability for each entity label category, W and b be learnable parameters, L be the loss function of the recognition model, γ be a regulation factor with a value range of [0,5], t be the total number of entity categories, and α be the probability of the entity label category. j The hyperparameter represents the loss weights for different entity label categories. I(·) is an indicator function; when j = y, y is the entity label category, the value of I(j == y) is 1, otherwise it is 0. If the value approaches 1, then the entity label j is of an easily distinguishable type, and the corresponding loss weight decreases. If the value approaches 1, then the entity label j is of a difficult-to-distinguish type, and the corresponding loss weight increases.

[0145] Another embodiment of the present invention also provides an electronic device, comprising:

[0146] At least one processor; and,

[0147] A memory communicatively connected to the at least one processor; wherein,

[0148] The memory stores instructions that can be executed by the at least one processor to implement the Chinese entity recognition method as described in any of the embodiments above.

[0149] Furthermore, one embodiment of the present invention also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the Chinese entity recognition method as described above. It should be understood that the various solutions in this embodiment have the corresponding technical effects in the above method embodiments, and will not be repeated here.

[0150] Furthermore, embodiments of the present invention also provide a computer program product, which is tangibly stored on a computer-readable medium and includes computer-readable instructions that, when executed, cause at least one processor to perform a Chinese entity recognition method such as the one described in the embodiments above.

[0151] It should be noted that the computer storage medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program configured for use by or in connection with an instruction execution system, system, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, antenna, optical fiber, RF, etc., or any suitable combination thereof.

[0152] Furthermore, those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0153] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0154] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0155] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0156] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0157] The above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the present invention. The scope of protection of the present invention is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to the present invention within its spirit and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of the present invention.

Claims

1. A Chinese entity recognition method, characterized in that, include: The original sample containing entity labels is processed based on the masked entity generation language model to generate an enhanced sample with a second entity, wherein the original sample contains a first entity and the second entity is different from the first entity; Obtain extended domain definitions for the entity labels; The original sample, the enhanced sample, and the domain-extended paraphrase are input into the recognition model for training. The input data is then subjected to label semantic fusion based on the label semantic fusion module in the recognition model, which guides the recognition model to learn and generate sample representations containing the semantic information of the entity labels. The training loss of the recognition model is adjusted based on the sample representation and the preset adjustment factor, so that the recognition model has an adaptive optimization function for the loss weights of easily distinguishable entity types and difficult-to-distinguish entity types. Based on the trained recognition model, Chinese entity recognition is performed on the test text.

2. The Chinese entity recognition method according to claim 1, characterized in that, Also includes: Linearly embed entity labels corresponding to the first entity into the original sample, and mask the first entity in the original sample; The masked entity generation language model processes the original sample containing entity labels to generate enhanced samples with second entities, including: The original sample containing the entity label and masking the first entity is input into the masked entity generation language model, so that the masked entity generation language model generates a second entity based on the input original sample, and embeds the second entity into the masked first entity in the original sample, thereby forming the enhanced sample.

3. The Chinese entity recognition method according to claim 2, characterized in that, The step of enabling the masked entity generation language model to generate an enhanced sample containing the second entity based on the input original sample includes: The masked entity generation language model analyzes the semantic information of the unmasked text data and linearly embedded entity labels in the original sample, and predicts and generates a second entity of the same category as the first entity based on the analysis results. The masked entity generation language model generates the enhanced sample based on the second entity and the original input sample.

4. The Chinese entity recognition method according to claim 1, characterized in that, The label semantic fusion module has a twin encoder with shared parameters, which is used to process the sample data and the extended label paraphrasing respectively to extract the hidden layer features of the sample data and the extended label paraphrasing.

5. The Chinese entity recognition method according to claim 4, characterized in that, The label semantic fusion module in the recognition model performs label semantic fusion on the input data, guiding the recognition model to learn and generate sample representations containing semantic information of the entity labels, including: The input data is processed by the twin encoder in the label semantic fusion module to obtain hidden layer features corresponding to the sample data and the extended interpretation of the labels, respectively. The hidden layer features of the sample data and the extended interpretation of the labels are processed using a cross-attention mechanism to generate a sample representation containing the semantic information of the entity labels.

6. The Chinese entity recognition method according to claim 5, characterized in that, The hidden layer features of the sample data and the extended interpretation of the labels are processed using a cross-attention mechanism to generate a sample representation containing the semantic information of the entity labels, including: Hidden layer features for extended interpretation using the aforementioned tags The query state serves as the cross-attention mechanism, and is based on the hidden layer features of the sample data. As key and value states in the cross-attention mechanism, the following formula is used to generate a weighted sample representation. : ， ， in, For model dimensions, , , All are learnable parameters. These are the query value and key value in the attention mechanism, respectively. The weighted sample representations are used as the key and value states of the cross-attention mechanism. To query the status, the attention mechanism is executed again to obtain a sample representation containing the semantic information of the entity label.

7. The Chinese entity recognition method according to claim 1, characterized in that, The adjustment of the training loss of the recognition model based on the sample representation and a preset adjustment factor includes: Based on the sample representation and the preset adjustment factor, the training loss of the recognition model is adjusted by combining an uncertainty weighting strategy.

8. The Chinese entity recognition method according to claim 7, characterized in that, The adjustment of the training loss of the recognition model based on the sample representation, a preset adjustment factor, and an uncertainty weighting strategy includes: Based on the sample representation and adjustment factor, the training loss of the recognition model is adjusted using the following formula: ， ， in, The model predicts the probability for each entity category. For sample characterization, and All are learnable parameters. Let be the loss function of the recognition model. This is an adjustment factor, with a value range of [0,5]. The total number of entity categories. is a hyperparameter representing the loss weights for different entity label categories. For indicator functions, when hour, For entity label categories, The value is 1 if it is 1 otherwise it is 0. If it approaches 1, then the entity category For easily identifiable types, the corresponding loss weights are reduced. If it approaches 1, then the entity category For types that are difficult to distinguish, the corresponding loss weight is increased.

9. A Chinese entity recognition device, characterized in that, include: The processing module is used to process the original sample containing entity labels through the masked entity generation language model to generate an enhanced sample with a second entity, wherein the original sample contains a first entity and the second entity is different from the first entity; The module is used to obtain extended domain definitions for the entity labels; The input module is used to input the original sample, the enhanced sample, and the domain extension interpretation into the recognition model for training, and to perform label semantic fusion on the input data based on the label semantic fusion module in the recognition model, so as to guide the recognition model to learn and generate sample representations containing the semantic information of the entity labels; The adjustment module is used to adjust the training loss of the recognition model according to the sample representation and the preset adjustment factor, so that the recognition model has an adaptive optimization function for the loss weights of easily distinguishable entity types and difficult-to-distinguish entity types. The recognition module is used to perform Chinese entity recognition on the test text using the trained recognition model.

10. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to implement the Chinese entity recognition method as described in any one of claims 1-8.

Images