Chinese and English bilingual Chinese herbal medicine and target entity recognition method based on double semantic alignment

Abstract

The invention provides a Chinese and English bilingual Chinese herbal medicine and target entity recognition method based on double semantic alignment. The method comprises the following steps: setting an entity contrast corpus of Chinese and English languages; calculating a loss function required by multi-language alignment by using an encoder mBERT model; calculating a loss function required by professional semantic alignment; calculating a regularization item; calculating the sum of loss functions and training an mBERT model; under the condition that part of parameters of the mBERT model trained in the fifth step are kept unchanged, the encoder mBERT model and the decoder CRF are trained at the same time. According to the cross-language text entity recognition method, a traditional entity recognition method is improved through a double semantic alignment technology, multi-language alignment, professional semantic alignment and a partial fine adjustment strategy are included, the accuracy and robustness of cross-language text entity recognition are improved, and through verification on a reference data set, it is proved that compared with the prior art, the performance of the cross-language text entity recognition method is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing and bioinformatics, and in particular relates to a Chinese-English bilingual Chinese herbal medicine and target entity recognition method based on dual semantic alignment. Background Art

[0002] Entity recognition is one of the key technologies in the field of natural language processing (NLP). It involves accurately identifying specific entity information in text, such as names of people, places, and organizations. It is widely used in information extraction, intelligent question answering, machine translation, and other fields. In the field of biomedicine, especially in the context of the modernization of traditional Chinese medicine, automatic identification of Chinese herbal medicines and targets from literature is of great significance for extracting key information and promoting knowledge discovery. With the continuous deepening of globalization, the demand for the identification of Chinese herbal medicines and targets in a multilingual environment has increased significantly.

[0003] Currently, most named entity recognition studies focus on pharmaceutical literature or general domain texts in a single language. However, these studies are significantly different from mixed Chinese and English pharmaceutical literature, which incorporates a large amount of Chinese and English professional domain knowledge. Existing entity recognition methods often have difficulty in fully expressing the deep knowledge in the Chinese and English medical fields when processing such literature. For example, the herb "Coptis chinensis" has many nicknames, including Yunlian, Yalian, Chuanlian, Weilian, Jizhaolian, etc. The professional term "serum cystatin C" has multiple expressions in different contexts, such as cystatin C, cystatin C, and the English name Cystatin C, abbreviated as CysC or CYS-C. The diversity and complexity of these expressions reveal the challenges of entity recognition in mixed Chinese and English literature, and also highlight the limitations of existing technologies in professional domain entity recognition tasks. Summary of the invention

[0004] In view of this, the present invention aims to overcome the shortcomings of the above-mentioned problems in the prior art and proposes a Chinese-English bilingual Chinese herbal medicine and target entity recognition method based on dual semantic alignment. The present invention improves the accuracy and robustness of entity recognition in cross-language texts through dual semantic alignment technology.

[0005] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0006] The first aspect of the present invention provides a method for identifying Chinese and English bilingual Chinese herbal medicines and target entities based on dual semantic alignment, comprising the following steps:

[0007] Step 1: Set up entity comparison corpora in Chinese and English;

[0008] Step 2: Use the encoder mBERT model to calculate the loss function required for multilingual alignment;

[0009] Step 3: Calculate the loss function required for professional semantic alignment;

[0010] Step 4: Calculate the regularization term;

[0011] Step 5: Calculate the sum of the loss functions and train the mBERT model;

[0012] Step 6: While keeping some parameters of the mBERT model trained in step 5 unchanged, train the encoder mBERT model and decoder CRF at the same time.

[0013] Furthermore, it also includes evaluating the recognition accuracy using precision, recall, and F1 indicators.

[0014] Furthermore, in step 1, a Chinese-English dictionary is used to select a Chinese sentence set (defined as D cn ) in a sentence of Chinese herbal medicine and target Chinese entity e cn , from the English sentence set (defined as D en ) to find the corresponding English entity e en , record the corresponding relationship, and form an entity comparison corpus in Chinese and English, expressed as

[0015] i is any integer, where is a pair of Chinese and English entities.

[0016] Furthermore, in step 2, the entities in the sentence are defined or The vector obtained by the encoder mBERT model is represented as or The calculation formula of the loss function required for multilingual alignment is as follows:

[0017]

[0018] Furthermore, in step 3, a professional synonym dictionary is used to obtain the word cn ×D en Entity e i and its equivalent semantics a i The set of aliases of is defined as {D a |(e1, a1), (e2, a2)...(e i , a i ), i is any integer}, and For their corresponding embedding vectors, the loss function L required for professional semantic alignment align The calculation formula is as follows:

[0019]

[0020] Furthermore, in step 4, the regularization term is:

[0021]

[0022] in, is generated by the initial parameters of the encoder mBERT model before training i The entity corresponds to the vector, and the regularization uses the mean square error loss, expressed as ||·||. The mean square error calculates the difference in the vector modulus.

[0023] Furthermore, in step 5, the parameters of the mBERT model are trained by minimizing the following alignment loss function to optimize the word vector alignment of dual semantics:

[0024] L=min∑L crosslingual +L align +L regularize (4).

[0025] The second aspect of the present invention provides a Chinese-English bilingual Chinese herbal medicine and target entity recognition device based on dual semantic alignment, comprising:

[0026] Data construction module, used to build entity comparison corpora in Chinese and English;

[0027] The first calculation module is used to calculate the loss function required for multilingual alignment using the encoder mBERT model;

[0028] The second calculation module is used to calculate the loss function required for professional semantic alignment;

[0029] The third calculation module is used to calculate the regularization term;

[0030] The fourth calculation module is used to calculate the sum of the loss function and train the mBERT model;

[0031] The fifth computing module is used to simultaneously train the encoder mBERT model and the decoder CRF while keeping some parameters of the mBERT model trained in step 5 unchanged.

[0032] A third aspect of the present invention provides an electronic device, comprising a processor and a memory connected to the processor for storing executable instructions for the processor, wherein the processor is used to execute the above-mentioned Chinese-English bilingual Chinese herbal medicine and target entity recognition method based on dual semantic alignment.

[0033] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned Chinese and English bilingual Chinese herbal medicine and target entity recognition method based on dual semantic alignment.

[0034] Compared with the prior art, the Chinese-English bilingual Chinese herbal medicine and target entity recognition method based on dual semantic alignment described in the present invention has the following advantages:

[0035] The present invention proposes an innovative multilingual entity recognition method based on dual semantic alignment, which is particularly suitable for the recognition of Chinese herbal medicines and targets in Chinese and English literature. The method of the present invention improves the traditional entity recognition method through dual semantic alignment technology, including multilingual alignment and professional semantic alignment, as well as some fine-tuning strategies, to improve the accuracy and robustness of entity recognition in cross-language texts. Through verification on benchmark datasets, it is proved that its performance is significantly improved compared with the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0037] Figure 1 A schematic flow chart of a method for identifying Chinese and English bilingual Chinese herbal medicine and target entities based on dual semantic alignment according to the present invention; Figure 2 This is a comparison chart of the results of the method of the present invention and other classical methods. DETAILED DESCRIPTION

[0038] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0039] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0040] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0041] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0042] Embodiment 1:

[0043] The present invention proposes an innovative named entity recognition method for the problem of multilingual entity recognition of Chinese herbal medicines and targets in Chinese and English literature. The method of the present invention is based on the multilingual BERT (mBERT) model as the encoder and uses conditional random fields (CRF) as the decoder output.

[0044] The method of the present invention is divided into two key stages, which are in a progressive relationship:

[0045] Model training phase of dual semantic alignment: This phase introduces the concepts of multilingual alignment and professional semantic alignment, collectively known as dual semantic alignment. Multilingual alignment aims to enable the model to understand the correspondence between Chinese herbal medicines and targets in Chinese and English, while professional semantic alignment helps the model identify situations where different professional vocabulary express the same concept. During the training process, only the encoder, i.e., the mBERT model, is trained to make the vectors of vocabulary with the same semantics closer to achieve the purpose of alignment. This not only improves the model's ability to understand professional terms, but also strengthens its generalization in multilingual texts.

[0046] Model training phase based on partial fine-tuning: Based on the first phase, the decoder, namely the conditional random field (CRF), is further trained. In this phase, the encoder mBERT model and the decoder CRF are jointly trained to improve the accuracy of entity recognition. This process retains most of the model parameters of the mBERT model trained in the first phase, and only fine-tunes some parameters to achieve fine-tuning of the model, which is called partial fine-tuning.

[0047] like Figure 1 As shown, the specific implementation steps are as follows:

[0048] 1. Model training phase for dual semantic alignment

[0049] Step 1: Prepare a corpus of entities in both Chinese and English. Using a Chinese-English dictionary, we can create a corpus of entities in both Chinese and English based on the Chinese sentence set (defined as D cn ) in a sentence of Chinese herbal medicine and target Chinese entity e cn , from the English sentence set (defined as D en ) to find the corresponding English entity e en . Record the corresponding relationship and form an entity comparison corpus in Chinese and English, expressed as

[0050] i is any integer,

[0051] in is a pair of Chinese and English entities.

[0052] Step 2: Calculate the loss function required for multilingual alignment. Define entities in sentences or The vector obtained by the encoder mBERT model is represented as or The calculation formula of the loss function required for multilingual alignment is as follows:

[0053]

[0054] Step 3: Calculate the loss function required for professional semantic alignment. Using professional synonym dictionaries, from the dataset D cn ×D en Entity e i and its equivalent semantics a i The set of aliases of is defined as {D a |(e1, a1), (e2, a2)...(e i , a i ), i is any integer}, and are their corresponding embedding vectors. The loss function L required for professional semantic alignment align The calculation formula is as follows:

[0055]

[0056] Step 4: Calculate the regularization term. To prevent the model from tending to the simple solution of all zero vectors, a regularization term is added:

[0057]

[0058] in, is generated by the initial parameters of the encoder mBERT model before training iThe entity corresponds to a vector. Regularization uses the mean square error loss, expressed as ||·||. The mean square error calculates the difference in the vector modulus.

[0059] Step 5: Calculate the sum of the loss functions and train the mBERT model. The parameters of the mBERT model are trained by minimizing the following alignment loss function to optimize the word vector alignment for dual semantics:

[0060] L=min∑L crosslingual +L align +L regularize (4)

[0061] 2. Model training phase method based on partial fine-tuning

[0062] Step 6: While keeping most of the parameters of the mBERT model trained in step 5 unchanged, train the encoder mBERT model and decoder CRF at the same time. The mBERT model consists of multiple identical encoder structures connected in series. During the training process, only the last four encoders and decoder CRFs in the mBERT model are trained.

[0063] In order to verify the effect of the present invention, the entity recognition dataset uses part of the data related to herbs and targets in the public Chinese diabetes research literature entity relationship dataset DiaKG. The evaluation indicators use the common precision, recall and F1 indicators, and the calculation formula is as follows:

[0064]

[0065]

[0066]

[0067] The comparative experiment selected some classic methods in named entity recognition, including BiLSTM-CRF model, BERT-MRC model and advanced Global Pointer entity recognition model. The experimental results are as follows: Figure 2 As shown in the figure. Through experimental data, the current method is the best when the sample size is 100, 300 and 500.

[0068] Embodiment 2:

[0069] A Chinese-English bilingual Chinese herbal medicine and target entity recognition device based on dual semantic alignment, comprising:

[0070] Data construction module, used to build entity comparison corpora in Chinese and English;

[0071] The first calculation module is used to calculate the loss function required for multilingual alignment using the encoder mBERT model;

[0072] The second calculation module is used to calculate the loss function required for professional semantic alignment;

[0073] The third calculation module is used to calculate the regularization term;

[0074] The fourth calculation module is used to calculate the sum of the loss function and train the mBERT model;

[0075] The fifth computing module is used to simultaneously train the encoder mBERT model and the decoder CRF while keeping some parameters of the mBERT model trained in step 5 unchanged.

[0076] Embodiment three:

[0077] An electronic device includes a processor and a memory connected to the processor for storing instructions executable by the processor, wherein the processor is used to execute the above-mentioned Chinese and English bilingual Chinese herbal medicine and target entity recognition method based on dual semantic alignment.

[0078] Embodiment 4:

[0079] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned Chinese and English bilingual Chinese herbal medicine and target entity recognition method based on dual semantic alignment.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A Chinese-English bilingual Chinese herbal medicine and target entity recognition method based on dual semantic alignment, characterized by: The steps include: Step 1: Set up entity comparison corpora in Chinese and English; Step 2: Use the encoder mBERT model to calculate the loss function required for multilingual alignment; Step 3: Calculate the loss function required for professional semantic alignment; Step 4: Calculate the regularization term; Step 5: Calculate the sum of the loss functions and train the mBERT model; Step 6: While keeping some parameters of the mBERT model trained in step 5 unchanged, train the encoder mBERT model and decoder CRF at the same time.

2. According to claim 1, a method for identifying Chinese and English bilingual Chinese herbal medicines and target entities based on dual semantic alignment is characterized by: It also includes the use of precision, recall, and F1 indicators to evaluate recognition accuracy.

3. The method for identifying Chinese and English herbal medicines and target entities based on dual semantic alignment according to claim 1, characterized in that: In the step 1, a Chinese-English dictionary is used to select a sentence set (defined as D cn ) in a sentence of Chinese herbal medicine and target Chinese entity e cn , from the English sentence set (defined as D en ) to find the corresponding English entity e en , record the corresponding relationship, and form an entity comparison corpus in Chinese and English, expressed as i is any integer, where is a pair of Chinese and English entities.

4. The method for identifying Chinese and English herbal medicines and target entities based on dual semantic alignment according to claim 3, characterized in that: In step 2, define the entities in the sentence or The vector obtained by the encoder mBERT model is represented as or The calculation formula of the loss function required for multilingual alignment is as follows:

5. The method for identifying Chinese and English herbal medicines and target entities based on dual semantic alignment according to claim 4, characterized in that: In step 3, a professional synonym dictionary is used to extract the word cn ×D en Entity e i and its equivalent semantics a i The set of aliases of is defined as {D a |(e1, a1), (e2, a2)...(e i , a i ), i is any integer}, and For their corresponding embedding vectors, the loss function L required for professional semantic alignment align The calculation formula is as follows:

6. The method for identifying Chinese and English herbal medicines and target entities based on dual semantic alignment according to claim 5, characterized in that: In step 4, the regularization term is: in, is generated by the initial parameters of the encoder mBERT model before training i The entity corresponds to the vector, and the regularization uses the mean square error loss, expressed as ||·||. The mean square error calculates the difference in the vector modulus.

7. The method for identifying Chinese and English herbal medicines and target entities based on dual semantic alignment according to claim 6, characterized in that: In step 5, the parameters of the mBERT model are trained by minimizing the following alignment loss function to optimize the word vector alignment of dual semantics: L=min∑L crosslingual +L align +L regularize (4) 8. A Chinese-English bilingual Chinese herbal medicine and target entity recognition device based on dual semantic alignment, characterized by: include: Data construction module, used to build entity comparison corpora in Chinese and English; The first calculation module is used to calculate the loss function required for multilingual alignment using the encoder mBERT model; The second calculation module is used to calculate the loss function required for professional semantic alignment; The third calculation module is used to calculate the regularization term; The fourth calculation module is used to calculate the sum of the loss function and train the mBERT model; The fifth computing module is used to simultaneously train the encoder mBERT model and the decoder CRF while keeping some parameters of the mBERT model trained in step 5 unchanged.

9. An electronic device, comprising a processor and a memory connected to the processor for storing instructions executable by the processor, characterized in that: The processor is used to execute the Chinese-English bilingual Chinese herbal medicine and target entity recognition method based on dual semantic alignment as described in any one of claims 1-7 above.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for identifying Chinese and English bilingual Chinese herbal medicines and target entities based on dual semantic alignment as described in any one of claims 1 to 7 is implemented.

Images