Method, device, electronic device and memory for identifying named entities in ancient Chinese medicine books

Abstract

The present application relates to a method, device, electronic device and storage medium for identifying named entities in ancient Chinese medicine books in the field of information processing and Chinese medicine literature technology. The method comprises: obtaining the text of ancient Chinese medicine books, and pre-processing the text of ancient Chinese medicine books to obtain a text sequence of ancient Chinese medicine books; constructing a named entity recognition model for ancient Chinese medicine books, wherein the named entity recognition model comprises an embedding layer, an ELECTRA module, a BiLSTM module and a CRF module connected in sequence; in the pre-training stage, considering that the pre-training data is too small, a semi-fixed random masking mechanism is added to the model; in the fine-tuning stage, the model introduces the characteristics of Chinese character radicals to enrich the connotation of text information. The use of this method can improve the recognition effect of prescription entities, Chinese medicine entities, syndrome entities, disease entities and syndrome manifestation entities in ancient Chinese medicine books.

Description

Technical Field

[0001] The present application relates to the field of information processing and traditional Chinese medicine literature technology, and in particular to a method, device, electronic device and memory for recognizing named entities in ancient Chinese medicine books. Background Art

[0002] As the pace of digitization accelerates in Traditional Chinese Medicine (TCM), vast amounts of ancient Chinese medical texts and classics have become a crucial foundation for TCM research. Simultaneously, the ever-increasing volume of textual literature and medical diagnosis and treatment data has eclipsed traditional literature research methods. In this era of rapidly increasing data, leveraging textual and technological resources to uncover the inherent connections between TCM entities remains a crucial challenge for the development of TCM research.

[0003] The integration of named entity recognition (NER) and Traditional Chinese Medicine (TCM) is a pressing need, but progress in this area has been slow, particularly in the field of ancient TCM texts, where relatively few research results have been achieved. Existing NER methods based on pre-trained models employ a random masking mechanism for training. However, due to the limited training corpus in this field and the numerous function words and onomatopoeia in these texts, the use of random masks cannot guarantee training quality, resulting in NER results in TCM texts falling short of practical requirements. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, electronic device and storage for identifying named entities in ancient Chinese medicine books to address the above technical problems.

[0005] A method for identifying named entities in ancient Chinese medicine books, the method comprising:

[0006] Obtaining ancient Chinese medicine texts, and preprocessing the ancient Chinese medicine texts to obtain ancient Chinese medicine text sequences.

[0007] A named entity recognition model for ancient Chinese medicine books is constructed, wherein the named entity recognition model includes an embedding layer, an ELECTRA module, a BiLSTM module, and a CRF module connected in sequence.

[0008] During the pre-training phase, a keyword masking mechanism is added to the ELECTRA module; the keyword masking mechanism adopts a semi-fixed random masking method to randomly select keywords from a preset keyword table for masking.

[0009] The ancient Chinese medicine text sequence is used to pre-train the ELECTRA module after the embedding layer and the newly added keyword masking mechanism to obtain a pre-trained ELECTRA module.

[0010] During the fine-tuning stage, the radical feature input of the character is added to the embedding encoding module of the named entity recognition model to obtain a new embedding layer.

[0011] The fine-tuning training samples are used to fine-tune the named entity recognition model in the fine-tuning stage to obtain a trained named entity recognition model; the named entity recognition model in the fine-tuning stage includes a new embedding layer, a pre-trained ELECTRA module, a BiLSTM module, and a CRF module.

[0012] The trained named entity recognition model is used to identify named entities in ancient Chinese medicine books.

[0013] In one embodiment, the ELECTRA module after the embedding layer and the newly added keyword masking mechanism is pre-trained using the ancient Chinese medicine text sequence to obtain the pre-trained ELECTRA module, including:

[0014] Set pre-training step split parameters.

[0015] The total number of pre-training steps is divided into the number of steps trained by the conventional training method and the number of steps trained by the keyword mask mechanism according to the pre-training step division parameter.

[0016] The ELECTRA module is trained using the random masking mechanism of the ELECTRA module according to the encoding obtained by inputting the ancient Chinese medicine text sequence into the embedding layer, until the number of training steps reaches the number of training steps using the conventional training method, thereby obtaining the ELECTRA module after conventional training.

[0017] The conventionally trained ELECTRA module is further trained using a keyword masking mechanism according to the encoding obtained by inputting the ancient Chinese medicine text sequence into the embedding layer until the number of training steps reaches the number of training steps using the keyword masking mechanism, thereby obtaining a pre-trained ELECTRA module; the keyword masking mechanism uses a method of randomly masking 15% of tokens, where the tokens are keywords in a preset keyword table.

[0018] In one embodiment, the step of constructing the preset keyword table includes:

[0019] Extract keyword features of prescription entities, Chinese medicine entities, disease entities, syndrome entities, and symptom entities from ancient Chinese medicine books, and determine the keywords corresponding to each entity.

[0020] Extract words with an appearance frequency greater than or equal to 1 from the term dictionary as keywords of the term dictionary.

[0021] A preset keyword table is constructed based on the keywords of the term dictionary and the keywords corresponding to various entities.

[0022] In one embodiment, fine-tuning the named entity recognition model in the fine-tuning stage using the fine-tuning training sample to obtain a trained named entity recognition model includes:

[0023] The fine-tuning training samples are input into the new embedding layer of the named entity recognition model in the fine-tuning stage. A lookup operation is used to obtain the character embedding information, the character position information, and the encoding information of the sentence in which the character is located. The radical encoding of the Chinese character is obtained by lookup based on the Chinese character-radical index.

[0024] The word embedding information, the word position information, the encoding information of the sentence in which the word is located, and the radical encoding of the Chinese character are input into the pre-trained ELECTRA module of the named entity recognition model in the fine-tuning stage to obtain the word embedding representation.

[0025] The word embedding representation is input into the BiLSTM module of the named entity recognition model in the fine-tuning stage for feature extraction, and the extracted features are input into the CRF module for decoding to obtain a predicted label sequence.

[0026] According to the predicted label sequence and the corresponding token, the model loss of the fine-tuning stage is determined, and the named entity recognition model of the fine-tuning stage is fine-tuned according to the model loss of the fine-tuning stage and the annotation of the fine-tuning training sample to obtain a trained named entity recognition model.

[0027] In one embodiment, the obtained labeled text sequences of ancient Chinese medicine books are used as fine-tuning training samples, including:

[0028] Obtain ancient Chinese medicine texts.

[0029] Data correction is performed on the text of the ancient Chinese medicine books, and the correction content includes: correcting typos, replacing traditional Chinese characters, and modifying ambiguous characters.

[0030] The corrected ancient Chinese medicine texts were manually annotated with entities, and five types of entities were annotated; the five types of entities included: prescriptions, Chinese medicines, syndrome types, diseases, and symptom manifestations.

[0031] The annotated ancient Chinese medicine texts are divided according to the preset ratio to obtain the training set, validation set and test set.

[0032] The Chinese period, exclamation mark, and question mark are used as segmentation points to perform sequence segmentation on the divided data set. When the length of the segmented sequence is less than the preset maximum sequence length, the length of the segmented sequences is guaranteed to be similar.

[0033] The results of sequence segmentation are processed using the BIO annotation mode to obtain the text sequence of ancient Chinese medicine books with entity annotation.

[0034] The text sequences of ancient Chinese medicine books with entity annotations are used as fine-tuning training samples.

[0035] A device for identifying named entities in ancient Chinese medicine books, comprising:

[0036] The module for determining the sequence of ancient Chinese medicine texts is used to obtain ancient Chinese medicine texts and pre-process the ancient Chinese medicine texts to obtain ancient Chinese medicine text sequences.

[0037] The named entity recognition model construction module is used to construct a named entity recognition model for ancient Chinese medicine books. The named entity recognition model includes an embedding layer, an ELECTRA module, a BiLSTM module, and a CRF module connected in sequence.

[0038] The pre-training module of the ELECTRA module is used to add a keyword masking mechanism to the ELECTRA module during the pre-training stage; the keyword masking mechanism adopts a semi-fixed random masking method to randomly select keywords from a preset keyword table for masking; the ELECTRA module after the embedding layer and the newly added keyword masking mechanism is pre-trained using the ancient Chinese medicine text sequence to obtain a pre-trained ELECTRA module.

[0039] The named entity recognition model fine-tuning training module is used to add radical feature input of a character to the embedding coding module of the named entity recognition model in the fine-tuning stage to obtain a new embedding layer; use the obtained marked ancient Chinese medicine text sequence as a fine-tuning training sample; use the fine-tuning training sample to fine-tune the named entity recognition model in the fine-tuning stage to obtain a trained named entity recognition model; the named entity recognition model in the fine-tuning stage includes the new embedding layer, the pre-trained ELECTRA module, the BiLSTM module and the CRF module.

[0040] The module for identifying named entities in ancient Chinese medicine books is used to identify named entities in ancient Chinese medicine books using a trained named entity recognition model.

[0041] In one embodiment, the pre-training module of the ELECTRA module is further used to set a pre-training step segmentation parameter; divide the total pre-training steps into steps trained using a conventional training method and steps trained using a keyword masking mechanism according to the pre-training step segmentation parameter; train the ELECTRA module using the random masking mechanism of the ELECTRA module according to the encoding obtained by inputting the ancient Chinese medicine text sequence into the embedding layer, until the number of training steps reaches the number of training steps trained using the conventional training method, thereby obtaining the conventionally trained ELECTRA module; continue to train the conventionally trained ELECTRA module using the keyword masking mechanism according to the encoding obtained by inputting the ancient Chinese medicine text sequence into the embedding layer, until the number of training steps reaches the number of training steps trained using the keyword masking mechanism, thereby obtaining the pre-trained ELECTRA module; the keyword masking mechanism adopts a method of randomly masking 15% of tokens, and the tokens are keywords in a preset keyword table.

[0042] In one embodiment, the pre-training module of the ELECTRA module includes a module for constructing a preset keyword table, which is used to extract keyword features of prescription entities, Chinese medicine entities, disease entities, syndrome entities, and symptom manifestation entities in ancient Chinese medicine books, and determine the keywords corresponding to each type of entity; extract words with an appearance frequency greater than or equal to 1 from the term dictionary as keywords of the term dictionary; and construct a preset keyword table based on the keywords of the term dictionary and the keywords corresponding to each type of entity.

[0043] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0044] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the above methods.

[0045] The aforementioned method, device, electronic device, and storage medium for identifying named entities in ancient Chinese medicine texts include: obtaining ancient Chinese medicine text and preprocessing it to obtain a text sequence; constructing a named entity recognition model for ancient Chinese medicine texts, wherein the model comprises an embedding layer, an ELECTRA module, a BiLSTM module, and a CRF module connected in sequence; during the pre-training phase, a semi-fixed random masking mechanism is added to the model to account for the random masking mechanism of the ELECTRA model and the scarcity of new pre-training data; during the fine-tuning phase, the model incorporates Chinese character radical features to enrich the textual information. This method can improve the recognition of prescription entities, herbal medicine entities, syndrome type entities, disease entities, and syndrome manifestation entities in ancient Chinese medicine texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 1 is a flow chart of a method for identifying named entities in ancient Chinese medicine books according to an embodiment;

[0047] Figure 2 A schematic diagram of a keyword masking mechanism introduced into the pre-training phase in another embodiment;

[0048] Figure 3 The embedding layer of the named entity recognition model for ancient Chinese medicine books in the fine-tuning stage in another embodiment;

[0049] Figure 4 The change of entity indicators in one embodiment;

[0050] Figure 5 This is a structural block diagram of a device for identifying named entities in ancient Chinese medicine books in one embodiment;

[0051] Figure 6 FIG. 1 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0053] In one embodiment, Figure 1 As shown, a method for identifying named entities in ancient Chinese medicine books is provided, which includes the following steps:

[0054] Step 100: Acquire ancient Chinese medicine texts and pre-process the ancient Chinese medicine texts to obtain an ancient Chinese medicine text sequence.

[0055] Specifically, the ancient Chinese medicine texts are selected from ancient Chinese medicine documents from the Ming, Qing and other periods.

[0056] Preprocessing mainly includes steps such as text correction and data segmentation of ancient Chinese medicine texts.

[0057] The text sequence of ancient Chinese medicine books begins with the identifier [cls] and ends with [seq], for example: [cls]Tai Yangming combined disease [seq].

[0058] The text sequences of ancient Chinese medicine books are unlabeled text data.

[0059] Step 102: Construct a named entity recognition model for ancient Chinese medicine books.

[0060] The named entity recognition model for traditional Chinese medicine ancient books includes an embedding layer, an ELECTRA module, a BiLSTM module, and a CRF module connected in sequence.

[0061] Specifically, the embedding layer obtains word embedding (Token embeddings) information through a look-up operation; obtains the position information of words (Position embeddings) and the encoding information of the sentences where the words are located (Segment embeddings).

[0062] The ELECTRA module is a pre-trained model, mainly used to obtain the embedding representation of words. The training method of the model adopts the fine-tuning method. Compared with the feature method, this method can dynamically represent words according to the context information of words, solving the problem of polysemy; and the ELECTRA model has deeper representation features and stronger representation ability compared with ordinary neural network models.

[0063] BiLSTM has strong feature extraction ability for traditional Chinese medicine texts. Therefore, in this invention, a BiLSTM model (selectively) is introduced after the pre-trained model to obtain better entity recognition results.

[0064] The decoding layer uses a CRF model. The feature functions of CRF include state feature functions and transition feature functions. The state feature represents the state information at the current position, and the transition feature contains the feature information from the previous state to the current state. The nature of the feature functions of CRF enables better constraint on the predicted output, thus ensuring that the predicted label sequence conforms to the constraints of basic dependency relationships, and finally outputs the predicted label sequence.

[0065] Step 104: In the pre-training stage, a new keyword masking mechanism will be added to the ELECTRA module; the keyword masking mechanism adopts a semi-fixed random masking method, randomly selecting keywords from a preset keyword table for masking.

[0066] Specifically, in traditional Chinese medicine ancient books, function words are an important part of ancient literature. Taking "A Supplement to the Famous Physicians' Discussions on Prescriptions" as an example, 18 common function words in ancient Chinese such as "zhi", "zhe", "hu", "ye" account for 13.69%. If a random masking (mask) mechanism is adopted, the model cannot systematically learn the key vocabulary in the text. In the pre-training stage, considering the random mask mechanism of the ELECTRA model and the过少 new pre-trained data, a new keyword mask mechanism is proposed in the pre-training stage of the ELECTRA model. This mechanism adopts a semi-fixed random mask method, randomly masking the words in the keyword table in the random mask sequence to better learn the keyword features of ancient Chinese texts. The schematic diagram of the model is as Figure 2 shown.

[0067] The reason why this paper chooses to add a keyword mask mechanism after the conventional mask mechanism is that, firstly, the training corpus is relatively small, and there are many function words and onomatopoeia in ancient Chinese medicine books, and the random mask mechanism cannot guarantee the training quality; secondly, learning tokens with greater difficulty in the subsequent stages of the model is more in line with training reality.

[0068] Step 106: Pre-train the ELECTRA module after the embedding layer and the newly added keyword masking mechanism using the ancient Chinese medicine text sequence to obtain a pre-trained ELECTRA module.

[0069] Specifically, considering that the ELECTRA Chinese model is trained with Chinese Wikipedia corpus, and there is a big gap between the texts in the field of traditional Chinese medicine and the general field, and the language differences between ancient Chinese and modern Chinese texts are large, this study chose to use the new training model of ancient Chinese medicine texts. The literature was selected from the Ming, Qing and other periods. The books included in each period are shown in Table 1.

[0070] Table 1 Books

[0071]

[0072] Note: The reason why the complete set of "Yi Zong Jin Jian" in the table does not include "Shan Bu Ming Yi Fang Lun" is that "Shan Bu Ming Yi Fang Lun" is a manually annotated dataset. During actual training, the dataset will be divided into training set: validation set: test set = 8:1:1. To ensure the fairness, validity, and objectivity of the test results, "Shan Bu Ming Yi Fang Lun" was not pre-trained to avoid affecting performance judgment.

[0073] Step 108: In the fine-tuning stage, the radical feature input of the character is added to the embedding encoding module of the named entity recognition model to obtain a new embedding layer;

[0074] Specifically, when fine-tuning the BiLSTM-CRF model after introducing the ELECTRA pre-trained model, considering that the existing pre-trained ELECTRA model has less information carried by some rare characters with low frequency of occurrence, the present invention proposes to add the radical feature input of the character to the embedded coding part of the model to enrich the connotation of the text information.

[0075] The specific implementation method is to first randomly initialize the parameters, and then obtain the radical encoding of the Chinese character based on the Chinese character-radical index lookup. The loss of the fine-tuning training phase is the sum of the losses of all tokens. The modified embedding layer diagram is as follows Figure 3 shown.

[0076] Step 110: Use the obtained labeled ancient Chinese medicine text sequence as a fine-tuning training sample.

[0077] Step 112: Fine-tune the named entity recognition model in the fine-tuning stage using the fine-tuning training sample to obtain a trained named entity recognition model; the named entity recognition model in the fine-tuning stage includes a new embedding layer, a pre-trained ELECTRA module, a BiLSTM module, and a CRF module.

[0078] Step 114: Use the trained named entity recognition model to identify the named entities in ancient Chinese medicine books.

[0079] The aforementioned method for identifying named entities in ancient Chinese medicine texts includes: obtaining ancient Chinese medicine texts and preprocessing them to obtain a text sequence; constructing a named entity recognition model for ancient Chinese medicine texts, wherein the model comprises an embedding layer, an ELECTRA module, a BiLSTM module, and a CRF module connected in sequence; during the pretraining phase, a semi-fixed random masking mechanism is added to the model to account for the random masking mechanism of the ELECTRA model and the scarcity of new pretraining data; during the fine-tuning phase, the model incorporates Chinese character radical features to enrich the textual information. This method can improve the recognition of prescription entities, Chinese medicine entities, syndrome entities, disease entities, and syndrome manifestation entities in ancient Chinese medicine texts.

[0080] In one embodiment, step 106 includes: setting a pre-training step segmentation parameter; dividing the total pre-training steps into steps trained using a conventional training method and steps trained using a keyword masking mechanism according to the pre-training step segmentation parameter; training the ELECTRA module using a random masking mechanism of the ELECTRA module according to the encoding obtained by inputting the ancient Chinese medicine text sequence into the embedding layer, until the number of training steps reaches the number of training steps using the conventional training method, thereby obtaining the conventionally trained ELECTRA module; continuing to train the conventionally trained ELECTRA module using a keyword masking mechanism according to the encoding obtained by inputting the ancient Chinese medicine text sequence into the embedding layer, until the number of training steps reaches the number of training steps using the keyword masking mechanism, thereby obtaining the pre-trained ELECTRA module; the keyword masking mechanism uses a method of randomly masking 15% of tokens, where the tokens are keywords in a preset keyword table.

[0081] Specifically, set the pre-training step split parameter divide_proportion to split the training steps. The number of steps trained using the conventional training method is divide_proportion num_train_steps, (where num_train_steps is the total number of pre-training steps), the remaining steps use the keyword mask mechanism. For the newly added keyword mask mechanism, the random mask 15% token method is still used ( ), at the same time, the transformer structure remains unchanged, and the loss of the model is still the sum of the losses of the generator and discriminator parts.

[0082] In one embodiment, the preset keyword table in step 104 is constructed using the following steps: extracting keyword features of prescription entities, Chinese medicine entities, disease entities, syndrome entities, and symptom manifestation entities in ancient Chinese medicine books, and determining keywords corresponding to each type of entity; extracting words with an appearance frequency greater than or equal to 1 from the terminology dictionary as keywords of the terminology dictionary; and constructing a preset keyword table based on the keywords of the terminology dictionary and the keywords corresponding to each type of entity.

[0083] Specifically, keyword data is used to improve the model during the pre-training phase. Keyword data consists of two parts: keyword features that can be directly summarized and obtained, and high-frequency characters from a collected terminology dictionary (to ensure smooth experimental progress, high-frequency characters refer to Chinese characters that appear at least once). This study examines entity recognition for five entity categories: prescriptions, Chinese herbal medicines, diseases, syndrome types, and symptom manifestations. Keyword features are present in each entity category. Prescription entities, such as "Shenfu Decoction" and "Baoyuan Decoction," are often based on dosage type keywords such as "soup," "medicine," and "drink." Disease entities, such as "Taiyang Disease" and "Taiyang-Yangming Combined Disease," often contain the character "disease." Syndrome type entities often contain syndrome elements related to the heart, liver, spleen, lung, and kidney. Symptom manifestations, such as pulse and tongue, often contain the keywords "pulse" and "tongue." To this end, the authors compiled keywords and terminology dictionaries for these five entity categories. The terminology dictionary includes 917 common prescription names, 2035 disease names, 3199 syndrome types, and 2461 symptom manifestation names. The sorted keyword table is shown in Table 2.

[0084] Table 2 Keyword table

[0085]

[0086] In one embodiment, step 112 includes: inputting the traditional Chinese medicine ancient book text sequence into the new embedding layer of the named entity recognition model in the fine-tuning stage, and using the look-up operation to obtain the character embedding information, the position information of the character, and the encoding information of the sentence where the character is located, and obtaining the radical encoding of the character according to the Chinese character-radical index look-up; inputting the character embedding information, the position information of the character, the encoding information of the sentence where the character is located, and the radical encoding of the character into the pre-trained ELECTRA module of the named entity recognition model in the fine-tuning stage to obtain the embedding representation of the character; inputting the embedding representation of the character into the BiLSTM module of the named entity recognition model in the fine-tuning stage for feature extraction, and inputting the extracted features into the CRF module for decoding to obtain the predicted label sequence; determining the model loss in the fine-tuning stage according to the predicted label sequence and the corresponding tokens, and fine-tuning and training the named entity recognition model in the fine-tuning stage according to the model loss in the fine-tuning stage and the annotation of the fine-tuning training samples to obtain the trained named entity recognition model.

[0087] Specifically, the radical data is used for the improvement of the model in the fine-tuning stage. Traditional Chinese medicine culture is the crystallization of human historical civilization and spiritual civilization, and the spread and development of traditional Chinese medicine culture are inseparable from Chinese character culture. Chinese characters originated from oracle bone inscriptions in 1000 BC. Oracle bone inscriptions are highly pictorial and are typical pictographic characters. After that, Chinese characters have been continuously developed, and the structure of Chinese characters has changed from complex to simple. However, as the recording symbols of Chinese, Chinese characters have always used a certain system of symbols to symbolize or express the semantics of words. Currently, simplified Chinese characters are common symbols in Chinese texts, and the basic unit of composition is radicals. Chinese character radicals also have meanings. For example, the radical "目" is mostly related to eyes, and the radical "疒" is mostly related to diseases, etc. In order to enable the machine to learn relevant features, this study added Chinese character radicals as inputs. The author counted the full text of "Shanbu Mingyi Fanglun", and a total of 1,831 Chinese characters (excluding duplicates) and 225 radicals were obtained. If a Chinese character has multiple radicals, the optimal radical is selected manually. The character radicals are obtained by crawling Baidu Chinese. The Chinese character-radical table is shown in Table 3, and the radicals are separately generated into a txt file for radical encoding.

[0088] Table 3 Chinese character-radical table

[0089]

[0090] In one embodiment, step 110 includes: obtaining traditional Chinese medicine ancient medical texts; performing data correction on the traditional Chinese medicine ancient medical texts in the fine-tuning stage for the labeled pairs of the out-of-consensus and fine-tuning training samples, and the correction content includes: correcting typos, replacing traditional Chinese characters, and modifying ambiguous characters; manually performing entity annotation on the corrected traditional Chinese medicine ancient medical texts, and annotating 5 types of entities; the labeled pairs of the out-of-consensus and fine-tuning training samples for the 5 types of entities in the fine-tuning stage include: prescriptions, traditional Chinese medicines, syndrome types, diseases, and symptom manifestations; dividing the labeled traditional Chinese medicine ancient medical texts according to a preset ratio to obtain a training set, a validation set, and a test set; using Chinese full stops, exclamation marks, and question marks as segmentation points to perform sequence segmentation processing on the divided data sets, and ensuring that the lengths of the segmented sequences are similar when the length of the segmented sequence is less than the maximum value of the preset sequence length; performing data processing on the results of the sequence segmentation processing using the BIO annotation mode to obtain the sequence of the traditional Chinese medicine ancient medical texts after entity annotation; using the sequence of the traditional Chinese medicine ancient medical texts after entity annotation as the fine-tuning training sample.

[0091] Specifically, the correction content includes correcting typos, replacing traditional Chinese characters, and modifying ambiguous characters, such as modifying "Astragalus membranaceus var. mongholicus" to "Astragalus membranaceus", and replacing "Ligusticum wallichii" with "Ligusticum chuanxiong".

[0092] Manually perform entity annotation, and select the doccano term annotation system as the annotation tool. A total of 5 types of entities, namely prescriptions, traditional Chinese medicines, syndrome types, diseases, and symptom manifestations, are annotated. The definitions and examples of each type of entity are shown in Table 4. Among them, the entity of symptom manifestations includes symptoms, pulse conditions, and tongue manifestations. For example, symptoms such as "vomiting roundworms immediately after eating", and pulse conditions such as "the pulse is weak". When performing entity annotation, no processing such as removing adverbs and adjectives is done. The annotation results are cross-reviewed and modified by multiple Chinese medicine doctors. Finally, the number of manually annotated entities of each type is as follows (including duplicates): 760 prescriptions, 3375 traditional Chinese medicines, 831 syndrome types, 593 diseases, and 1762 symptom manifestations.

[0093] Table 4 Entities, Entity Categories, and Entity Definitions

[0094]

[0095] After data segmentation, the BIO annotation mode is used to process the data. B represents the beginning, indicating the start of the entity. I represents the inside, indicating other positions except the start of the entity. O represents the outside, indicating non-entity categories. For example, for the prescription "Renshen Yangrong Decoction", the annotation for "Ren" is B-Prescription, and the annotations for the remaining parts of the entity are I-Prescription. Specific annotation examples are shown in Table 5.

[0096] Table 5 Specific Annotation Example Table

[0097]

[0098] (2) Data Segmentation: Since the pre-trained model ELECTRA limits the length of the input sequence, and considering the limited training data in this study, the authors performed sequence segmentation on the divided dataset. During sequence segmentation, the Chinese period, exclamation mark, and question mark were used as the segmentation points. As long as the length of the segmented sequence was less than the maximum sequence length preset by the hyperparameter (max_seq_length), the length of the segmented sequences was kept as close as possible.

[0099] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0100] In a verification example, this study conducted experiments on the proposed optimization scheme. In Experiment 1, optimization was performed only during the model pre-training phase; in Experiment 2, optimization was performed only during the model fine-tuning phase; and in Experiment 3, a keywords mask mechanism was added during the pre-training phase and Chinese character radicals were introduced as input features during the fine-tuning phase.

[0101] (1) Experiment 1 Core parameters: divide_proportion = 7 / 8, num_train_steps = 10, 0000, learning_rate = 5e-4, electra_objective = True, model_size = "base"; epoch = 40, learning-rate = 3e-4; other parameters are shown in Table 6, and the experimental results are shown in Table 7;

[0102] Table 6 Other parameters

[0103]

[0104] Table 7 Index values ​​of the model when only the semi-fixed random mask strategy is introduced

[0105]

[0106] (2) Experiment 2: The core parameters are use_radical_embedding=True, epoch=40, Learning-rate=3e-4; the remaining parameters are the same as those in Table 6, and the experimental results are shown in Table 8;

[0107] Table 8 Index values of the model when only radical features are introduced

[0108]

[0109] (3) Experiment 3: The core parameters are use_radical_embedding=True, epoch=40, Learning-rate=3e-4; the remaining parameters are the same as those in Table 6, and the experimental results are shown in Table 9.

[0110] Table 9 Index values of the optimized model

[0111]

[0112] Table 10 Index values of the model before optimization

[0113]

[0114] By comparing the evaluation indexes of the models before and after optimization, it can be found that the method of adding the keyword mask mechanism in the pre-training stage has a greater improvement on the model performance than the method of introducing Chinese character radical features in the fine-tuning stage. If the two methods are combined, greater benefits can be obtained. The average P, R, and F1 values of each model entity are as Figure 4 shown.

[0115] By comparing Table 9 and Table 10, it can be found that after the model is improved, the recognition effects of the entities of prescriptions, traditional Chinese medicines, syndrome types, diseases, and syndrome manifestations have all increased slightly, by 3.1%, 0.4%, 5.01%, 7.77%, and 7.05% respectively. Among them, the disease entity has the largest increase, and the traditional Chinese medicine entity has the smallest increase. The main reason for the large increase in the disease entity is that the keywords and radical features of the text are obvious. The reason for the small increase in the traditional Chinese medicine entity is that most of the entities have been recognized by the system, and for the remaining entities such as Ephedra and Cassia Twig in "Mahuang Guizhi", it is difficult to solve only by adding radical features. From the experimental results, it can be found that this optimization method does help with recognition. For example, for the syndrome manifestation "resembling malaria", it was judged as "O" before optimization, and the disease "malaria" can be recognized after optimization; for "intermittent and irregular pulse", "intermittent pulse" can be recognized.

[0116] In one embodiment, such as Figure 5As shown, a device for identifying named entities in ancient Chinese medicine books is provided, comprising: a module for determining a text sequence of ancient Chinese medicine books, a module for building a named entity recognition model, a pre-training module for an ELECTRA module, a module for fine-tuning a named entity recognition model, and a module for identifying named entities in ancient Chinese medicine books, wherein:

[0117] The module for determining the sequence of ancient Chinese medicine texts is used to obtain the ancient Chinese medicine texts and pre-process the ancient Chinese medicine texts to obtain the ancient Chinese medicine text sequences.

[0118] The named entity recognition model construction module is used to construct a named entity recognition model for ancient Chinese medicine books. The named entity recognition model includes an embedding layer, an ELECTRA module, a BiLSTM module, and a CRF module connected in sequence.

[0119] The pre-training module of the ELECTRA module is used to add a keyword masking mechanism to the ELECTRA module during the pre-training stage. The keyword masking mechanism uses a semi-fixed random masking method to randomly select keywords from a preset keyword table for masking. The ELECTRA module with the embedding layer and the newly added keyword masking mechanism is pre-trained using a text sequence of ancient Chinese medicine books to obtain a pre-trained ELECTRA module.

[0120] The named entity recognition model fine-tuning training module is used to add character radical feature input to the embedding coding module of the named entity recognition model in the fine-tuning stage to obtain a new embedding layer; use the obtained marked traditional Chinese medicine ancient book text sequence as a fine-tuning training sample; use the fine-tuning training sample to fine-tune the named entity recognition model in the fine-tuning stage to obtain a trained named entity recognition model; the named entity recognition model in the fine-tuning stage includes the new embedding layer, the pre-trained ELECTRA module, the BiLSTM module and the CRF module.

[0121] The module for identifying named entities in ancient Chinese medicine books is used to identify named entities in ancient Chinese medicine books using a trained named entity recognition model.

[0122] In one embodiment, the pre-training module of the ELECTRA module is further used to set a pre-training step segmentation parameter; divide the total pre-training steps into steps trained using a conventional training method and steps trained using a keyword masking mechanism according to the pre-training step segmentation parameter; train the ELECTRA module using the random masking mechanism of the ELECTRA module according to the encoding obtained by inputting the ancient Chinese medicine text sequence into the embedding layer until the number of training steps reaches the number of training steps trained using the conventional training method, thereby obtaining the conventionally trained ELECTRA module; continue to train the conventionally trained ELECTRA module using the keyword masking mechanism according to the encoding obtained by inputting the ancient Chinese medicine text sequence into the embedding layer until the number of training steps reaches the number of training steps trained using the keyword masking mechanism, thereby obtaining the pre-trained ELECTRA module; the keyword masking mechanism adopts a method of randomly masking 15% of tokens, and the tokens are keywords in a preset keyword table.

[0123] In one embodiment, the pre-training module of the ELECTRA module includes a module for constructing a preset keyword table, which is used to extract keyword features of prescription entities, Chinese medicine entities, disease entities, syndrome entities, and symptom manifestation entities in ancient Chinese medicine books, and determine the keywords corresponding to each type of entity; extract words with an appearance frequency greater than or equal to 1 from the terminology dictionary as keywords of the terminology dictionary; and construct a preset keyword table based on the keywords of the terminology dictionary and the keywords corresponding to each type of entity.

[0124] In one embodiment, the named entity recognition model fine-tuning training module is further used to input the ancient Chinese medicine text sequence into the new embedding layer of the named entity recognition model in the fine-tuning stage, use a lookup operation to obtain character embedding information, character position information, and encoding information of the sentence in which the character is located, and obtain the radical encoding of the Chinese character according to the Chinese character-radical index lookup; input the character embedding information, character position information, encoding information of the sentence in which the character is located, and the radical encoding of the Chinese character into the pre-trained ELECTRA module of the named entity recognition model in the fine-tuning stage to obtain the character embedding representation; input the character embedding representation into the BiLSTM module of the named entity recognition model in the fine-tuning stage for feature extraction, input the extracted features into the CRF module for decoding to obtain a predicted label sequence; determine the model loss of the fine-tuning stage based on the predicted label sequence and the corresponding token, and fine-tune the named entity recognition model in the fine-tuning stage based on the model loss of the fine-tuning stage and the annotation of the fine-tuning training sample to obtain a trained named entity recognition model.

[0125] In one embodiment, the ancient Chinese medicine text sequence determination module is also used to obtain ancient Chinese medicine texts; perform data correction on the ancient Chinese medicine texts, and the correction content includes: correcting typos, replacing traditional Chinese characters, and modifying ambiguous characters; manually perform entity annotation on the corrected ancient Chinese medicine texts, and mark out 5 types of entities; the 5 types of entities include: prescriptions, Chinese medicines, syndrome types, diseases, and symptom manifestations; divide the annotated ancient Chinese medicine texts according to a preset ratio to obtain a training set, a validation set, and a test set; use Chinese periods, exclamation marks, and question marks as segmentation points to perform sequence segmentation processing on the divided data set, and ensure that the lengths of the segmented sequences are similar when the length of the segmented sequence is less than the preset maximum sequence length; use the BIO annotation mode to process the results of the sequence segmentation processing to obtain an entity-annotated ancient Chinese medicine text sequence; and use the entity-annotated ancient Chinese medicine text sequence as a fine-tuning training sample.

[0126] Regarding the specific limitations of the device for identifying named entities in ancient Chinese medicine books, please refer to the limitations of the method for identifying named entities in ancient Chinese medicine books above, which will not be repeated here. The various modules in the above-mentioned device for identifying named entities in ancient Chinese medicine books can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0127] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for identifying named entities in ancient Chinese medicine books is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0128] Those skilled in the art will understand that Figure 6The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0129] In one embodiment, an electronic device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiment when executing the computer program.

[0130] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiment are implemented.

[0131] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0132] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0133] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for identifying named entities in ancient Chinese medicine books, characterized in that: The method comprises: Acquiring ancient Chinese medicine texts, and preprocessing the ancient Chinese medicine texts to obtain an ancient Chinese medicine text sequence; Construct a named entity recognition model for ancient Chinese medicine books, wherein the named entity recognition model includes an embedding layer, an ELECTRA module, a BiLSTM module, and a CRF module connected in sequence; During the pre-training phase, a keyword masking mechanism is added to the ELECTRA module. The keyword masking mechanism uses a semi-fixed random masking method to randomly select keywords from a preset keyword table for masking. Pre-training the ELECTRA module after the embedding layer and the newly added keyword masking mechanism using the ancient Chinese medicine text sequence to obtain a pre-trained ELECTRA module; In the fine-tuning stage, the radical feature input of the character is added to the embedding coding module of the named entity recognition model to obtain a new embedding layer; The obtained labeled text sequences of ancient Chinese medicine books are used as fine-tuning training samples; Fine-tuning the named entity recognition model in the fine-tuning stage using the fine-tuning training sample to obtain a trained named entity recognition model; the named entity recognition model in the fine-tuning stage includes a new embedding layer, a pre-trained ELECTRA module, a BiLSTM module, and a CRF module; The trained named entity recognition model is used to identify named entities in ancient Chinese medicine books.

2. The method according to claim 1, characterized in that The ELECTRA module after the embedding layer and the newly added keyword masking mechanism is pre-trained using the ancient Chinese medicine text sequence to obtain the pre-trained ELECTRA module, including: Set the pre-training step split parameters; Splitting the total number of pre-training steps into steps trained using a conventional training method and steps trained using a keyword masking mechanism according to the pre-training step splitting parameter; According to the encoding obtained by inputting the ancient Chinese medicine text sequence into the embedding layer, the ELECTRA module is trained using the random masking mechanism of the ELECTRA module until the number of training steps reaches the number of training steps using the conventional training method, thereby obtaining the conventionally trained ELECTRA module; According to the encoding obtained by inputting the ancient Chinese medicine text sequence into the embedding layer, the keyword masking mechanism is used to continue training the ELECTRA module after conventional training until the number of training steps reaches the number of training steps using the keyword masking mechanism, thereby obtaining a pre-trained ELECTRA module; the keyword masking mechanism uses a method of randomly masking 15% of tokens, and the tokens are keywords in a preset keyword table.

3. The method according to claim 1, characterized in that The steps for building a preset keyword table include: Extract keyword features of prescription entities, Chinese medicine entities, disease entities, syndrome entities, and symptom entities from ancient Chinese medicine books, and determine the keywords corresponding to each entity; Extract words with a frequency of occurrence greater than or equal to 1 from the term dictionary as keywords of the term dictionary; A preset keyword table is constructed based on the keywords of the term dictionary and the keywords corresponding to various entities.

4. The method according to claim 1, wherein Fine-tuning the named entity recognition model in the fine-tuning stage using the fine-tuning training samples to obtain a trained named entity recognition model, including: Inputting the fine-tuning training sample into a new embedding layer of the named entity recognition model in the fine-tuning stage, performing a lookup operation to obtain character embedding information, character position information, and encoding information of the sentence in which the character is located, and obtaining the radical encoding of the Chinese character based on the Chinese character-radical index lookup; The word embedding information, the word position information, the encoding information of the sentence in which the word is located, and the radical encoding of the Chinese character are input into the pre-trained ELECTRA module of the named entity recognition model in the fine-tuning stage to obtain the word embedding representation. The word embedding representation is input into the BiLSTM module of the named entity recognition model in the fine-tuning stage for feature extraction, and the extracted features are input into the CRF module for decoding to obtain a predicted label sequence; According to the predicted label sequence and the corresponding token, the model loss of the fine-tuning stage is determined, and the named entity recognition model of the fine-tuning stage is fine-tuned according to the model loss of the fine-tuning stage and the annotation of the fine-tuning training sample to obtain a trained named entity recognition model.

5. The method according to claim 1, wherein The obtained labeled text sequences of ancient Chinese medicine books are used as fine-tuning training samples, including: Access to ancient Chinese medicine texts; Performing data correction on the text of the ancient Chinese medicine books, including: correcting typos, replacing traditional Chinese characters, and modifying ambiguous characters; The corrected ancient Chinese medicine texts were manually annotated with entities, and five types of entities were annotated. The five types of entities include: prescriptions, Chinese medicines, syndrome types, diseases, and symptom manifestations. The annotated ancient Chinese medicine texts are divided according to the preset ratio to obtain the training set, validation set and test set; Using Chinese periods, exclamation marks, and question marks as segmentation points, the data set is segmented and processed. The lengths of the segmented sequences are kept similar, provided that the length of the segmented sequences is less than the preset maximum length. The results of sequence segmentation are processed using the BIO annotation mode to obtain the text sequence of ancient Chinese medicine books with entity annotation; The text sequences of ancient Chinese medicine books with entity annotations are used as fine-tuning training samples.

6. A device for identifying named entities in ancient Chinese medicine books, characterized in that: The device comprises: A module for determining a sequence of ancient Chinese medicine texts, configured to obtain ancient Chinese medicine texts and pre-process the ancient Chinese medicine texts to obtain a sequence of ancient Chinese medicine texts; A named entity recognition model construction module is used to construct a named entity recognition model for ancient Chinese medicine books. The named entity recognition model includes an embedding layer, an ELECTRA module, a BiLSTM module, and a CRF module connected in sequence; The pre-training module of the ELECTRA module is used to add a keyword masking mechanism to the ELECTRA module during the pre-training phase; the keyword masking mechanism adopts a semi-fixed random masking method to randomly select keywords from a preset keyword table for masking; the ELECTRA module after the embedding layer and the keyword masking mechanism is pre-trained using the ancient Chinese medicine text sequence to obtain a pre-trained ELECTRA module; A named entity recognition model fine-tuning training module is configured to, during the fine-tuning phase, add character radical feature inputs to the embedding coding module of the named entity recognition model to obtain a new embedding layer; use the acquired labeled ancient Chinese medicine text sequence as a fine-tuning training sample; and fine-tune the named entity recognition model in the fine-tuning phase using the fine-tuning training sample to obtain a trained named entity recognition model; the named entity recognition model in the fine-tuning phase includes the new embedding layer, the pre-trained ELECTRA module, the BiLSTM module, and the CRF module; The module for identifying named entities in ancient Chinese medicine books is used to identify named entities in ancient Chinese medicine books using a trained named entity recognition model.

7. The device according to claim 6, characterized in that The pre-training module of the ELECTRA module is further used to set a pre-training step segmentation parameter; divide the total pre-training steps into steps trained using a conventional training method and steps trained using a keyword masking mechanism according to the pre-training step segmentation parameter; train the ELECTRA module using the random masking mechanism of the ELECTRA module according to the encoding obtained by inputting the ancient Chinese medicine text sequence into the embedding layer, until the number of training steps reaches the number of training steps trained using the conventional training method, thereby obtaining the conventionally trained ELECTRA module; continue to train the conventionally trained ELECTRA module using the keyword masking mechanism according to the encoding obtained by inputting the ancient Chinese medicine text sequence into the embedding layer, until the number of training steps reaches the number of training steps trained using the keyword masking mechanism, thereby obtaining the pre-trained ELECTRA module; the keyword masking mechanism adopts a method of randomly masking 15% of tokens, and the tokens are keywords in a preset keyword table.

8. The device according to claim 6, characterized in that The pre-training module of the ELECTRA module includes a module for constructing a preset keyword table, which is used to extract keyword features of prescription entities, Chinese medicine entities, disease entities, syndrome entities, and symptom manifestation entities in ancient Chinese medicine books, and determine the keywords corresponding to each entity; Extract words with a frequency of occurrence greater than or equal to 1 from the term dictionary as keywords of the term dictionary; A preset keyword table is constructed based on the keywords of the term dictionary and the keywords corresponding to various entities.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images