A method, device and equipment for entity recognition and relationship extraction of traditional Chinese medicine medical records

Abstract

The present invention discloses a method, device and equipment for entity recognition and relationship extraction of traditional Chinese medical records, and relates to the technical field of natural language processing. In order to solve the inaccurate relationship extraction caused by the multi-level meanings of named entities, the present invention obtains character vectors, associated word vectors and radical vectors corresponding to each character in the to-be-processed traditional Chinese medical records; inputs each vector into a first neural network model to obtain a named entity recognition result; and inputs the medical record sentences with the recognition results into a second neural network model to obtain the relationship between each named entity. The method extracts the named entities associated with each character, and obtains all possible semantic environments of each character in the medical record; classifies the named entities in the associated word set, and obtains the usage patterns of each character in different semantic environments; weights the named entities in the associated word set, and reveals which semantic environment each character is more inclined to, which helps to obtain more accurate named entity recognition results.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method, device and equipment for entity recognition and relationship extraction of traditional Chinese medicine medical records. Background Art

[0002] Medical records are true records of TCM clinical practice, which contain rich diagnosis and treatment experience and treatment methods, and are the experience and lessons summarized by doctors in clinical practice. With the advancement of the modernization strategy of TCM, FCRs (Free-text Clinical Records) such as famous medical records have begun to be informatized or digitized. Structuring TCM texts and extracting named entity information (referring to specific identifiable objects in medical records, such as diseases, syndromes, etc.) and relationship information between named entities (referring to the associations between entities, such as the relationship between diseases and syndromes, the relationship between syndromes and treatment methods, etc.) are of great significance to promoting the development of TCM.

[0003] In the prior art, Chinese patent CN113962227A discloses a named entity extraction system for TCM clinical medical case symptoms based on LSTM-CRF. The system includes: an LSTM-CRF network model, which includes an input layer, a feature extraction layer, an LSTM layer, and a CRF layer; the input layer is used to obtain text information of TCM clinical medical case symptoms; the feature extraction layer is used to extract features for each word of the input text information and formulate a feature template; the LSTM layer is used to automatically extract the features of the text; the CRF layer is used for text-level sequence annotation to generate the optimal annotation sequence. Among them, the text features extracted by the feature extraction layer include the character features, character features, radical features, and word meaning features of the text.

[0004] However, the terms in TCM medical records often have multi-level meanings. The same named entity may be both a disease and a symptom. This multi-level meaning will affect the understanding of the semantics of the entire TCM medical record. As a result, when extracting the relationships between named entities in TCM medical records based on the above method, the accuracy of the extracted relationships needs to be improved. Summary of the invention

[0005] Based on this, in order to solve the technical problems in the prior art, the present invention provides a method, device and equipment for entity recognition and relationship extraction of traditional Chinese medicine medical records.

[0006] The present invention provides a method for entity recognition and relationship extraction of traditional Chinese medicine medical records, comprising:

[0007] Segment the TCM medical cases to be processed and obtain the n Characters: { c 1,..., c i ,..., c n};in, i ∈{ 1 ,..., n}, n is a positive integer;

[0008] right n The characters are embedded in the character level, the radical level and the associated word level respectively, and the characters after the character level embedding, radical level embedding and associated word level embedding are embedded in the character level, the radical level and the associated word level n Convert the characters to numeric values n Character feature vectors, n Radical feature vectors and n The feature vectors of related words;

[0009] Will n Character feature vectors, n Radical feature vectors and n The associated word feature vectors are input into a first neural network model for identifying TCM named entities to obtain the named entity recognition results of the TCM medical case to be processed;

[0010] The to-be-processed TCM medical case with the named entity recognition result is input into the second neural network model for extracting the relationship between the named entities in the TCM medical case to obtain the relationship between the named entities in the to-be-processed TCM medical case.

[0011] Furthermore, the associated word level embedding operation specifically includes:

[0012] Get a dictionary containing several named entities appearing in TCM medical records, and extract all the named entities containing the first i Characters c i , to obtain the named entity i Characters c i All relevant semantic environments in the medical records are obtained i Characters c i The associated word set W i :

[0013] According to i Characters c i At the position in the named entity, associate the word set W iThe named entities in are classified into head-related named entities, middle-related named entities, tail-related named entities and full-related named entities;

[0014] Count the number of head-related named entities, middle-related named entities, tail-related named entities, and full-related named entities in the associated word set respectively. W i The frequency of occurrence in; based on the frequency of occurrence, the associated word set W i The named entities in are weighted to extract the i Characters c i The semantic environment preference is obtained i Characters c i The associated word feature vector of :

[0015]

[0016]

[0017]

[0018]

[0019]

[0020] in, For the i Characters c i The associated word feature vector of ; B ( c i ), M ( c i ), E ( c i )and S ( c i ) are respectively i Characters c i The corresponding head-related named entity set, middle-related named entity set, tail-related named entity set and full-related named entity set; z ( w ) is a character w In the associated word set W i The frequency of occurrence in For characters w The corresponding word vector matrix; Z A set of related words W i The frequency with which all words in the dictionary appear.

[0021] Furthermore, the character-level embedding operation is to use the BERT-Ancient-Chinese model to process TCM medical records. n Characters are embedded into the character, and the obtained n Character feature vectors { ,..., ,..., }.

[0022] Furthermore, the radical level embedding operation specifically includes:

[0023] Obtain the pending TCM medical records n The radical sequence corresponding to the characters:

[0024] { r 1 ,..., r i ,..., r n};

[0025] r i = Radical ( c i )

[0026] in, Radical ( c i ) indicates the i Characters c i Perform character radical extraction operation;

[0027] Perform random embedding operation on each radical to obtain the corresponding n Radical feature vectors { ,..., ,..., }.

[0028] Furthermore, the n Character feature vectors, n Radical feature vectors and n The associated word feature vectors are input into the first neural network model for identifying TCM named entities to obtain the named entity recognition results of the TCM medical case to be processed; specifically including:

[0029] Will n Character feature vectors, n Radical feature vectors andn The associated word feature vectors are input into the bidirectional long short-term memory network layer in the first neural network model to extract the long-distance dependency relationship between the characters and obtain the medical case semantic vector;

[0030] The medical case semantic vector is input into the conditional random field layer in the first neural network model to decode the medical case semantic vector and obtain the named entity recognition result of the traditional Chinese medicine medical case to be processed.

[0031] Furthermore, the method of inputting the to-be-processed TCM medical case with the named entity recognition result into the second neural network model for extracting the relationship between the named entities in the TCM medical case to obtain the relationship between the named entities in the to-be-processed TCM medical case specifically includes:

[0032] Inputting the to-be-processed TCM medical case with the named entity recognition result into the embedding layer in the second neural network model to map each word in the to-be-processed TCM medical case into a low-dimensional vector through a random character embedding operation to obtain a medical case relationship vector;

[0033] Inputting the medical case relationship vector into the long short-term memory network layer in the second neural network model to obtain high-level association semantic features in the to-be-processed TCM medical case from the low-dimensional vector;

[0034] The high-level associated semantic features are input into the attention layer in the second neural network model to weight the high-level associated semantic features; the weighted high-level associated semantic features are classified through the classification layer in the second neural network model to obtain the relationship between the named entities in the TCM medical case to be processed.

[0035] The present invention provides an entity recognition and relationship extraction device for traditional Chinese medicine medical records, comprising:

[0036] The word segmentation module is used to segment the TCM medical cases to be processed and obtain the n Characters: { c 1 ,..., c i ,..., c n};in, i ∈{ 1 ,..., n}, n is a positive integer;

[0037] Feature extraction module is used to n The characters are embedded in the character level, the radical level and the associated word level respectively, and the characters after the character level embedding, radical level embedding and associated word level embedding are embedded in the character level, the radical level and the associated word level n Convert the characters to numeric valuesn Character feature vectors, n Radical feature vectors and n The feature vectors of related words;

[0038] Named Entity Module, which is used to n Character feature vectors, n Radical feature vectors and n The associated word feature vectors are input into a first neural network model for identifying TCM named entities to obtain the named entity recognition results of the TCM medical case to be processed;

[0039] The relationship extraction module is used to input the to-be-processed TCM medical case with the named entity recognition result into the second neural network model used to extract the relationship between the named entities in the TCM medical case, and obtain the relationship between the named entities in the to-be-processed TCM medical case.

[0040] The present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned entity recognition and relationship extraction method of traditional Chinese medicine medical records is implemented.

[0041] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:

[0042] In the entity recognition and relationship extraction method of TCM medical cases provided by the present invention, named entities associated with each character in the TCM medical case to be processed are obtained through a set of associated words, that is, all possible semantic environments of each character in the medical case are obtained; and because the multi-level meanings of TCM medical cases are often related to the positions of characters in sentences, the named entities in the associated word set are classified according to the positions of the characters in the named entities, and the usage patterns of each character in different semantic environments in the medical case are obtained; the named entities in the associated word set are weighted according to the frequencies of occurrence of the head-associated named entity, the middle-associated named entity, the tail-associated named entity and the full-associated named entity in the associated word set, which can highlight which characters are more frequent in specific positions and reveal which semantic environment each character prefers. Even when the same entity has multiple meanings, the first neural network model can obtain the possible semantic environment of the character by considering the position information of the character in the named entity to eliminate multi-level ambiguity, thereby helping the first neural network model to obtain more accurate named entity recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0044] Figure 1 Schematic flow chart of a method for entity recognition and relationship extraction of traditional Chinese medicine case records provided by the present invention;

[0045] Figure 2 Schematic diagram of the process of performing named entity matching for the character "blood" provided by the present invention;

[0046] Figure 3 Schematic diagram of the structure of the first neural network model provided by the present invention;

[0047] Figure 4 Schematic flow chart of the pipeline method provided by the present invention;

[0048] Figure 5 Schematic diagram of the structure of the second neural network model provided by the present invention;

[0049] Figure 6 Schematic diagram of the named entity relationship structure provided by the present invention;

[0050] Figure 7 Schematic diagram of an example of relationship annotation provided by the present invention;

[0051] Figure 8 All named entities related to "vertigo" provided by the present invention. Detailed implementation manners

[0052] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the specification, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0053] Most of the content in traditional Chinese medicine free text records is unstructured. Therefore, the analysis method based on structured data verification cannot be directly applied to the knowledge discovery of traditional Chinese medicine FCRs. To solve this problem, it is necessary to perform pre-structuring processing on traditional Chinese medicine FCRs. Manually structuring traditional Chinese medicine FCRs is a cumbersome, time-consuming and labor-intensive task. Therefore, there is an urgent need to study an effective method for automatically structuring traditional Chinese medicine FCRs, that is, to identify traditional Chinese medicine named entities in traditional Chinese medicine FCRs, such as the identification of traditional Chinese medicine entities such as "drugs" and "prescriptions". When performing entity recognition and relationship extraction of traditional Chinese medicine case records in the prior art, the single feature representation in traditional methods makes it difficult to obtain accurate named entity results.

[0054] In this context, the present invention provides a method for entity recognition and relationship extraction of traditional Chinese medicine medical records. The method first uses LSTM (Long Short-Term Memory) + CRF (Conditional Random Field) + comprehensive result optimal segmentation method to segment the medical record text into words or phrases. Then, 9 entities appearing in traditional Chinese medicine medical records and 12 types of relationships between the 9 entities are defined, and the segmentation results are processed by character embedding, associated word embedding and radical embedding. The embedded results are input into a network composed of a bidirectional long short-term memory network + a conditional random field for entity recognition. Then, the input medical record text after entity identification is input into a network composed of a bidirectional long short-term memory network + an attention layer for relationship extraction. Finally, a medical record knowledge graph is constructed based on the extracted entity relationships.

[0055] Example 1.

[0056] Figure 1 The process of entity recognition and relationship extraction method of TCM medical records in this embodiment is shown. Figure 1 The method is described in detail and specifically comprises the following steps:

[0057] S1: Segment the TCM medical records to be processed and obtain the n Characters: { c 1 ,..., c i ,..., c n};in, i ∈{ 1 ,..., n}, n is a positive integer. Specifically, the following steps are included:

[0058] S101: Construct a medical case segmentation dictionary.

[0059] In the field of traditional Chinese medicine, there are many specific terms and vocabulary that are essential for understanding and analyzing medical case texts. First, relevant glossaries, guidelines, and standards were collected from multiple authoritative sources, including official documents issued by the State Administration of Traditional Chinese Medicine, the Chinese Association of Traditional Chinese Medicine, and some authoritative academic journals and books on traditional Chinese medicine. The terminology source documents are shown in Table 1.

[0060] Table 1. Terminology source list

[0061]

[0062] Since the glossary, guidelines, standards and other documents have different formats, the different formats of the documents are processed as follows: (1) Excel / txt: directly read the terms according to the structure. (2) Word: match and extract using regular expressions based on the term distribution rules. (3) PDF: use OCR (Optical Character Recognition) software to convert it into word, and then process the word file. A large number of TCM terminology vocabulary was obtained through the above methods. Next, the collected terms were classified and integrated into the term dictionary.

[0063] S102: Determine the medical record segmentation method.

[0064] Word segmentation refers to dividing meaningful words by defining the boundaries between words. It is the basis of NLP (Natural Language Processing) tasks such as entity recognition and relationship extraction. The optional word segmentation methods include Jieba word segmentation method, Stanford Parser word segmentation method, LTP (Language Technology Platform) word segmentation method, THULAC word segmentation method (Tsinghua University Lexical Analysis and POS tagging System), and LSTM+CRF+comprehensive result selection word segmentation method. Among them, the LSTM+CRF+comprehensive result selection word segmentation method includes: first processing the corpus, using the word2vec operation to embed the words of the corpus, each word feature is 50 dimensions, after obtaining the word embedding, the word embedding feature is fed to the bidirectional LSTM, a linear layer is added to the output hidden layer, and then a CRF is added to obtain the model for word segmentation.

[0065] S2: Yes n The characters are embedded in the character level, the radical level and the associated word level respectively, and the characters after the character level embedding, radical level embedding and associated word level embedding are embedded in the character level, the radical level and the associated word level n Convert the characters to numeric values n Character feature vectors, n Radical feature vectors and n The feature vector of related words.

[0066] Medical record text character-level embedding operation: Bert is a neural network model stacked by multiple Transformer encoders. During training, about 15% of the input information is randomly masked. Through the masking strategy and the next sentence prediction task, the learning of the context is completed. Compared with the static word vectors constructed by word2vec, one-hot, etc., the word vectors output by the Bert model can dynamically express the word features accurately according to the specific context, and can better solve the problem of polysemy in Chinese. bert-ancient-chinese is a Chinese pre-trained model based on the Bert model. It continues to be pre-trained on the basis of bert-base-chinese combined with ancient Chinese corpus, has a larger vocabulary, and also includes more rare characters, which is more conducive to improving the performance of the model in the task of processing ancient Chinese in traditional Chinese medical records. For a text sequence of length n text sequence S ={ c 1 ,..., c i ,..., c n}; where i ∈{ 1 ,..., n}, n is a positive integer, where c i represents the i th character in the text. Using bert-ancient-chinese for character embedding, a vector sequence { ,..., ,..., } is obtained.

[0067] Medical record text radical-level embedding operation: Traditional Chinese medical record data has strong domain characteristics compared with general datasets, which is mainly reflected in the fact that entities in the traditional Chinese medicine field have distinct radical characteristics. For example, traditional Chinese medicine entities such as "Poria cocos", "Angelica dahurica", and "Atractylodes lancea" contain the radical "艹", commonly known as the "grass head", which is a manifestation of Chinese herbal medicines in the Chinese medicine system. Another example is that traditional Chinese medicine disease entities such as "scabies", "dysentery", and "consumption" contain the radical "疒", whose original meaning is disease. It is not difficult to see that radical characteristics can enhance the recognition of entities in the traditional Chinese medicine field. For a text sequence of length n text sequence S ={ c 1 ,..., c i ,..., c n}, obtain its corresponding radical sequence { r 1 ,..., ri ,..., r n},in r i = Radical ( c i ), Radical ( c i ) indicates the i Characters c i The character radical extraction operation is performed using the cnradical library in Python. r i Perform random embedding operations : , the radical vector sequence of the text is obtained as { ,..., ,..., }.

[0068] Medical case text associated word level embedding operation: The disadvantage of character-based named entity recognition methods is that vocabulary information is not fully utilized, and vocabulary boundaries usually play a crucial role in entity boundaries. This embodiment uses a lightweight dictionary matching method in TCM medical case named entity recognition. First, the text sequence S ={ c 1 ,..., c i ,..., c n} is matched with a dictionary containing several named entities that have appeared in TCM medical records, and all the entities containing the first i Characters c i Named Entity w i,j To form the i Characters c i The associated word set W i In order to preserve the segmentation information, according to i Characters c i Position in the named entity, replace each character c iAll matching words are classified into four word sets "BMES": head-related named entity set, middle-related named entity set, tail-related named entity set and full-related named entity set. The construction of these four sets is as follows, where L (Lexicon, dictionary) represents the dictionary used in this embodiment that contains several named entities appearing in TCM medical records:

[0069]

[0070]

[0071]

[0072]

[0073] in, B ( c i ), M ( c i ), E ( c i )and S ( c i ) are respectively i Characters c i The corresponding head-related named entity set, middle-related named entity set, tail-related named entity set and full-related named entity set. Figure 2 As shown, taking "Qi and blood deficiency" as an example, the character "blood" is matched with the pre-constructed dictionary to obtain the corresponding 4 word sets: B={ ,("blood deficiency")},M={ , ("Qi and blood deficiency")}, E={ , ("Qi and blood")}, S = {("None")} (if there is no matching word, "None" is used to represent the set). At the same time, this embodiment introduces pre-trained word vectors, and each word in the word set will be converted into a corresponding word vector; then all words in the four word sets are weighted normalized, and a statistical static weighting method is used here (the frequency of occurrence of the head-associated named entity, the middle-associated named entity, the tail-associated named entity, and the full-associated named entity in the associated word set is counted respectively; the named entities in the associated word set are weighted based on the frequency of occurrence), that is, the frequency of occurrence of each word in the static data. This frequency can reflect the importance of the word or the semantic environment preference to a certain extent, and the weighting method is:

[0074]

[0075]

[0076]

[0077]

[0078] in, For the i Characters c i The set of all associated named entities / “S” word set that matches; z(w) is the word w in the “S” word set in static data, i.e., the associated word set W i The frequency of occurrence in statistics; Z is the set of related words W i The frequency of all words in the dictionary; For lookup characters w Embedded word vector matrix; concatenate the representations of the four word sets into a one-dimensional feature to obtain the associated word enhanced vector of the character: .

[0079] S3: n Character feature vectors, n Radical feature vectors and n The associated word feature vectors are input into the first neural network model for identifying TCM named entities to obtain the named entity recognition results of the TCM medical case to be processed.

[0080] The method for medical record entity recognition in this embodiment is based on a deep neural network model. Deep learning does not require complex feature engineering and can automatically discover information from input and output and learn the representation of information. The network structure of the first neural network model is as follows: Figure 3 As shown in the figure, it consists of three parts: an improved embedding layer, an encoding layer, and a decoding layer. The embedding layer converts the medical record text into an embedding vector, which includes three modules: character embedding, associated word embedding, and radical embedding. The encoding layer uses a bidirectional long short-term memory network (forward LSTM and backward LSTM) to encode the embedding vector into a vector containing contextual semantic information. The decoding layer uses a conditional random field to decode the encoded vector into the corresponding label. represents the output vector of the t-th unit of the forward LSTM, Represents the output vector of the t-th unit of the backward LSTM.

[0081] The above three types of vectors are concatenated to obtain =[ ; ], the embedding vector of the text sequence is expressed as { , ,……,}, the embedding vector is the input of the encoding layer.

[0082] S4: Inputting the to-be-processed TCM medical case with the named entity recognition result into the second neural network model for extracting the relationship between the named entities in the TCM medical case to obtain the relationship between the named entities in the to-be-processed TCM medical case.

[0083] The traditional approach to relation extraction is the pipeline method. The process of the pipeline method is as follows: Figure 4 As shown in FIG. 1 , the method is performed in two steps. The first step is to identify named entities in the sentence text through the entity recognition module, and the second step is to sequentially form candidate entity pairs with the entities obtained by the entity recognition module and use them together with the sentence text as input for subsequent relationship classification.

[0084] The network structure of the second neural network model is as follows Figure 5 As shown in the figure, the model uses the neural attention mechanism of the bidirectional long short-term memory network to capture the most important semantic information in the sentence. It can automatically focus on the words that have a decisive influence on classification and capture the most important semantic information in the sentence without using additional vocabulary resources. It contains five components:

[0085] ⑴ Input layer: The model’s input sentence {X1,X2,...,X T}, such as the sentence "Chrysanthemum calms the liver and extinguishes wind" becomes "< >Chrysanthemum< / >< > Calming the Liver and Extinguishing Wind< / >".

[0086] ⑵ Embedding layer: Each word is mapped to a low-dimensional vector {e1,e2,...,e T}.

[0087] ⑶ Bidirectional LSTM layer: Use the bidirectional LSTM layer to obtain high-level semantic features {h1,h2,...,h T}, where represents the output vector of the t-th unit of the forward LSTM, Represents the output vector of the t-th unit of the backward LSTM.

[0088] ⑷Attention layer: Generate a weight vector, and merge the character-level features at each moment into a sentence-level feature vector by multiplying the weight vector.

[0089] ⑸Output layer: The sentence-level feature vector is finally used for relation classification.

[0090] The first neural network model was used as the named entity recognition module and the second neural network model was used as the relationship classification module to conduct a pipeline extraction experiment of medical case text. The experiment was conducted in the Windows system Python3.6.15 and pytorch1.10.2 environment. The maximum sentence length of the data was set to 200, batchsize was 8, epoch was 20, dropout was 0.5, and the learning rate was 1e -3 , the character embedding dimension is 100, and the bidirectional LSTM hidden layer dimension is 128.

[0091] Although the pipeline-based relation extraction method is easy to model and flexible in operation, it has the following problems.

[0092] (1) Error accumulation: Named entities that are incorrectly identified in the named entity recognition module will be input into the relationship classification module for relationship classification, and the relationship classification module cannot identify and correct the incorrect entities, resulting in a decrease in the quality of the relationship triples obtained by the pipeline system. (2) Lack of interaction: In the pipeline method, the named entity recognition module and the relationship classification module are trained and predicted separately, lacking interactive information. For example, the named entity recognition module cannot use the relationship types contained in the text to enhance the entity recognition effect, and the relationship classification module rarely uses entity types to assist in the classification of relationships. (3) Redundant information: The pipeline method combines the named entities recognized by the named entity recognition module into entity pairs, which will produce entity pairs that do not contain any predefined relationships (i.e., redundant entity pairs), increasing the difficulty of relationship classification.

[0093] In order to solve the problems existing in the pipeline mode, this embodiment adopts a joint entity relationship extraction method, in which different subtasks interact with each other, make full use of the information in the subtasks, and further improve the model performance. The CasRel model is an advanced entity relationship extraction model based on joint decoding, which can simultaneously extract entities and relationships of medical case texts to form triples such as <entity 1, relationship, entity 2>. The CasRel model mainly consists of three parts: encoding end, decoding end, and label assignment layer. At the encoding end, CasRel uses Bert to deeply encode the input text to obtain contextual semantic information and further characterize the characters or words. This step is crucial for understanding the relationship between words in a sentence and accurately extracting target information. Next is the decoding end, where CasRel operates in a cascade binary annotation framework. In this framework, the model first predicts the subject, then the predicate and the object, which can effectively extract the entity relationship in the sentence. Finally, the label assignment layer is responsible for mapping the predicted entities and relationships to a predefined label set.

[0094] based on Figure 1The entity recognition and relationship extraction method of TCM medical records shown in the present invention, by combining character vectors, associated word vectors and radical vectors, the named entity recognition system can more comprehensively capture the true meaning of the named entity; specifically, the character vector provides the basic feature representation of each Chinese character, the associated word vector can distinguish the different contexts and meanings of polysemous words, and the radical vector helps to process the structural information of Chinese characters, helping the system to better understand the association and combination between Chinese characters; the above-mentioned vectors are introduced into a bidirectional long short-term memory network to capture the long-distance dependency between different words, so as to more comprehensively understand the semantic information and connection of words in the medical records. By processing from two dimensions, namely, the embedding operation of local words and the extraction of global semantic relationships, a more accurate recognition result of named entities is obtained, thereby obtaining a more accurate relationship.

[0095] Example 2.

[0096] This embodiment provides a process for constructing a TCM medical case corpus dataset for training a first neural network model and a second neural network model.

[0097] In the semantic network framework of the TCM language system, nodes represent entities in TCM, such as diseases, symptoms, drugs, treatment methods, etc. Edges represent the relationships between these entities, such as causal relationships, interaction relationships, etc. Each node and edge can be assigned one or more attributes to further describe its characteristics and features. The main advantage of this framework is that it can represent the complex knowledge structure of TCM in an intuitive and easy-to-understand way. By analyzing and studying this network, we can better understand the theoretical system of TCM, discover new knowledge and laws, and improve the therapeutic effect of TCM. In addition, the semantic network framework of the TCM language system can also be used for the retrieval and recommendation of TCM information. By comparing user queries and knowledge nodes in the network, the most relevant information can be found to provide personalized services to users.

[0098] Under the guidance of TCM clinicians, we read a large number of medical case texts, analyzed the organization of medical case texts, and learned from TCM clinical experts how doctors write medical cases in clinical work. We summarized the diagnosis and treatment process of doctors to patients as observing the patient's physical condition (including general symptoms and tongue, pulse and other physical signs), exploring the internal causes and pathogenesis, summarizing the syndromes, adopting corresponding treatment methods for the syndromes, and using specific drugs (traditional Chinese medicine or prescriptions, etc.) according to the treatment methods. Therefore, the following 9 types of named entities are mainly studied in TCM medical cases.

[0099] ① Disease: Under the influence of pathogenic factors such as the six evils, seven emotions, heredity, diet, work and rest, and trauma, the relationship between the body and the environment is disturbed, the yin and yang, qi and blood in the body are disordered, the physiological state is destroyed, abnormal changes in functional morphology and mental activity occur, and are reflected in the pathological process of the struggle between evil and righteousness with certain symptoms and signs.

[0100] ② Tongue image: Judge the health status of the human body by observing the shape, color, moisture, tongue coating and other characteristics of the tongue.

[0101] ③ Pulse: including the pulse's rhythm, speed, strength, width, depth, smoothness, and other characteristics, which reflect the body's yin and yang balance, the flow of qi and blood, and the functional state of the internal organs. By carefully observing and feeling the changes in the pulse, Chinese medicine doctors can determine the patient's constitution, condition, and cause of the disease.

[0102] ④Symptoms: abnormal state of the body due to disease, including various abnormal sensations of the patient himself and various abnormal manifestations perceived by the doctor's sensory organs.

[0103] ⑤ Etiology and pathogenesis: Etiology refers to the various causes that lead to physiological dysfunction and disease in the human body, and pathogenesis refers to the internal laws and processes of disease occurrence, development and change.

[0104] ⑥Syndrome: The external symptoms of syndrome, which are the reaction state of the organic connection between the location, cause, nature, course of disease and the strength of the body's disease resistance at a certain stage in the disease process, manifested as clinically observable symptoms.

[0105] ⑦ Treatment method: The method of treating diseases under the guidance of treatment principles.

[0106] ⑧Prescription: A Chinese medicine prescription composed of several medicines based on the compatibility principles and the physician's clinical experience.

[0107] ⑨ Traditional Chinese medicine: in this embodiment, it specifically refers to Chinese medicinal materials, which are raw medicinal materials formed by the medicinal parts of medicinal plants, animals, and minerals being harvested and initially processed at the place of production.

[0108] Based on these nine entities and their semantic relationships in medical records, we define Figure 6 As shown in the named entity relationship structure diagram, there are 12 types of entity relationships, and the specific descriptions are shown in Table 2.

[0109] Table 2 Entity relationship definition

[0110]

[0111] The collected medical record PDF data was converted into a word document through OCR text recognition software, and the converted data was manually proofread to correct incorrect characters and text structure, and each medical record was saved as a txt file.

[0112] Considering that the domain-specific medical dictionary is not perfect enough, the diversity of the expressions of traditional Chinese medicine knowledge by various veteran traditional Chinese medicine doctors, and the deviation in the understanding of medical record texts by annotators, in order to ensure the acquisition of high-quality medical record corpora, the following specifications need to be followed for annotation: ① Non-overlapping, for example, "headache" cannot be marked as both a disease and a symptom at the same time. ② Non-nesting, for example, after "Tianma Gouteng Decoction" is marked as a prescription, it is not allowed to mark Tianma and Gouteng in it as traditional Chinese medicines anymore. ③ For traditional Chinese medicine entities with processing methods in front, they need to be marked together. For example, "prepared rehmannia root", "roasted gastrodia elata", "prepared tortoise plastron" are marked as traditional Chinese medicines instead of only marking "rehmannia root", "gastrodia elata", "tortoise plastron" in them as traditional Chinese medicines. For details in processing such as "autumn stone decocted in water and poured in", only "autumn stone" needs to be marked as a traditional Chinese medicine. ④ For symptoms that have improved such as "dizziness has stopped, tinnitus has greatly reduced", only "dizziness" and "tinnitus" need to be marked as symptoms. ⑤ The dialectical classification of diseases such as "wind-heat headache" and "phlegm syncope headache" is marked as syndromes. Based on this specification, an entity recognition corpus dataset is constructed, and this corpus dataset is used to train the network model.

[0113] The process of annotating the entity recognition corpus needs to be completed with the help of annotation tools. Currently, common annotation tools include "Elf Annotation Assistant", "Baidu EasyDL", "Label Studio", etc. In this embodiment, the "Semantic Annotation System for Traditional Chinese Medicine Literature" developed by the Institute of Information of the China Academy of Chinese Medical Sciences is used for corpus annotation. This annotation tool can automatically match and annotate the corpus text according to the uploaded term dictionary, which is a function that other annotation tools do not have. However, the quality of the automatically annotated corpus is not high and needs to be manually reviewed and corrected. To ensure the quality of the corpus, it is necessary to evaluate the consistency of the results of two annotators.

[0114]

[0115]

[0116]

[0117] Among them, , respectively represent two annotators, represents the number of entities with consistent annotations by the two annotators, , respectively represent the number of entities annotated by the two annotators themselves, Represents the consistency evaluation index. In this embodiment, a pre-annotation iteration method is adopted to ensure obtaining a high-quality annotated corpus, that is, before starting the formal annotation, multiple rounds of pre-annotations are carried out to ensure the consistency of the annotation of medical record entities by two annotators. The specific approach is to randomly select 10 medical records in each round and hand them over to two annotators for annotation respectively. After the pre-annotation is completed, the inconsistent annotations are discussed, so as to continuously improve the annotation rules and conduct a consistency evaluation on the annotation of this round. If the consistency evaluation index does not reach 0.9, it is considered that the consistency of the two annotators for the annotation of medical records is not high, and it is necessary to enter the next round of pre-annotation process. Otherwise, it is considered that the two annotators have high consistency, and at this time, the formal annotation of all medical records can be started. It is considered that the two annotators have high consistency, and the formal annotation of all medical records is started, and finally a medical record entity annotation corpus with high quality consistency is obtained, and on this basis, relationship annotation is carried out to obtain an entity relationship annotation corpus.

[0118] The annotated medical record dataset is converted into the "BMESO" tag format. If X is the entity category, the tag B-X corresponds to the starting character of entity X, the tag M-X corresponds to the middle character of the entity, the tag E-X corresponds to the ending character of the entity, the tag S-X corresponds to a single-character entity, and the tag O corresponds to non-entity characters. For example, the tag of "leprosy" is "B-disease M-disease E-disease". Each medical record is segmented at the period, and a number of annotated corpus are obtained, which are randomly divided into a training set, a validation set and a test set according to 6:2:2.

[0119] On the basis of completing the construction of the medical record entity recognition dataset, this embodiment further screens out sentences that contain two entities and can clearly express the semantic relationships defined in Table 2 for manual annotation. This process aims to construct a medical record relationship extraction dataset. However, due to the significant differences between the expression characteristics of classical Chinese in ancient medical record literature and modern vernacular Chinese, especially its elliptical and concise language style, it is challenging to annotate the relationships between entities, and the amount of data that can be annotated is limited. Given the small scale of the dataset, the directly trained model is difficult to accurately capture the complex relationships between entities.

[0120] To overcome this problem, this embodiment adopts a data augmentation strategy. 704 Chinese medical ancient book texts in txt format are downloaded from GitHub to expand and enrich the data sources for relationship annotation. These additional data not only help to improve the training effect of the model, but also enhance the performance and accuracy of the model in practical applications. Potential relationship sentences containing keywords such as "by", "use", "then" are screened out from these ancient book texts, and the annotation work is carried out using the Baidu easydata data annotation platform. The newly annotated relationship data is merged with the previous medical record relationship data, and finally a medical record relationship extraction dataset is formed. Figure 7This is a specific example of a relationship annotation provided in this embodiment. In order to effectively train and evaluate the performance of the relationship extraction model, the data set is randomly divided into three parts: a training set, a validation set, and a test set, with a ratio of 6:2:2 respectively.

[0121] The BRL neural network model was trained using the training set data, and the model converged by minimizing the loss function. To prevent overfitting, the model with the best effect on the validation set was saved during the training process. The training was implemented using Python 3.6.15 and the package pytorch 1.10.2 in the Windows system environment. Among them, the maximum sentence length of the data was set to 200, the batchsize (the number of samples selected for one training) was 8, the epoch (a complete cycle of the entire training set in the neural network training) was 20, and the dropout (random inactivation ratio of neurons) was 0.5. The model set the layered learning rate: the pre-trained model layer learning rate was 1e -5 , the learning rate of other layers is 1e -3 .

[0122] Example 3.

[0123] A medical case knowledge graph was constructed. Neo4j is a high-performance NoSQL graph database. Unlike traditional relational databases, it stores data in the form of nodes and relationships, forming a network or graph structure. To facilitate user interaction with the database, Neo4j provides an easy-to-use web interface. Users can perform database configuration, data input, and query operations by accessing the web server on the local host (usually located at http: / / localhost:7474 / ). In addition, the interface also supports data visualization, which helps users intuitively understand and analyze the structure of graph data.

[0124] The specific process of using Neo4j to build a knowledge graph is as follows:

[0125] (1) Data preparation: Extract entity data and relationship data from manually annotated entity recognition datasets and relationship extraction datasets. Using the entity recognition model BRL and the relationship extraction model CasRel, intelligently extract entity and relationship triples from 115,428 sentences in the text to be extracted from ancient Chinese medical records and other texts. Manual review is used to remove erroneous data, and combined with the data in the annotated text, 9 entity node files and 12 relationship triple files in Excel format are formed. Examples of prescription entity files are shown in Table 3, and examples of disease-prescription relationship triple files are shown in Table 4.

[0126] Table 3 Prescription entity files

[0127]

[0128] Table 4. Disease-prescription relationship triplet file

[0129]

[0130] (2) Import data: Use Python's py2neo package to interact with the Neo4j graph database programmatically. First, establish a connection with the Neo4j database. You need to provide the database server address, username, and password. After the connection is successful, programmatically read the entity node file and relationship triple file in (1), and use the interface and create method provided by the py2neo package to create nodes and relationships to complete the construction of the knowledge graph.

[0131] (3) Query data: After the data is successfully imported, use the Cypher query language to extract the required information from the database. For example, use the following MATCH statement to query all entities related to the specific disease "vertigo":

[0132] MATCH p=(n:`disease`{name:"vertigo"})-[r]-() RETURN p.

[0133] The query results are as follows Figure 8 As shown in the figure, vertigo forms a "symptom" relationship with a variety of symptoms, among which there are more symptoms that are biased towards the upper part of the body. Among them, headache, palpitations, insomnia, sallow complexion, and tinnitus are obvious upper-part heart and brain system diseases. It can be seen that among the symptoms with strong correlations with vertigo, the heart, brain, and liver have strong internal organs. This is highly consistent with the current principle of "brain and heart are the same disease" and "brain and heart are treated the same way" for heart and brain diseases. For these symptoms, the prescriptions used mainly include kidney-tonifying, Chuanxiong Powder, Zuojin Pills, and Yiyuan Powder, which use methods such as kidney-tonifying, liver-clearing, blood-activating, and dampness-removing treatments, reflecting the treatment principle of treating different diseases with the same treatment and treating the same disease with different treatments when symptoms of the heart, liver, and brain occur.

[0134] By visualizing the triples of medical case knowledge in the form of knowledge graphs and querying and analyzing the connections between related entities, we can not only discover more valuable treatment methods and medical theories, but also provide references and inspirations for modern TCM clinics. This will provide practical tools and methods for TCM research. This approach will help promote the standardization of TCM information, improve research efficiency, and provide support for the inheritance and development of TCM knowledge.

[0135] The above is a method for entity recognition and relationship extraction of TCM medical records provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding device for entity recognition and relationship extraction of TCM medical records, including:

[0136] The word segmentation module is used to segment the TCM medical cases to be processed and obtain the n Characters: { c 1 ,..., c i ,..., c n};in, i ∈{ 1 ,..., n}, n Is a positive integer.

[0137] Feature extraction module is used to n The characters are embedded in the character level, the radical level and the associated word level respectively to embed n Convert the characters to numeric values n Character feature vectors, n Radical feature vectors and n The feature vector of related words.

[0138] Among them, i Characters c i The steps for performing the word-level embedding operation include:

[0139] Get a dictionary containing several named entities appearing in TCM medical records, and extract all the named entities containing the first i Characters c i , to obtain the named entity i Characters c i All relevant semantic environments in the medical records are obtained i Characters c i The associated word set W i .

[0140] According to i Characters c i At the position in the named entity, associate the word set W i The named entities in the are classified into head-related named entities, middle-related named entities, tail-related named entities and full-related named entities to obtain the first i Characters c i Usage patterns in different semantic contexts in medical records.

[0141] Count the number of head-related named entities, middle-related named entities, tail-related named entities, and full-related named entities in the associated word set respectively. W i The frequency of occurrence in; based on the frequency of occurrence, the associated word set W i The named entities in are weighted to extract the i Characters c i The semantic environment preference is obtained i Characters c i The associated word feature vector of .

[0142] Named Entity Module, which is used to n Character feature vectors, n Radical feature vectors and n The associated word feature vectors are input into the first neural network model for identifying TCM named entities to obtain the named entity recognition results of the TCM medical case to be processed.

[0143] The relationship extraction module is used to input the to-be-processed TCM medical case with the named entity recognition result into the second neural network model used to extract the relationship between the named entities in the TCM medical case, and obtain the relationship between the named entities in the to-be-processed TCM medical case.

[0144] For the specific definition of the entity recognition and relationship extraction device for TCM medical records, please refer to the definition of the entity recognition and relationship extraction method for TCM medical records above, which will not be repeated here. Each module in the above-mentioned entity recognition and relationship extraction device for TCM medical records can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0145] The present invention also provides a computer device structure. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Provides entity recognition and relationship extraction methods for traditional Chinese medicine medical records.

[0146] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0147] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A method for entity recognition and relationship extraction of traditional Chinese medicine medical records, characterized in that: include: Perform word segmentation on the TCM medical case to be processed and obtain n characters in the TCM medical case to be processed: {c1,...,c i ,...,c n }; where i∈{1,...,n}, n is a positive integer; Performing character-level embedding operations, radical-level embedding operations, and associated-word-level embedding operations on n characters respectively, and converting the n characters after the character-level embedding operations, radical-level embedding operations, and associated-word-level embedding operations into n character feature vectors, n radical feature vectors, and n associated-word feature vectors in numerical form respectively; Inputting n character feature vectors, n radical feature vectors and n associated word feature vectors into a first neural network model for identifying TCM named entities, and obtaining a named entity recognition result of the TCM medical case to be processed; The to-be-processed TCM medical case with the named entity recognition result is input into the second neural network model for extracting the relationship between the named entities in the TCM medical case, and the relationship between the named entities in the to-be-processed TCM medical case is obtained, specifically including: Inputting the to-be-processed TCM medical case with the named entity recognition result into the embedding layer in the second neural network model to map each word in the to-be-processed TCM medical case into a low-dimensional vector through a random character embedding operation to obtain a medical case relationship vector; Inputting the medical case relationship vector into the long short-term memory network layer in the second neural network model to obtain high-level association semantic features in the to-be-processed TCM medical case from the low-dimensional vector; The high-level associated semantic features are input into the attention layer in the second neural network model to weight the high-level associated semantic features; the weighted high-level associated semantic features are classified through the classification layer in the second neural network model to obtain the relationship between the named entities in the TCM medical case to be processed.

2. The entity recognition and relationship extraction method of TCM medical records as claimed in claim 1, characterized in that: The associated word level embedding operation specifically includes: Get a dictionary containing several named entities appearing in TCM medical records, and extract all the named entities containing the i-th character c from the dictionary. i Named entity to get the i-th character c i In all relevant semantic environments in the medical records, get the i-th character c i The associated word set Wc i : According to the i-th character c i The position in the named entity, the associated word set Wc i The named entities in are classified into head-related named entities, middle-related named entities, tail-related named entities and full-related named entities; Count the number of head-related named entities, middle-related named entities, tail-related named entities, and full-related named entities in the associated word set Wc respectively. i The frequency of occurrence in the associated word set Wc i The named entities in are weighted to extract the i-th character c i The semantic environment preference of the i-th character c is obtained i The associated word feature vector of : in, is the i-th character c i The associated word feature vector of B(c i )、M(c i )、E(c i ) and S(c i ) are the i-th character c i The corresponding head associated named entity set, middle associated named entity set, tail associated named entity set and full associated named entity set; z(w) is the character w in the associated word set Wc i The frequency of occurrence in w (w) is the word vector matrix corresponding to character w; Z is the associated word set Wc i The frequency with which all words in the dictionary appear.

3. The entity recognition and relationship extraction method of TCM medical records as claimed in claim 1, characterized in that: The character level embedding operation is to use the BERT-Ancient-Chinese model to perform character embedding operations on n characters in the TCM medical records to obtain n character feature vectors 4. The entity recognition and relationship extraction method of TCM medical records as claimed in claim 1, characterized in that: The radical level embedding operation specifically includes: Get the radical sequence corresponding to n characters in the TCM medical records to be processed: {r1,...,r i ,...,r n}； r i =Radical(c i ) Among them, Radical(c i ) means for the i-th character c i Perform character radical extraction operation; Perform random embedding operations on each radical to obtain the corresponding n radical feature vectors 5. The entity recognition and relationship extraction method of TCM medical records as claimed in claim 1, characterized in that: The method of inputting n character feature vectors, n radical feature vectors and n associated word feature vectors into a first neural network model for identifying TCM named entities to obtain a named entity recognition result of a TCM medical case to be processed specifically includes: Inputting n character feature vectors, n radical feature vectors and n associated word feature vectors into the bidirectional long short-term memory network layer in the first neural network model to extract the long-distance dependency relationship between the characters and obtain the medical case semantic vector; The medical case semantic vector is input into the conditional random field layer in the first neural network model to decode the medical case semantic vector and obtain the named entity recognition result of the traditional Chinese medicine medical case to be processed.

6. A device for entity recognition and relationship extraction of traditional Chinese medicine medical records, characterized in that: include: The word segmentation module is used to segment the TCM medical records to be processed and obtain n characters in the TCM medical records to be processed: {c1,...,c i ,...,c n }; where i∈{1,...,n}, n is a positive integer; A feature extraction module is used to perform character-level embedding operations, radical-level embedding operations, and associated-word-level embedding operations on n characters, and convert the n characters after the character-level embedding operations, radical-level embedding operations, and associated-word-level embedding operations into n character feature vectors, n radical feature vectors, and n associated-word feature vectors in numerical form, respectively; A named entity module, used for inputting n character feature vectors, n radical feature vectors and n associated word feature vectors into a first neural network model for identifying TCM named entities, to obtain a named entity recognition result of a TCM medical case to be processed; The relationship extraction module is used to input the to-be-processed TCM medical case with the named entity recognition result into the second neural network model for extracting the relationship between the named entities in the TCM medical case to obtain the relationship between the named entities in the to-be-processed TCM medical case, specifically including: Inputting the to-be-processed TCM medical case with the named entity recognition result into the embedding layer in the second neural network model to map each word in the to-be-processed TCM medical case into a low-dimensional vector through a random character embedding operation to obtain a medical case relationship vector; Inputting the medical case relationship vector into the long short-term memory network layer in the second neural network model to obtain high-level association semantic features in the to-be-processed TCM medical case from the low-dimensional vector; The high-level associated semantic features are input into the attention layer in the second neural network model to weight the high-level associated semantic features; the weighted high-level associated semantic features are classified through the classification layer in the second neural network model to obtain the relationship between the named entities in the TCM medical case to be processed.

7. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 5 is implemented.

Images