A Structural Information Enhancement Method and System for Entity Relationship Classification in Chinese Medical Text

Abstract

This invention belongs to the field of natural language processing relation classification / extraction technology, specifically providing a method and system for entity relation classification in Chinese medical text with enhanced structural information. The method includes: acquiring a training corpus; expanding the training corpus to obtain an expanded corpus; feeding the expanded corpus into a pre-trained Chinese language model BERT to obtain text sequence representations; feeding the expanded corpus into a bidirectional long short-term memory network (BiLSTM) to output text sequence representations incorporating structural information; concatenating entity referential vectors and entity type vectors to obtain the final vector representation of a given entity. For any two entities i and j, the entity referential vectors and entity type vectors are concatenated and then fed into a fully connected layer for relation classification. By expanding the corpus in terms of the representation type and structural composition of entity referentials, the problem of scarce medical text corpus is solved, and the diversity of the rich training corpus improves the generalization performance of the model.

Description

Technical Field

[0001] This invention relates to the field of natural language processing relation classification / extraction technology, and more specifically, to a method and system for classifying entity relations in Chinese medical text with enhanced structural information. Background Technology

[0002] One approach to improving the performance of entity relation classification in Chinese medical text is to incorporate textual structural information, such as dependency syntax or semantic dependency relations, into the model. The hope is that the model can learn this structural information to help it classify the relational categories between entities. However, existing methods have the following problems: (1) Most of the work uses an external parser to generate the structural information of the text and then integrates this information into the model. This pipeline architecture may cause errors to propagate to later processing stages, affecting the performance of the model.

[0003] (2) The word segmentation results of the structure parser are not in one-to-one correspondence with the entity references, and it is uncertain whether such structural information is helpful for classifying the relationships between entities.

[0004] For example: Text: "Ischemic stroke @ treatment center uses MRI instead of CT as the preferred imaging modality" Entity relation triple: <Ischemic stroke, imaging modality, MRI> Problem: The entity "ischemic stroke" was split into two words during dependency parsing.

[0005] (3) Due to the special nature of medical texts, there are few labeled corpora, which makes it difficult to meet the training needs of deep learning models. Summary of the Invention

[0006] This invention addresses the technical problem of scarce medical text corpora in existing technologies.

[0007] This invention provides a structural information-enhanced method for classifying entity relationships in Chinese medical texts, comprising the following steps: S1, Obtain the training corpus; S2, the expanded corpus is obtained by expanding the training corpus; S3, The expanded corpus is fed into the BERT language model pre-trained in Chinese to obtain the vector representation of each character. Permutations and combinations yield text sequence representations ; S4, representing the text sequence After being fed into a bidirectional long short-term memory network (BiLSTM), the output is a text sequence representation that incorporates structural information. S5, Perform vector representation of entity types in the text to obtain entity type vectors. ; S6, Perform vector representation on entity references to obtain entity reference vectors. ; S7 concatenates the entity reference vector and the entity type vector to obtain the final vector representation of a certain entity: ; S8. For any two entities i and j, the entity referential vector and entity type vector are concatenated and then fed into the fully connected layer for relation classification.

[0008] Preferably, S2 specifically includes: Dependency structure parsing is performed on the text. If a certain entity reference is word-segmented, the entity reference is split according to the dependency relationship type between the word groups, the entity type and relation type are copied, and new triplet samples are generated, or Fuzzy matching of labeled entities in the corpus was performed using a medical terminology dictionary, and some Chinese / English expressions of medical terms were replaced with English / Chinese to generate a new text corpus.

[0009] Preferably, S3 specifically includes: S31 uses a self-attention mechanism to model the directional correlation between each pair of character representations, where the inductive probability of the text structure is modeled as the correlation summation of all edges; S32 uses softmax for normalization; S33, Using the matrix tree theorem to calculate marginal probability distributions The text structure is inductively calculated by sampling from independent distributions, and a text sequence representation with fused structural information is output. This represents the vector representation of the i-th character output after passing through a BiLSTM with a self-attention mechanism.

[0010] Preferably, S3 specifically includes: A language model is built using an inductive structure learning neural network with an attention fusion mechanism. The training corpus is used to study the connections between entity words, between entity words and non-entity words, and between non-entity words and non-entity words.

[0011] Preferably, S5 specifically includes: Constructing a network of encoded entity types The network For one The matrix is ​​given by q, where q represents the number of entity types and d represents the vector dimension. For the k-th entity type The vector representation is the entity type vector.

[0012] Preferably, S6 specifically includes: The average of each dimension of the vector representation containing the fused structural information of characters in the entity reference is taken: That is, entity reference. The semantic representation vector that integrates textual structural information is the entity reference vector.

[0013] Preferably, after S8, the method further includes: Predict its relation type label using the softmax function: in, Let be the final vector representation of the i-th entity. Let be the final vector representation of the j-th entity. This is the parameter matrix of the fully connected layer. is the bias vector of the fully connected layer, and f is the one-hot vector of the predicted label output.

[0014] This invention also provides a structure-enhanced Chinese medical text entity relation classification system, the system being used to implement a structure-enhanced Chinese medical text entity relation classification method, comprising: The corpus acquisition module is used to acquire the corpus training set; The corpus augmentation module is used to augment the training corpus to obtain an augmented corpus. The text sequence representation module is used to feed the expanded corpus into the BERT language model pre-trained in Chinese to obtain a vector representation of each character. Permutations and combinations yield text sequence representations ; The structural information fusion module is used to represent text sequences. The text sequence representation, which is fed into a Bidirectional Long Short-Term Memory (BiLSTM) network, outputs fused structural information. ; The entity type vector representation module is used to represent entity types in text using vectors to obtain entity type vectors. ; The entity reference vector representation module is used to perform vector representation on entity references to obtain entity reference vectors. ; The concatenation module is used to concatenate the entity reference vector and the entity type vector to obtain the final vector representation of a given entity. ; The relation classification module is used to classify the relation between any two entities i and j by concatenating their entity reference vectors and entity type vectors and then feeding them into the fully connected layer.

[0015] The present invention also provides an electronic device, including a memory and a processor, wherein the processor is used to implement a structure-enhanced Chinese medical text entity relation classification method when executing a computer management program stored in the memory.

[0016] The present invention also provides a computer-readable storage medium storing a computer management program thereon, wherein the computer management program, when executed by a processor, implements the steps of a structure-enhanced Chinese medical text entity relation classification method.

[0017] Beneficial Effects: The present invention provides a structural information-enhanced Chinese medical text entity relation classification method and system, wherein the method includes: acquiring a training corpus; expanding the training corpus to obtain an expanded corpus; feeding the expanded corpus into a pre-trained Chinese language model BERT to obtain vector representations of each character, and arranging and combining them to obtain text sequence representations; feeding the text sequence representations into a bidirectional long short-term memory network BiLSTM to output text sequence representations fused with structural information; concatenating the entity referential vector and the entity type vector as the final vector representation of a certain entity; for any two entities i and j, concatenating their entity referential vectors and entity type vectors and feeding them into a fully connected layer for relation classification. By encoding the structural information of the text using an inductive method, the error propagation caused by using an external structure parser to pre-analyze the text structure as model input is eliminated. By expanding the corpus with the help of a terminology dictionary and a dependency structure analyzer, the corpus is expanded in terms of the expression type and structural composition of entity references, which solves the problem of scarce medical text corpus. The rich diversity of training corpus improves the generalization performance of the model. Attached Figure Description

[0018] Figure 1 Flowchart of the structure information-enhanced Chinese medical text entity relationship classification method provided by this invention; Figure 2 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention; Figure 3 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. Detailed Implementation

[0019] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0020] Figure 1 The structural information-enhanced Chinese medical text entity relation classification method provided by this invention specifically includes: S1, Obtain the training corpus; S2, the expanded corpus is obtained by expanding the training corpus; Expansion Method 1: Perform dependency structure parsing on the text. If a certain entity reference is word-segmented, split the entity reference according to the dependency relationship type between the phrases, copy the entity type and relation type, and generate a new triplet sample. For example, in the above example of the entity-relation triplet: <Ischemic stroke, imaging examination, MRI>, the relationship between "ischemic" and "stroke" is ATT (attributive-headed relation), then the expanded triplet is: <stroke, imaging examination, MRI>.

[0021] Expansion Method Two: Use a medical terminology dictionary to perform fuzzy matching on the labeled entities in the corpus, and replace the Chinese / English expressions of some medical professional terms with English / Chinese to generate a new text corpus.

[0022] S3, The expanded corpus is fed into the BERT language model pre-trained in Chinese to obtain the vector representation of each character. Permutations and combinations yield text sequence representations Specifically, the expanded corpus is fed into the BERT (Bidirectional Encoder Representation from Transformers) language model, which is pre-trained in Chinese, to obtain the embedding of each character in the text and thus the text sequence representation. L represents the number of BERT layers.

[0023] S4, representing the text sequence After being fed into a bidirectional long short-term memory network (BiLSTM), the output is a text sequence representation that incorporates structural information. Specifically, representing text sequences The vector representation of each character is fed into a bidirectional long short-term memory (BiLSTM) network. Decomposed into two parts For the semantic representation vector of a character, This is a vector representing the structural information associated with the character.

[0024] The directed correlation between each pair of character representations is modeled using a self-attention mechanism. The inductive probability of text structure A is modeled as the correlation sum of all edges, then normalized using softmax, and the marginal probability distribution is calculated using the matrix tree theorem. By sampling from independent distributions and performing inductive calculations on the text structure, a text sequence representation that integrates structural information is finally output. , Let i be the text structure of the i-th character.

[0025] S5, Perform vector representation of entity types in the text to obtain entity type vectors. ; Specifically: Constructing a network that encodes entity types The network is a The matrix is ​​q, where q is the number of entity types and d is the vector dimension. For the k-th entity type The vector representation is the entity type vector.

[0026] S6, Perform vector representation on entity references to obtain entity reference vectors. Specifically, the average of each dimension of the vector representation containing the fused structural information of characters in the entity reference is taken: That is, entity reference. The semantic representation vector that integrates textual structural information is the entity reference vector.

[0027] S7 concatenates the entity reference vector and the entity type vector to obtain the final vector representation of a certain entity: ; S8. For any two entities i and j, the entity referential vector and entity type vector are concatenated and then fed into the fully connected layer for relation classification.

[0028] Specifically, for any two entities i and j, the entity label vector and entity type vector are concatenated and then fed into a fully connected layer for relation classification. The softmax function is then used to predict their relation type labels. in, Let be the final vector representation of the i-th entity. This is the final vector representation of the j-th entity.

[0029] S9 uses the cross-entropy loss function to calculate the model loss and performs gradient backpropagation and parameter optimization.

[0030] Instead of using pre-parsed structures from an external structure parser as model input, this model employs an inductive structure learning neural network with an attention-integrated mechanism. It learns inductively the structures between entity words, between entity words and non-entity words, and between non-entity words and non-entity words from the training corpus, avoiding the error propagation issues inherent in existing pipeline-style models that integrate structural information. Simultaneously, a dependency structure parser is used to parse the structure of the training corpus. The training corpus is expanded by comparing entity references and dependency structure parsing results with existing labeled corpora, as well as through dictionary matching. This addresses the issue of insufficient labeled corpora and provides richer textual data for subsequent automatic learning of text structure information.

[0031] This invention also provides a structure-enhanced Chinese medical text entity relation classification system. The system is used to implement the structure-enhanced Chinese medical text entity relation classification method described above, including: The corpus acquisition module is used to acquire the corpus training set; The corpus augmentation module is used to augment the training corpus to obtain an augmented corpus. The text sequence representation module is used to feed the expanded corpus into the BERT language model pre-trained in Chinese to obtain a vector representation of each character. Permutations and combinations yield text sequence representations ; The structural information fusion module is used to represent text sequences. After being fed into a bidirectional long short-term memory network (BiLSTM), the output is a text sequence representation that incorporates structural information. The entity type vector representation module is used to represent entity types in text using vectors to obtain entity type vectors. ; The entity reference vector representation module is used to perform vector representation on entity references to obtain entity reference vectors. ; The concatenation module is used to concatenate the entity reference vector and the entity type vector to obtain the final vector representation of a given entity. ; The relation classification module is used to classify the relation between any two entities i and j by concatenating their entity reference vectors and entity type vectors and then feeding them into the fully connected layer.

[0032] Please see Figure 2 This is a schematic diagram illustrating an embodiment of the electronic device provided in this invention. For example... Figure 2 As shown, the present invention provides an electronic device, including a memory 1310, a processor 1320, and a computer program 1311 stored in the memory 1310 and executable on the processor 1320. When the processor 1320 executes the computer program 1311, it performs the following steps: S1, acquiring a corpus training set; S2, the expanded corpus is obtained by expanding the training corpus; S3, The expanded corpus is fed into the BERT language model pre-trained in Chinese to obtain the vector representation of each character. Permutations and combinations yield text sequence representations ; S4, representing the text sequence After being fed into a bidirectional long short-term memory network (BiLSTM), the output is a text sequence representation that incorporates structural information. S5, Perform vector representation of entity types in the text to obtain entity type vectors. ; S6, Perform vector representation on entity references to obtain entity reference vectors. ; S7 concatenates the entity reference vector and the entity type vector to obtain the final vector representation of a certain entity: ; S8. For any two entities i and j, the entity referential vector and entity type vector are concatenated and then fed into the fully connected layer for relation classification.

[0033] Please see Figure 3 A schematic diagram illustrating an embodiment of the computer-readable storage medium provided by the present invention. (See diagram below.) Figure 3 As shown, this embodiment provides a computer-readable storage medium 1400, on which a computer program 1411 is stored. When the computer program 1411 is executed by a processor, it performs the following steps: S1, acquiring a corpus training set; S2, the expanded corpus is obtained by expanding the training corpus; S3, The expanded corpus is fed into the BERT language model pre-trained in Chinese to obtain the vector representation of each character. The text sequence representation is obtained through permutations and combinations. ; S4, representing the text sequence After being fed into a bidirectional long short-term memory network (BiLSTM), the output is a text sequence representation that incorporates structural information. S5, Perform vector representation of entity types in the text to obtain entity type vectors. ; S6, Perform vector representation on entity references to obtain entity reference vectors. ; S7 concatenates the entity reference vector and the entity type vector to obtain the final vector representation of a certain entity: ; S8. For any two entities i and j, the entity referential vector and entity type vector are concatenated and then fed into the fully connected layer for relation classification.

[0034] Beneficial effects: This invention improves the performance of an automatic entity relation classification model for Chinese medical texts in two ways. First, it uses attention mechanisms and inductive learning methods to summarize text structural information and integrate the learned structural information into the text representation vector, eliminating the need for pre-analysis of the text structure. This enhances the model's ability to represent text and improves its classification performance. Second, it augments the training corpus by using a medical terminology dictionary and a dependency structure analyzer to expand the corpus from two aspects: the expression type of entity references and the structural composition of entity references. This enriches the diversity of the training corpus and improves the model's generalization performance.

[0035] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0036] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0037] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A structural information-enhanced method for classifying entity relationships in Chinese medical texts, characterized in that, Includes the following steps: S1, Obtain the training corpus; S2, the expanded corpus is obtained by expanding the training corpus; S3, The expanded corpus is fed into the BERT language model pre-trained in Chinese to obtain the vector representation of each character. To obtain the text sequence representation L represents the BERT layer number, and i and n are the character indices; S4, representing the text sequence The text sequence representation, which is fed into a Bi-LSTM network with an fused attention mechanism, outputs fused structural information. ; S4 specifically includes: S41 uses a BiLSTM network layer with fused self-attention mechanism to model the directional correlation between each pair of character representations, where the inductive probability of the text structure is modeled as the correlation sum of all edges; S42 uses softmax for normalization; S43, Calculate the marginal probability distribution using the matrix tree theorem. The text structure is inductively calculated by sampling from independent distributions, and a text sequence representation with fused structural information is output. Let represent the vector representation of the i-th character output after passing through a BiLSTM with a self-attention mechanism. The text structure for the i-th character; S5, Perform vector representation of entity types in the text to obtain entity type vectors. ; S5 specifically includes: Constructing a network of encoded entity types The network For one The matrix is ​​given by q, where q represents the number of entity types and d represents the vector dimension. For the k-th entity type The vector representation is the entity type vector; S6, Perform vector representation on entity references to obtain entity reference vectors. ; S6 specifically includes: The average of each dimension of the vector representation containing the fused structural information of characters in the entity reference is taken: That is, entity reference. The semantic representation vector that integrates textual structural information, i.e., the entity reference vector; S7 concatenates the entity reference vector and the entity type vector to obtain the final vector representation of a certain entity: ; S8. For any two entities b and c, concatenate their entity referential vectors and entity type vectors and then feed them into the fully connected layer for relation classification. Following S8, the following is also included: Predict its relation type label using the softmax function: ,in, Let be the final vector representation of the t-th entity. Let f be the final vector representation of the f-th entity. This is the parameter matrix of the fully connected layer. is the bias vector of the fully connected layer, and f is the one-hot vector of the predicted label output.

2. The structural information-enhanced Chinese medical text entity relation classification method according to claim 1, characterized in that, S2 specifically includes: Dependency structure parsing is performed on the text. If a certain entity reference is word-segmented, the entity reference is split according to the dependency relationship type between the word groups, the entity type and relation type are copied, and new triplet samples are generated, or Fuzzy matching of labeled entities in the corpus was performed using a medical terminology dictionary. Some Chinese expressions of medical terms were replaced with English, and English expressions were replaced with Chinese, generating a new text corpus.

3. The structural information-enhanced Chinese medical text entity relation classification method according to claim 1, characterized in that, S3 specifically includes: A language model is built using an inductive structure learning neural network with an attention fusion mechanism. The training corpus is used to study the connections between entity words, between entity words and non-entity words, and between non-entity words and non-entity words.

4. A Chinese medical text entity relation classification system with enhanced structural information, characterized in that, The system is used to implement the structure information-enhanced Chinese medical text entity relation classification method as described in any one of claims 1-3, including: The corpus acquisition module is used to acquire the corpus training set; The corpus augmentation module is used to augment the training corpus to obtain the augmented corpus. The text sequence representation module is used to feed the expanded corpus into the BERT language model pre-trained in Chinese to obtain a vector representation of each character. The text sequence representation is obtained through permutations and combinations. ; The structural information fusion module is used to represent text sequences. The text sequence representation, which is fed into a Bidirectional Long Short-Term Memory (BiLSTM) network, outputs fused structural information. ; The entity type vector representation module is used to represent entity types in text using vectors to obtain entity type vectors. ; The entity reference vector representation module is used to perform vector representation on entity references to obtain entity reference vectors. ; The concatenation module is used to concatenate the entity reference vector and the entity type vector to obtain the final vector representation of a given entity. ; The relation classification module is used to classify the relations between any two entities b and c by concatenating their entity reference vectors and entity type vectors and then feeding them into the fully connected layer.

5. An electronic device, characterized in that, The system includes a memory and a processor, wherein the processor is used to implement the steps of the structure information-enhanced Chinese medical text entity relation classification method as described in any one of claims 1-3 when executing a computer management program stored in the memory.

6. A computer-readable storage medium, characterized in that, It stores a computer management program, which, when executed by a processor, implements the steps of the structure information-enhanced Chinese medical text entity relation classification method as described in any one of claims 1-3.

Images