Electronic medical record feature extraction method based on natural language processing

Abstract

The invention provides an electronic medical record feature extraction method based on natural language processing, and the method comprises the steps: obtaining an electronic medical record of a brain injury patient, the electronic medical record comprising preoperative brain map data, medical record text records, and postoperative brain map data; extracting medical record character records by using a natural language processing model, and determining text features; on the basis of the preoperative brain image data and the postoperative brain image data, brain region features are determined; based on data time nodes in the electronic medical record of the brain injury patient, performing feature fusion on the text features by using the preoperative brain map data and the postoperative brain map data; and based on the brain region features and the text features, determining electronic medical record features of the brain injury patient. According to the scheme, an electronic medical record is processed by using a natural language through a multi-modal data processing scheme, and the electronic medical record is further combined with the brain map features of the neural image data, so that organically combined multi-modal data features capable of being applied by an artificial intelligence technology are formed, and a foundation is laid for subsequent application.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an electronic medical record feature extraction method based on natural language processing. BACKGROUND

[0002] Brain injury mainly includes traumatic brain injury (TBI), stroke, hypoxic brain injury, etc. For brain injury patients, surgical treatment is a common means. In addition to collecting patient information (through electronic medical record), preoperative assessment also needs to collect neuroimaging data such as CT scan, magnetic resonance imaging (MRI), structural MRI (sMRI) displays brain tissue atrophy, edema or lesion location through T1 / T2 weighted images, functional MRI (fMRI) reflects brain activation and functional connectivity (used to assess cognitive recovery potential) through BOLD signal, and diffusion MRI (dMRI) reflects the integrity of white matter fiber bundles through the directional diffusion characteristics of water molecules in white matter. After surgery, neuroimaging data will also be collected to determine the completion of the operation, estimate the recovery of the patient, and develop appropriate rehabilitation treatment plans.

[0003] For brain injury patients, although there are more data (electronic medical records, neuroimaging data, etc.), there is currently no related technology that can effectively utilize these data. Diagnosis, treatment, and prognosis almost entirely rely on the experience of doctors, making it difficult to effectively utilize these data to provide valuable reference suggestions for doctors. The reason is that the electronic medical records and neuroimaging data of brain injury patients are vastly different and difficult to effectively combine. Simple electronic medical records, neuroimaging data, etc. are difficult to give appropriate references and suggestions in the overall direction of brain injury patients.

[0004] Therefore, the processing technology of multi-modal data of brain injury patients is the key to hinder its expansion application. How to process data into appropriate features and combine it with current advanced artificial intelligence technology is a technical problem in the field. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide an electronic medical record feature extraction method based on natural language processing, to utilize natural language processing electronic medical records through a multi-modal data processing scheme, and further combine them with brain map features of neuroimaging data to form multi-modal data features that can be applied by artificial intelligence technology, laying a foundation for subsequent applications (such as prediction of prognosis, suggestion of rehabilitation treatment plan, evaluation of state during rehabilitation, etc.).

[0006] To achieve the above purpose, the embodiments of the present application are implemented in the following way:

[0007] The embodiment of the application provides a kind of based on natural language processing's electronic medical record feature extraction method, comprising: obtaining the electronic medical record of brain injury patient, wherein, electronic medical record includes preoperative brain map data, medical record literal record, postoperative brain map data, preoperative brain map data contains preoperative structural connection atlas and preoperative functional connection atlas based on preoperative sMRI, fMRI and dMRI, postoperative brain map data contains postoperative structural connection atlas and postoperative functional connection atlas based on postoperative sMRI, fMRI and dMRI, medical record literal record includes patient basic information, condition analysis, operation arrangement, medication, postoperative observation record;Text features are determined by using natural language processing model to extract medical record literal record;Determine the brain region feature based on preoperative brain map data and postoperative brain map data;Based on the data time node in the electronic medical record of brain injury patient, the text features are fused using preoperative brain map data and postoperative brain map data;Determine the electronic medical record features of brain injury patient based on brain region feature and text feature.

[0008] Further, using natural language processing model to extract medical record literal record, determine text features, comprising: pre-processing medical record literal record to form case text;Entity, relationship and attribute extraction is carried out on the case text to form several triplets as text features, wherein, one triplet is "entity-relation-entity" triplet, or, "entity-attribute-attribute value" triplet, and the word vector length of entity is consistent with the word vector length of attribute value.

[0009] Further, entity, relationship and attribute extraction is carried out on the case text to form several triplets, comprising: using BERT model trained by brain region terminology dictionary strengthening mask to extract word vector of each word segmentation, to determine the word vector of each word segmentation;Using unified pointer network to identify entity, attribute and relationship of word vector, to generate several entity-relation-entity triplets and entity-attribute-attribute value triplets.

[0010] Further, the brain region terminology dictionary strengthening mask training mode of BERT model is: obtaining brain region terminology dictionary;In BERT model training, brain region terminology word segmentation belonging to brain region terminology dictionary is strengthened mask:

[0011]

[0012] Wherein, P mask (w i ) represents the probability that the i th word is replaced by the MASK mark, w i The i th word of current input sentence text, p0 The basic mask probability of BERT model, Δp The additional mask probability of brain region terminology, V brain Brain region terminology dictionary.

[0013] Further, the unified pointer network is used for entity recognition, attribute recognition and relationship recognition on the word vector, including: predicting the entity start position and end position of each word through the entity pointer to realize entity recognition; predicting the relationship type of each entity pair through the relationship pointer to realize relationship recognition; predicting the position of the attribute value of each entity through the attribute pointer to realize attribute recognition.

[0014] Further, based on the preoperative brain mapping data and the postoperative brain mapping data, the brain region characteristics are determined, including: based on the preoperative structural connectivity atlas and the preoperative functional connectivity atlas in the preoperative brain mapping data, the preoperative brain region characteristics of each brain region are determined, wherein the length of the preoperative brain region characteristics is consistent with the length of the entity word vector; based on the postoperative structural connectivity atlas and the postoperative functional connectivity atlas in the postoperative brain mapping data, the postoperative brain region characteristics of each brain region are determined, wherein the length of the postoperative brain region characteristics is consistent with the length of the entity word vector; based on the preoperative brain region characteristics and the postoperative brain region characteristics, the preoperative-postoperative difference of each brain region is determined, wherein the length of the preoperative-postoperative difference is consistent with the length of the preoperative brain region characteristics and the postoperative brain region characteristics; based on the preoperative-postoperative difference of the brain region, the brain region characteristics are determined.

[0015] Further, based on the preoperative-postoperative difference of the brain region, the brain region characteristics are determined, including: for each brain region: determining the “entity-attribute-attribute value” triple corresponding to this brain region as the brain region characteristics of this brain region, wherein the entity is the word vector of the brain region name of this brain region spliced with the preoperative-postoperative difference of this brain region, the attribute is “numerical value”, and the attribute value is the preoperative-postoperative difference of this brain region.

[0016] Further, based on the data time nodes in the electronic medical record of the brain injury patient, the preoperative brain mapping data and the postoperative brain mapping data are used for feature fusion of the text features, including: for each "entity-relation-entity" triple in the text feature: when the preposition entity of the current text feature has an associated brain region, based on the data time node of the current text feature, the target feature corresponding to the associated brain region is determined from the preoperative brain region feature or the postoperative brain region feature, and the target feature is spliced after the word vector of the preposition entity to form the text feature after feature fusion; when the postposition entity of the current text feature has an associated brain region, the "entity-relation-entity" triple of the current text feature is inverted and converted, and then based on the data time node of the current text feature, the target feature corresponding to the associated brain region is determined from the preoperative brain region feature or the postoperative brain region feature, and the target feature is spliced after the word vector of the inverted and converted preposition entity to form the text feature after feature fusion; when the preposition entity and the postposition entity of the current text feature have associated brain regions, based on the data time node of the current text feature, the target feature corresponding to the associated brain region of the preposition entity is determined from the preoperative brain region feature or the postoperative brain region feature, and the target feature is spliced after the word vector of the preposition entity to form one text feature after feature fusion, and the "entity-relation-entity" triple of the current text feature is inverted and converted, and then based on the data time node of the current text feature, the target feature corresponding to the associated brain region of the inverted and converted preposition entity is determined from the preoperative brain region feature or the postoperative brain region feature, and the target feature is spliced after the word vector of the inverted and converted preposition entity to form another text feature after feature fusion; when neither the preposition entity nor the postposition entity of the text feature has an associated brain region, a zero vector is spliced after the word vector of the preposition entity to form the text feature after feature fusion, wherein the length of the zero vector is consistent with the length of the preoperative brain region feature and the postoperative brain region feature.

[0017] Further, based on the data time nodes in the electronic medical record of the brain injury patient, the preoperative brain mapping data and the postoperative brain mapping data are used for feature fusion of the text features, including: for each "entity-relation-entity" triple in the text feature: when the preposition entity of the current text feature has an associated brain region, based on the data time node of the current text feature, the target feature corresponding to the associated brain region is determined from the preoperative brain region feature or the postoperative brain region feature, and the target feature is spliced after the word vector of the preposition entity to form the text feature after feature fusion; when the postposition entity of the current text feature has an associated brain region, the "entity-relation-entity" triple of the current text feature is inverted and converted, and then based on the data time node of the current text feature, the target feature corresponding to the associated brain region is determined from the preoperative brain region feature or the postoperative brain region feature, and the target feature is spliced after the word vector of the inverted and converted preposition entity to form the text feature after feature fusion; when the preposition entity and the postposition entity of the current text feature have associated brain regions, based on the data time node of the current text feature, the target feature corresponding to the associated brain region of the preposition entity is determined from the preoperative brain region feature or the postoperative brain region feature, and the target feature is spliced after the word vector of the preposition entity to form one text feature after feature fusion, and the "entity-relation-entity" triple of the current text feature is inverted and converted, and then based on the data time node of the current text feature, the target feature corresponding to the associated brain region of the inverted and converted preposition entity is determined from the preoperative brain region feature or the postoperative brain region feature, and the target feature is spliced after the word vector of the inverted and converted preposition entity to form another text feature after feature fusion; when neither the preposition entity nor the postposition entity of the text feature has an associated brain region, a zero vector is spliced after the word vector of the preposition entity to form the text feature after feature fusion, wherein the length of the zero vector is consistent with the length of the preoperative brain region feature and the postoperative brain region feature.

[0018] Further, based on the brain region features and the text features, the electronic medical record features of the brain injury patient are determined, including: based on the data time nodes of the brain region features and the data time nodes of the text features, the brain region features and the text features are sorted; for multiple brain region features and text features of the same data time node, (x-1)Δt is sequentially added to the same data time node in order according to the sequence, wherein x is the feature sequence of each brain region feature and text feature at the same data time node, and Δt is the time difference; all brain region features and text features are integrated according to the data time nodes to obtain the electronic medical record features of the brain injury patient.

[0019] Advantages:

[0020] 1. The scheme obtains the electronic medical record of the brain injury patient (including preoperative brain map data, medical record text record, postoperative brain map data, the preoperative brain map data includes the preoperative structural connection atlas (sMRI, fMRI and dMRI are needed) and the preoperative functional connection atlas (sMRI, fMRI are needed) constructed based on preoperative sMRI, fMRI and dMRI, the postoperative brain map data includes the postoperative structural connection atlas (sMRI, fMRI and dMRI are needed) and the postoperative functional connection atlas (sMRI, fMRI are needed) constructed based on postoperative sMRI, fMRI and dMRI, and the medical record text record includes patient basic information, condition analysis, operation arrangement, medication, and postoperative observation record); the natural language processing model is used to extract the medical record text record to determine the text features; based on the preoperative brain map data and the postoperative brain map data, the brain region features are determined; based on the data time nodes in the electronic medical record of the brain injury patient, the preoperative brain map data and the postoperative brain map data are used for feature fusion of the text features; based on the brain region features and the text features, the electronic medical record features of the brain injury patient are determined. The scheme uses natural language processing technology to extract the data of the text part in the electronic medical record, forms a triple, and fuses the triple by using the brain region features extracted from the brain map data, and converts the brain region features into consistency, so that the multiple source data is organically combined, and a feature vector with consistent feature length is processed, and the number of feature vectors is indefinite (the feature vectors extracted from the electronic medical records of different patients are different), thereby forming a feature sequence with indefinite sequence length and time node, thereby forming an input feature form that can be applied to existing artificial intelligence technology (such as neural differential equation, Neural ODE), and laying a foundation for subsequent application (such as prediction of prognosis, suggestion of rehabilitation treatment scheme, and evaluation of state in rehabilitation process).

[0021] 2. Considering the specificity of brain region terminology, a brain region terminology dictionary is constructed for intensive mask training. The intensive mask mechanism can significantly improve the recognition accuracy of the model for brain region names (such as "hippocampus" and "prefrontal cortex") and reduce the semantic drift of general natural language processing technology in the professional field. By masking the brain region terminology with a high probability, the model is forced to rely on the surrounding words (such as symptom description and surgical operation) to infer the masked word, thereby learning more robust medical entity relevance (such as the implicit relationship between "hippocampal atrophy" and "memory disorder"). In the process of feature extraction and fusion, in order to ensure that the length of each relatively independent feature vector (i.e. the triplets, including "entity-relation-entity" triplets and "entity-attribute-attribute value" triplets) remains consistent, the scheme creatively combines the triplets and brain map features, and sets the length of the brain map feature to ensure the consistency of the vector dimension. For brain regions not recorded in the electronic medical record in the brain map feature, the triplet form of "entity-attribute-attribute value" is designed to ensure the consistency of the vector dimension and comprehensively supplement the features of all brain regions, ensuring the integrity of the information and facilitating the discovery of potential features and improving the feasibility of the application. According to the data time node, the brain map feature (brain map preoperative feature or brain map postoperative feature) is dynamically selected and spliced into the word vector of the associated entity. For entities without associated brain regions, zero vectors are used to fill in, which can ensure the consistency of the length of the triplets. For the case where both entities in the triplet ("entity-relation-entity") are associated with brain regions, two feature vectors are generated by using forward and reverse conversion, which can ensure that all text features are aligned with brain map features and avoid information loss. Finally, according to the time node, a time difference Δt (for example, 1 second, which is different from the time interval of the data time node, and the data time node of the electronic medical record is generally in hours or even days) is added to the features of the same time node, which can better apply to Neural ODE and ensure the availability and accuracy of the data, thereby laying a good data foundation for subsequent applications.

[0022] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0024] Figure 1A flowchart of the electronic medical record feature extraction method based on natural language processing provided in the embodiment of the present application.

[0025] Figure 2 A schematic diagram of the structural connection atlas.

[0026] Figure 3 A schematic diagram of the functional connection brain atlas.

[0027] Figure 4 A schematic diagram of the dimension relationship between the text features and the brain atlas features. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

[0029] Figure 1 A flowchart of the electronic medical record feature extraction method based on natural language processing provided in the embodiment of the present application. In the embodiment, the electronic medical record feature extraction method based on natural language processing is applied to an electronic device (such as a smart terminal, a server, etc.), which includes steps S10, S20, S30, S40 and S50.

[0030] Since the present solution mainly introduces the feature extraction method of the electronic medical record, and does not involve the specific application of the extracted features in the artificial intelligence model, the present embodiment mainly introduces the present solution through a complete electronic medical record. In practice, there may be missing of some data, for example, some patients with brain injury do not have fMRI done in time after surgery (for example, one month, three months, etc.), but do it six months later, one year later, or even do not do it. This is not within the scope of discussion of the present embodiment.

[0031] First, the electronic device can run step S10.

[0032] Step S10: Obtain the electronic medical record of the patient with brain injury, wherein the electronic medical record includes preoperative brain atlas data, medical record text records, postoperative brain atlas data, the preoperative brain atlas data includes preoperative structural connection atlas and preoperative functional connection atlas constructed based on preoperative sMRI, fMRI and dMRI, the postoperative brain atlas data includes postoperative structural connection atlas and postoperative functional connection atlas constructed based on postoperative sMRI, fMRI and dMRI, and the medical record text records include patient basic information, condition analysis, operation arrangement, medication situation and postoperative observation records.

[0033] In this embodiment, the electronic device can obtain the electronic medical record of the brain injury patient, which includes preoperative brain map data, medical record text record, and postoperative brain map data. The preoperative brain map data includes preoperative structural connectivity atlas and preoperative functional connectivity atlas constructed based on preoperative sMRI, fMRI and dMRI, and the postoperative brain map data includes postoperative structural connectivity atlas and postoperative functional connectivity atlas constructed based on postoperative sMRI, fMRI and dMRI. The construction of the structural connectivity atlas requires sMRI, fMRI and dMRI, and the construction of the functional connectivity atlas requires sMRI and fMRI. The structural connectivity atlas and the functional connectivity atlas are as shown in Figure 2 and Figure 3 The medical record text record contains various information: patient basic information (such as name, age, gender, etc.), condition analysis (such as admission situation, neuroimaging report text, patient symptoms, etc.), operation arrangement, medication situation (such as patient's disease history, medication type, medication dose, medication frequency, etc.), postoperative observation record (such as patient's symptom change, index recovery, etc.). Different hospitals may have different types and records of medical record text records, but in general, these several large modules are basically available.

[0034] After obtaining the electronic medical record of the brain injury patient, the electronic device can run step S20.

[0035] Step S20: Extract the medical record text record using a natural language processing model to determine the text features.

[0036] In this embodiment, the electronic device can use a natural language processing model to extract the medical record text record to determine the text features.

[0037] First, the electronic device can preprocess the medical record text record to form a case text. The preprocessing here includes sentence division, symbol elimination, etc. The case text is divided into sentences as input: S = [s1, s2, …, sn], where sn is the nth input sentence. n ], wherein sn is the nth input sentence. n

[0038] Then, the electronic device can perform entity, relationship, and attribute extraction on the case text to form a number of triples as text features, wherein a triple is an "entity-relation-entity" triple, or an "entity-attribute-attribute value" triple, and the word vector length of the entity is consistent with the word vector length of the attribute value.

[0039] For example, the BERT model can be used as a word vector extraction tool, but in order to make it more effective in the medical field, the BERT model is reinforced mask training using a brain region terminology dictionary to reduce semantic drift.

[0040] ​The brain region terminology dictionary reinforced mask training manner of the BERT model is: a brain region terminology dictionary is acquired (the brain region terminology dictionary can be constructed, or a medical terminology dictionary in the brain injury field is expanded), and then brain region terminology segmentation belonging to the brain region terminology dictionary is reinforced in mask training of the BERT model:

[0041]

[0042] Wherein, P mask (w i ) represents the probability that the ith word is replaced by the MASK label, w i is the ith word of the current input sentence text, p0 is the basic mask probability of the BERT model, Δp is the additional mask probability of the brain region terminology, V brain is the brain region terminology dictionary.

[0043] Accordingly, the sentence can be input into the BERT model after the brain region terminology dictionary reinforced mask training, and the word vector of each segmentation is determined through word vector extraction.

[0044] After obtaining the word vector of each segmentation, the electronic equipment can use the unified pointer network to perform entity recognition, attribute recognition and relationship recognition on the word vector, and generate a plurality of entity-relation-entity triplets and entity-attribute-attribute value triplets.

[0045] For example, the entity start position and end position of each word can be predicted through the entity pointer to realize entity recognition, the relationship type of each entity pair can be predicted through the relationship pointer to realize relationship recognition, and the position of the attribute value of each entity can be predicted through the attribute pointer to realize attribute recognition.

[0046] Specifically, the entity type can be defined as a brain region entity and a non-brain region entity, the attribute type can be defined as a size, a size change, a numerical value, a numerical value change, a severity, and the like, and the relationship type can be defined as a symptom, an attribute, and no relationship. It should be noted that this is only an example and should not be considered as a limitation of the present application (for example, a position can also be introduced in the attribute).

[0047] Then, the joint probability model is:

[0048]

[0049] Wherein, y is the structured output (in the form of a triplet, it is an “entity-relation-entity” triplet or an “entity-attribute-attribute value” triplet), x is the input (such as the set of word vectors W extracted before), T is the total number of entities, e t is the tth entity, r ij is the relationship between entity i and entity j, l k is the attribute of entity k, P(et |x) represents an entity recognition item, P(l k |e k ,x) is an entity attribute recognition item, P(r ij |e i ,e j ,x) is an entity relationship recognition item, A = {(k, l) | e k needs to be predicted attribute type l}, is the set of all entity-attribute pairs that need to be predicted attributes, and B is the set of all entity pairs.

[0050] Then, the loss function needs to be adjusted to a loss function containing three kinds of losses (entity recognition loss, attribute prediction loss, relationship classification loss):

[0051] L = λ1L ent + λ2L att + λ3L rel , (3)

[0052] Wherein, L is the loss function, λ1, λ2 and λ3 are hyperparameters for balancing the loss contribution of different tasks, which can be set by grid search or experience: λ1 = 0.4, λ2 = 0.3, λ3 = 0.3. And L ent , L att and L rel are entity recognition loss, attribute prediction loss and relationship classification loss, respectively, which satisfy:

[0053]

[0054] L att = -∑ (k,l)∈A log(l k | e k,x), (5)

[0055] L rel = -∑ (i,j)∈B,i≠j logP(r ij |e i ,e j ,x), (6)

[0056] Accordingly, the entity, attribute and relationship can be identified by using the entity recognition head, attribute recognition head and relationship recognition head, and a plurality of triples: “entity-relationship-entity” triples and “entity-attribute-attribute value” triples are obtained as text features.

[0057] After completing the extraction of the text features, the electronic device can run step S30.

[0058] Step S30: Based on the preoperative brain map data and the postoperative brain map data, the brain region features are determined.

[0059] In this embodiment, the electronic device can determine the preoperative brain region feature of each brain region based on the preoperative structural connectivity atlas and the preoperative functional connectivity atlas in the preoperative brain mapping data, wherein the length of the preoperative brain region feature is consistent with the word vector length of the entity.

[0060] For example, taking a 768-dimensional word vector as an example, both the entity and the attribute are 768-dimensional, and the preoperative brain region feature of each brain region needs to be designed as 768-dimensional. For example, the preoperative brain region feature of each brain region can be formed as a 768-dimensional feature vector by splicing the partial feature (e.g., 96-dimensional) of the brain region in the preoperative structural connectivity atlas + the connection strength (e.g., 372-dimensional) of the brain region with other brain regions in the preoperative functional connectivity atlas + the time series data (e.g., 300-dimensional) of the brain region in the preoperative functional connectivity atlas.

[0061] Similarly, the electronic device can determine the postoperative brain region feature of each brain region based on the postoperative structural connectivity atlas and the postoperative functional connectivity atlas in the postoperative brain mapping data, wherein the length of the postoperative brain region feature is consistent with the word vector length of the entity.

[0062] For example, the postoperative brain region feature of each brain region can be formed as a 768-dimensional feature vector by splicing the partial feature (e.g., 96-dimensional) of the brain region in the postoperative structural connectivity atlas + the connection strength (e.g., 372-dimensional) of the brain region with other brain regions in the postoperative functional connectivity atlas + the time series data (e.g., 300-dimensional) of the brain region in the postoperative functional connectivity atlas.

[0063] Further, the electronic device can also determine the preoperative-postoperative difference of each brain region based on the preoperative brain region feature and the postoperative brain region feature, wherein the length of the preoperative-postoperative difference is consistent with the length of the preoperative brain region feature and the postoperative brain region feature.

[0064] Similarly, the 768-dimensional postoperative brain region feature of the brain region needs to be subtracted from the 768-dimensional preoperative brain region feature to obtain the 768-dimensional preoperative-postoperative difference.

[0065] Accordingly, the preoperative brain region feature, the postoperative brain region feature, and the preoperative-postoperative difference of each brain region can be determined, and the electronic device can further determine the brain region feature based on the preoperative-postoperative difference.

[0066] For example, for each brain region, the electronic device can determine the “entity-attribute-attribute value” triple corresponding to the brain region as the brain region feature of the brain region, wherein the entity is the word vector of the brain region name of the brain region spliced with the preoperative-postoperative difference of the brain region, the attribute is “numerical value”, and the attribute value is the preoperative-postoperative difference of the brain region, such as Figure 4Of course, the features can also be more perfectly obtained, that is, three brain region features are determined for each brain region, and the three brain region features are all in the form of "entity-attribute-attribute value" triplets, and are respectively: (1) the entity is the word vector of the brain region name of this brain region, followed by the brain region preoperative and postoperative difference of this brain region, the attribute is "numerical value", and the attribute value is the brain region preoperative and postoperative difference of this brain region; (2) the entity is the word vector of the brain region name of this brain region, followed by the brain region preoperative feature of this brain region, the attribute is "numerical value", and the attribute value is the brain region preoperative and postoperative difference of this brain region; (3) the entity is the word vector of the brain region name of this brain region, followed by the brain region postoperative feature of this brain region, the attribute is "numerical value", and the attribute value is the brain region preoperative and postoperative difference of this brain region.

[0067] Then, the triplets of the brain region features are "entity (brain region entity concatenation brain region preoperative and postoperative difference)-attribute-attribute value", and accordingly the brain region features of each brain region can be obtained.

[0068] After that, the electronic device can run step S40.

[0069] Step S40: based on the data time node in the electronic medical record of the brain injury patient, the preoperative brain map data and the postoperative brain map data are used for feature fusion of the text features.

[0070] In the present embodiment, in order to realize the organic fusion of the text features and the brain region features, the present embodiment is introduced respectively in multiple cases.

[0071] For each "entity-relation-entity" triplet in the text feature:

[0072] When the preposition entity of the current text feature has an associated brain region, the electronic device can determine the target feature corresponding to the associated brain region from the brain region preoperative feature or the brain region postoperative feature based on the data time node of the current text feature (i.e. the node recording the electronic medical record, generally in units of hours or days, of course, it can also be simply distinguished between preoperative and postoperative), and the target feature is concatenated after the word vector of the preposition entity (for example, the triplet is entity A-relation-entity B, A is the preposition entity and B is the postposition entity), to form the text feature after feature fusion. The text feature after fusion can be consistent with the dimension of the brain region feature.

[0073] When the post-entity of the current text feature (for example, the triple is entity A-relation-entity B, A is the pre-entity, and B is the post-entity) has an associated brain region, the electronic device can invert and convert the "entity-relation-entity" triple of the current text feature (for example, entity A-relation-entity B, after inversion and conversion, it becomes entity B-relation'-entity A, where relation' can be realized by inverting the word vector of the relation, or can be realized in other ways), and determine the target feature corresponding to the associated brain region of the brain region from the brain region preoperative feature or the brain region postoperative feature based on the data time node of the current text feature, and concatenate the target feature after the word vector of the pre-entity after inversion and conversion (here, the pre-entity after inversion and conversion becomes entity B) to form a text feature after feature fusion. In this way, the text feature after fusion can also maintain consistency with the dimension of the brain region feature.

[0074] When the pre-entity and the post-entity of the current text feature have associated brain regions, the electronic device needs to determine the target feature corresponding to the associated brain region of the pre-entity from the brain region preoperative feature or the brain region postoperative feature based on the data time node of the current text feature, and concatenate the target feature after the word vector of the pre-entity to form one text feature after feature fusion, and invert and convert the "entity-relation-entity" triple of the current text feature, and determine the target feature corresponding to the associated brain region of the pre-entity after inversion and conversion from the brain region preoperative feature or the brain region postoperative feature based on the data time node of the current text feature, and concatenate the target feature after the word vector of the pre-entity after inversion and conversion to form another text feature after feature fusion. That is, in order to ensure that the text feature can maintain consistency with the dimension of the brain region feature, we take the splitting scheme, that is, generate two text features, so that the information can be completely reflected, and the dimension of the text feature is consistent with the dimension of the brain region feature.

[0075] When neither the pre-entity nor the post-entity of the previous text feature has an associated brain region, the electronic device needs to concatenate a zero vector after the word vector of the pre-entity to form a text feature after feature fusion, where the length of the zero vector is consistent with the length of the brain region preoperative feature and the brain region postoperative feature. Therefore, even if the entity does not have an associated brain region, the dimension of the text feature can be ensured to be consistent with the dimension of the brain region feature by concatenating the zero vector.

[0076] As for the "entity-attribute-attribute value" triple in each text feature:

[0077] When the entity of the current text feature has an associated brain region, the electronic device can determine the target feature corresponding to the associated brain region from the brain region preoperative feature or the brain region postoperative feature based on the data time node of the current text feature, and concatenate the target feature after the word vector of the entity to form a text feature after feature fusion.

[0078] And in the current text feature entity does not have the associated brain area, the electronic device can be spliced after the word vector of the entity A zero vector, forming a text feature after feature fusion, wherein the length of the zero vector is consistent with the length of the preoperative brain region feature and the postoperative brain region feature.

[0079] So this way can "entity-attribute-attribute value" triplets, but also with the dimension of the brain region characteristics consistent, such as Figure 4 As shown.

[0080] It should be noted that in this embodiment, the text feature (feature fusion) and brain region feature, the middle of the triplets are "attributes" or "relationships", and in order to maintain the uniform dimension of the feature, the specific "attributes" or "relationships" need to be converted into a uniform dimension (such as 768 dimensions) vector by Bert, and the attribute value also needs to be consistent with the length of the entity word vector.

[0081] After completing the feature fusion, the electronic device can run step S50.

[0082] Step S50: Based on the brain region feature and the text feature, the electronic medical record feature of the brain injury patient is determined.

[0083] In this embodiment, the electronic device can sort the brain region features and text features based on the data time nodes of the brain region features and the data time nodes of the text features. For multiple brain region features and text features with the same data time node, (x-1)Δt is sequentially added to the same data time node in order (for example, the order of the text features), where x is the feature order of each brain region feature and text feature at the same data time node, and Δt is the time difference (here, 1 second is taken as an example, and in other embodiments, other numerical values can be taken, which are mainly much smaller than the interval of the data time nodes, and the data time nodes in the electronic medical record are usually at least in hours, and more commonly in days). Then, all brain region features and text features can be integrated according to the data time nodes to obtain the electronic medical record feature of the brain injury patient. This electronic medical record feature can be said to be a sequence data of indefinite length, and the data feature dimension of each node in the sequence is consistent, which has the basis for application in artificial intelligence technology, such as some models that can handle indefinite length, such as neural differential equation (Neural ODE), RNN, LSTM and related variant models, and due to the uncertainty of the time node interval, the model can be completely adapted, and at present, the neural differential equation is mainly used, and the RNN, LSTM related variant model needs to be further adapted to be better applied.

[0084] In summary, the embodiment of the present application provides a natural language processing-based electronic medical record feature extraction method. The electronic medical record of a brain injury patient is obtained (including preoperative brain map data, medical record text record, postoperative brain map data, the preoperative brain map data includes preoperative structural connectivity atlas (sMRI, fMRI and dMRI are needed) and preoperative functional connectivity atlas (sMRI and fMRI are needed) constructed based on preoperative sMRI, fMRI and dMRI, the postoperative brain map data includes postoperative structural connectivity atlas (sMRI, fMRI and dMRI are needed) and postoperative functional connectivity atlas (sMRI and fMRI are needed) constructed based on postoperative sMRI, fMRI and dMRI, and the medical record text record includes patient basic information, condition analysis, operation arrangement, medication, and postoperative observation record); a natural language processing model is used to extract the medical record text record to determine the text features; the brain region features are determined based on the preoperative brain map data and the postoperative brain map data; the text features are fused by using the preoperative brain map data and the postoperative brain map data based on the data time nodes in the electronic medical record of the brain injury patient; and the electronic medical record features of the brain injury patient are determined based on the brain region features and the text features. The scheme uses natural language processing technology to extract the features of the text data in the electronic medical record, forms a triple, fuses the triple by using the brain region features extracted from the brain map data, converts the brain region features into consistency, organically combines the multi-source data, processes the multi-source data into a single feature vector with consistent feature length, the number of feature vectors is indefinite (the feature vectors extracted from the electronic medical records of different patients are different), thereby forming a feature sequence with indefinite sequence length and time nodes, and an input feature form applicable to existing artificial intelligence technology (such as neural differential equation, Neural ODE) is formed, which lays a foundation for subsequent applications (such as prediction of prognosis, suggestion of rehabilitation treatment scheme, and evaluation of state in the rehabilitation process).

[0085] In view of the specificity of brain region terminology, a brain region terminology dictionary is constructed for intensive mask training. The intensive mask mechanism can significantly improve the recognition accuracy of brain region names (such as "hippocampus" and "prefrontal cortex") by the model and reduce the semantic drift of general natural language processing technology in the professional field. By masking the brain region terminology with a high probability, the model is forced to rely on the surrounding words (such as symptom description and surgical operation) to infer the masked word, thereby learning more robust medical entity correlation (such as the implicit relationship between "hippocampal atrophy" and "memory disorder"). And in the process of feature extraction and fusion, in order to ensure that the length of each relatively independent feature vector (i.e. triplets, including "entity-relation-entity" triplets and "entity-attribute-attribute value" triplets) remains consistent, the scheme creatively combines triplets and brain map features, and sets the length of the brain map feature to ensure the consistency of the vector dimension. For brain regions not recorded in the electronic medical record in the brain map feature, the triplet form of "entity-attribute-attribute value" is designed to ensure the consistency of the vector dimension and comprehensively supplement the features of all brain regions, ensuring the integrity of the information and facilitating the discovery of potential features and improving the feasibility of the application. According to the data time node, the brain map feature (brain map preoperative feature or brain map postoperative feature) is dynamically selected and spliced into the word vector of the associated entity. For entities without associated brain regions, zero vectors are used to fill in, which can ensure the consistency of the length of the triplets. For the case where both entities in the triplet ("entity-relation-entity") are associated with brain regions, two feature vectors are generated by using forward and inverted conversion, which can ensure that all text features are aligned with brain map features and avoid information loss. Finally, according to the time node, a time difference Δt (for example, 1 second, which is different from the time interval of the data time node, and the data time node of the electronic medical record is generally in hours or even days) is added to the features of the same time node, which can better apply to Neural ODE and ensure the availability and accuracy of the data, thereby laying a good data foundation for subsequent applications.

[0086] The above only describes the embodiments of the present application and does not limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A natural language processing-based electronic medical record feature extraction method, characterized in that, The method comprises the following steps: obtaining an electronic medical record of a brain injury patient, wherein the electronic medical record comprises preoperative brain map data, medical record text, and postoperative brain map data, the preoperative brain map data comprises preoperative structural connectivity atlas and preoperative functional connectivity atlas constructed based on preoperative sMRI, fMRI and dMRI, the postoperative brain map data comprises postoperative structural connectivity atlas and postoperative functional connectivity atlas constructed based on postoperative sMRI, fMRI and dMRI, and the medical record text comprises patient basic information, condition analysis, surgical arrangement, medication, and postoperative observation record; extracting the medical record text using a natural language processing model to determine text features; determining brain region features based on the preoperative brain map data and the postoperative brain map data; performing feature fusion on the text features based on the preoperative brain map data and the postoperative brain map data according to the data time nodes in the electronic medical record of the brain injury patient; determining electronic medical record features of the brain injury patient based on the brain region features and the text features. 2.The natural language processing based electronic medical record feature extraction method of claim 1, wherein, The method for extracting the medical record text using a natural language processing model to determine text features comprises the following steps: preprocessing the medical record text to form a case text; performing entity, relationship, and attribute extraction on the case text to form a plurality of triplets as text features, wherein one triplet is an "entity-relationship-entity" triplet or an "entity-attribute-attribute value" triplet, and the word vector length of the entity is consistent with the word vector length of the attribute value. 3.The natural language processing based electronic medical record feature extraction method of claim 2, wherein, The method for performing entity, relationship, and attribute extraction on the case text to form a plurality of triplets comprises the following steps: extracting word vectors of each word by using a BERT model trained by brain region terminology dictionary strengthening mask to determine the word vectors of each word; performing entity recognition, attribute recognition, and relationship recognition on the word vectors by using a unified pointer network to generate a plurality of entity-relationship-entity triplets and entity-attribute-attribute value triplets. 4.The natural language processing based electronic medical record feature extraction method of claim 3, wherein, The method for performing brain region terminology dictionary strengthening mask training on the BERT model comprises the following steps: obtaining a brain region terminology dictionary; performing brain region terminology word segmentation strengthening mask in the BERT model training on brain region terminology belonging to the brain region terminology dictionary; where P mask (w i ) represents the probability of the ith word being replaced by the MASK label, w i is the ith word of the current input sentence text, p0 is the base masking probability of the BERT model, Δp is the additional masking probability for brain region terms, and V brain is the brain region term dictionary. 5.The natural language processing based electronic medical record feature extraction method of claim 3, wherein, The method for performing entity recognition, attribute recognition, and relationship recognition on the word vectors by using a unified pointer network comprises the following steps: predicting the entity start position and end position of each word by an entity pointer to realize entity recognition; predicting the relationship type of each entity pair by a relationship pointer to realize relationship recognition; predicting the attribute value position of each entity by an attribute pointer to realize attribute recognition. 6.The natural language processing based electronic medical record feature extraction method according to claim 2, characterized in that, The method for determining brain region features based on the preoperative brain map data and the postoperative brain map data comprises the following steps: determining brain region preoperative features of each brain region based on the preoperative structural connectivity atlas and the preoperative functional connectivity atlas in the preoperative brain map data, wherein the length of the brain region preoperative features is consistent with the word vector length of the entity; determining brain region postoperative features of each brain region based on the postoperative structural connectivity atlas and the postoperative functional connectivity atlas in the postoperative brain map data, wherein the length of the brain region postoperative features is consistent with the word vector length of the entity; Based on the preoperative characteristics of the brain region and the postoperative characteristics of the brain region, the preoperative and postoperative differences of each brain region are determined, wherein the length of the preoperative and postoperative differences of the brain region is consistent with the length of the preoperative characteristics of the brain region and the postoperative characteristics of the brain region; Based on the preoperative and postoperative differences of the brain region, the brain region characteristics are determined. 7.The natural language processing based electronic medical record feature extraction method according to claim 6, characterized in that, Based on the preoperative and postoperative differences of the brain region, the brain region characteristics are determined, including: For each brain region: determine the "entity-attribute-attribute value" triple corresponding to this brain region as the brain region characteristics of this brain region, wherein the entity is the word vector of the brain region name of this brain region concatenated with the preoperative and postoperative differences of the brain region, the attribute is "numerical value", and the attribute value is the preoperative and postoperative differences of the brain region. 8.The natural language processing based electronic medical record feature extraction method of claim 6, wherein, Based on the data time nodes in the electronic medical record of the brain injury patient, the preoperative brain map data and the postoperative brain map data are used to fuse the text features, including: For each "entity-relation-entity" triple in the text feature: When the preposition entity of the current text feature has an associated brain region, based on the data time node of the current text feature, the target feature corresponding to the associated brain region is determined from the preoperative characteristics of the brain region or the postoperative characteristics of the brain region, and the target feature is concatenated after the word vector of the preposition entity to form the text feature after feature fusion; When the postposition entity of the current text feature has an associated brain region, the "entity-relation-entity" triple of the current text feature is inverted and converted, and then based on the data time node of the current text feature, the target feature corresponding to the associated brain region is determined from the preoperative characteristics of the brain region or the postoperative characteristics of the brain region, and the target feature is concatenated after the word vector of the inverted and converted preposition entity to form the text feature after feature fusion; When the preposition entity and the postposition entity of the current text feature have associated brain regions at the same time, based on the data time node of the current text feature, the target feature corresponding to the associated brain region of the preposition entity is determined from the preoperative characteristics of the brain region or the postoperative characteristics of the brain region, and the target feature is concatenated after the word vector of the preposition entity to form one text feature after feature fusion, and the "entity-relation-entity" triple of the current text feature is inverted and converted, and then based on the data time node of the current text feature, the target feature corresponding to the associated brain region of the inverted and converted preposition entity is determined from the preoperative characteristics of the brain region or the postoperative characteristics of the brain region, and the target feature is concatenated after the word vector of the inverted and converted preposition entity to form another text feature after feature fusion; When neither the preposition entity nor the postposition entity of the previous text feature has an associated brain region, a zero vector is concatenated after the word vector of the preposition entity to form the text feature after feature fusion, wherein the length of the zero vector is consistent with the length of the preoperative characteristics of the brain region and the postoperative characteristics of the brain region. 9.The natural language processing based electronic medical record feature extraction method of claim 6, wherein, Based on the data time nodes in the electronic medical record of the brain injury patient, the preoperative brain map data and the postoperative brain map data are used to fuse the text features, including: For each "entity-attribute-attribute value" triple in the text feature: When the entity of the current text feature has an associated brain region, based on the data time node of the current text feature, the target feature corresponding to the associated brain region is determined from the brain preoperative feature or the brain postoperative feature, and the target feature is spliced after the word vector of the entity to form the text feature after feature fusion. When the entity of the current text feature does not have an associated brain region, a zero vector is spliced after the word vector of the entity to form the text feature after feature fusion, wherein the length of the zero vector is consistent with the length of the brain preoperative feature and the brain postoperative feature. 10.The natural language processing based electronic medical record feature extraction method of claim 6, wherein, Based on the brain region feature and the text feature, the electronic medical record feature of the brain injury patient is determined, including: Based on the data time node of the brain region feature and the data time node of the text feature, the brain region feature and the text feature are sorted; For multiple brain region features and text features of the same data time node, (x-1)Δt is sequentially added in order at the same data time node, wherein x is the feature order of each brain region feature and text feature at the same data time node, and Δt is the time difference; Integrate all brain region features and text features according to the data time node to obtain the electronic medical record feature of the brain injury patient.

Images