Bi-LSTM-based medical document analysis method

Abstract

The invention discloses a Bi-LSTM-based medical document analysis method, and relates to the technical field of medical information, and the method comprises the steps: obtaining a to-be-analyzed medical document, and carrying out the preprocessing and word vectorization of the to-be-analyzed medical document; identifying key medical concept entities from the word vector sequence by using a medical entity identification model, and defining position indexes of the key medical concept entities as a concept anchor point set; inputting the word vector sequence into a Bi-LSTM network for coding; generating a weighted context vector based on the hidden state sequence and the concept anchor point set, and fusing the key hidden state and the context vector; and inputting the feature vector into a classifier, and outputting structured information. According to the method, the medical concept entities are introduced as concept anchor points to guide attention distribution, and term-level and context-level layered attention mechanisms are combined, so that the problems of insufficient attention to key information and single semantic understanding in the prior art are effectively solved; the accuracy of entity recognition, disease coding and relation extraction in medical document analysis is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of medical information technology, and in particular to a medical document analysis method based on Bi-LSTM. Background Technology

[0002] With the popularization of hospital informatization, a massive amount of unstructured medical documents (such as electronic medical records, diagnostic reports, discharge summaries, etc.) have been accumulated. These documents contain rich clinical knowledge, including disease diagnosis, symptom description, medication records, and treatment process. How to automatically and accurately extract key structured information from these unstructured texts is of great value for assisting clinical decision-making, automatic disease coding (such as ICD-10), medical quality control, and clinical research. General models process every word in the text equally, but the core value of medical documents is often concentrated on a few key medical entities (such as diseases, symptoms, and drugs). Existing attention mechanisms are usually global and data-driven, lacking the ability to actively guide the model to focus on these key concepts using prior medical knowledge. This can lead to the model being interfered with by a large amount of irrelevant text, affecting the accuracy of key information extraction.

[0003] However, current common solutions have many drawbacks, including: existing intelligent medical document analysis methods, based on data-driven general attention mechanisms, lack guidance from prior medical knowledge and are difficult to accurately focus on key entities; a single semantic understanding level cannot effectively integrate term-level local importance with document-level global context, resulting in insufficient in-depth clinical semantic mining; at the same time, feature representation does not sufficiently enhance entity information and the decision-making process has poor interpretability, which seriously restricts its accuracy and practicality in high-requirement scenarios such as assisted diagnosis. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the problems existing in the current Bi-LSTM-based medical document analysis method, the present invention is proposed.

[0006] Therefore, the purpose of this invention is to provide a medical document analysis method based on Bi-LSTM, which is applicable to solving the problems of existing intelligent medical document analysis methods that rely on data-driven general attention mechanisms, which lack guidance from prior medical knowledge and are difficult to accurately focus on key entities; the single semantic understanding level cannot effectively integrate term-level local importance with document-level global context, resulting in insufficient depth mining of clinical semantics; at the same time, feature representation does not sufficiently enhance entity information and the decision-making process has poor interpretability, which seriously restricts its accuracy and practicality in high-requirement scenarios such as assisted diagnosis.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, embodiments of the present invention provide a medical document analysis method based on Bi-LSTM, comprising: acquiring the medical document to be analyzed; preprocessing and vectorizing it to obtain a word vector sequence; using a medical entity recognition model to identify key medical concept entities from the word vector sequence and defining their position indices as a set of concept anchor points; inputting the word vector sequence into a Bi-LSTM network for encoding to obtain a hidden state sequence; dynamically calculating hierarchical attention weights based on the hidden state sequence and the set of concept anchor points to generate a weighted context vector, and fusing the key hidden states with the context vector to obtain a fused feature vector; and inputting the feature vector into a classifier to output structured information for assisting clinical diagnostic decision-making.

[0008] As a preferred embodiment of the Bi-LSTM-based medical document analysis method of the present invention, the medical concept entities include diseases, symptoms, drugs, and examination items; the hierarchical attention includes term-level attention for focusing on medical entities and context-level attention for understanding clinical semantics; the structured information includes one of the following: disease auto-coding results, medical entity identification results, or medical event relationship extraction results.

[0009] As a preferred embodiment of the Bi-LSTM-based medical document analysis method of the present invention, the specific construction steps of the medical entity recognition model are as follows: acquiring medical text data and constructing an annotated corpus including medical concept entities; selecting a pre-trained language model in the medical field as the basic model; training and adjusting the basic model on the annotated corpus to learn the mapping relationship between medical entity boundaries and types; using the adjusted model as the medical entity recognition model to identify and output medical concept entities and their position indices from word vector sequences.

[0010] As a preferred embodiment of the Bi-LSTM-based medical document analysis method of the present invention, the specific content of encoding the word vector sequence is as follows: the word vector sequence is input into a bidirectional long short-term memory network; the forward LSTM layer of the Bi-LSTM network processes the word vectors in sequence order to generate a forward hidden state sequence; the backward LSTM layer of the Bi-LSTM network processes the word vectors in reverse sequence order to generate a backward hidden state sequence; for each position in the sequence, the forward hidden state and the backward hidden state of the corresponding position are concatenated to form a complete hidden state containing bidirectional contextual information; the complete hidden states of all positions form a hidden state sequence, which is used for subsequent hierarchical attention calculation.

[0011] As a preferred embodiment of the Bi-LSTM-based medical document analysis method of the present invention, the following steps are taken: Based on the hidden state sequence and the concept anchor set, hierarchical attention weights are dynamically calculated to generate a weighted context vector. Specifically: Based on the concept anchor set, key hidden states corresponding to medical concept entities are extracted from the hidden state sequence; the similarity between each hidden state and the key hidden states is calculated to obtain term-level attention weights; based on the entire hidden state sequence, the contribution of each hidden state to the overall clinical semantic understanding is calculated through an attention mechanism to obtain context-level attention weights; the term-level attention weights and context-level attention weights are weighted and fused to obtain the final hierarchical attention weights; the hierarchical attention weights are used to perform a weighted summation of the hidden state sequence to generate a weighted context vector.

[0012] As a preferred embodiment of the Bi-LSTM-based medical document analysis method of the present invention, the method of fusing key hidden states and context vectors to obtain a fused feature vector specifically includes the following steps: extracting key hidden states corresponding to the position indices in the concept anchor set from the hidden state sequence; concatenating the key hidden states with the weighted context vector to form a preliminary fused vector; and performing a nonlinear transformation on the preliminary fused vector through a fully connected layer to obtain the final fused feature vector.

[0013] As a preferred embodiment of the Bi-LSTM-based medical document analysis method of the present invention, the classifier is a multilayer perceptron classifier, and its specific processing includes: inputting the fused feature vector into a multilayer perceptron including at least one hidden layer; introducing nonlinear transformation capability through an activation function; using a softmax function in the output layer to generate probability distributions of various types of structured information; and outputting the final structured information according to the probability distribution, including automatic disease coding results, medical entity recognition results, or medical event relationship extraction results.

[0014] Secondly, to further address the aforementioned technical problems, this invention provides a Bi-LSTM-based medical document analysis system, comprising: a data preprocessing module for acquiring the medical document to be analyzed, preprocessing it, and vectorizing it into word vectors to obtain a word vector sequence; an entity recognition module for identifying key medical concept entities from the word vector sequence using a medical entity recognition model, and defining their position indices as a set of concept anchor points; a network encoding module for inputting the word vector sequence into a Bi-LSTM network for encoding to obtain a hidden state sequence; a feature fusion module for dynamically calculating hierarchical attention weights based on the hidden state sequence and the set of concept anchor points, generating a weighted context vector, and fusing the key hidden states and the context vector to obtain a fused feature vector; and a classification output module for inputting the feature vector into a classifier and outputting structured information for assisting clinical diagnostic decisions.

[0015] Thirdly, embodiments of the present invention provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements any step of the Bi-LSTM-based medical document analysis method described in the first aspect of the present invention.

[0016] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the Bi-LSTM-based medical document analysis method as described in the first aspect of the present invention.

[0017] The beneficial effects of this invention are as follows: By introducing medical conceptual entities as conceptual anchors to guide attention allocation, and combining a hierarchical attention mechanism at the term and context levels, this invention effectively overcomes the problems of insufficient attention to key information and limited semantic understanding in existing technologies, thereby significantly improving the accuracy of entity recognition, disease coding, and relation extraction in medical document analysis. At the same time, this method generates more discriminative feature representations by strengthening the deep integration of key entity features and global semantics. Furthermore, because its attention mechanism is clearly associated with medical concepts, it greatly enhances the transparency and clinical credibility of the model's decision-making process, ultimately achieving accurate and interpretable intelligent analysis of medical documents. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1This is a flowchart illustrating the implementation of the present invention in Example 1. Detailed Implementation

[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0020] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0021] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0022] Example 1 Reference Figure 1 This is the first embodiment of the present invention, which provides a medical document analysis method based on Bi-LSTM, including the following steps: S1: Obtain the medical documents to be analyzed, preprocess them, and vectorize them into word vectors to obtain a sequence of word vectors.

[0023] Specifically, the medical documents to be analyzed refer to unstructured or semi-structured text data generated by medical institutions in clinical activities, such as electronic medical records, diagnostic reports, discharge summaries, etc. The medical documents receive clinical data through a real-time interface.

[0024] Specifically, preprocessing includes medical document cleaning and denoising, normalization of medical terminology, word segmentation, and sentence segmentation; word vectorization is the process of mapping discrete words after word segmentation to a continuous vector space, enabling computers to understand the semantics of words.

[0025] Preferably, word vectors pre-trained in the medical field are used to improve semantic representation performance.

[0026] It should be noted that by combining preprocessing operations such as medical document cleaning and denoising, medical terminology normalization, word segmentation, and sentence segmentation, redundant information such as headers, footers, and garbled characters is effectively removed. This solves the problems of inconsistent medical document formats and non-standard terminology (such as ambiguity in abbreviations like "myocardial infarction" and "diabetic retinopathy"), providing a high-quality text foundation for subsequent semantic analysis. At the same time, pre-trained word vectors in the medical field (such as PCL-MedBERT) are selected for word vectorization, which more accurately captures medical-specific semantic associations such as "disease-symptom-drug" compared to general word vectors. This avoids the misunderstanding of medical terms by general semantic representations. Furthermore, by receiving clinical data through a real-time interface, the timeliness of the documents to be analyzed is ensured, providing standardized, accurate, and real-time input data support for the entire analysis process, thereby improving the processing accuracy of subsequent modules from the source.

[0027] For example, taking the electronic medical record of a hospital outpatient department as an example, the unstructured text of the patient "Zhang, who visited the hospital for 'excessive thirst and urination for more than 1 month,' has a history of 'myocardial infarction,' and has been taking 'aspirin' for a long time" is obtained through the real-time interface of the hospital's HIS system. The document is first preprocessed to remove the header "XX Hospital Outpatient Medical Record (2025-XX-XX)" and garbled characters, normalize "myocardial infarction" to the standard medical term "myocardial infarction," and split it into "patient / Zhang / due to / excessive thirst and urination / more than 1 month / visited / , / past / have / myocardial infarction / , / long-term / taking / aspirin / " according to the medical text segmentation rules, and segment it into a single sentence sequence. Then, the medical domain pre-trained word vector model PCL-MedBERT is used to map each word segment to a 128-dimensional continuous vector, and finally a word vector sequence containing 22 128-dimensional vectors is obtained, which provides standardized semantic input for subsequent analysis.

[0028] S2: Use a medical entity recognition model to identify key medical concept entities from word vector sequences and define their position indices as a set of concept anchor points.

[0029] Furthermore, medical concepts include diseases, symptoms, medications, and tests.

[0030] Specifically, the construction steps for the medical entity recognition model are as follows: Acquire textual data in the medical field and construct an annotated corpus including medical conceptual entities.

[0031] A pre-trained language model from the medical field was selected as the base model.

[0032] The basic model is trained and adjusted on an annotated corpus to learn the mapping relationship between medical entity boundaries and types.

[0033] The adjusted model is used as a medical entity recognition model to identify and output medical concept entities and their location indices from word vector sequences.

[0034] Specifically, the details of training and adjusting the base model on the labeled corpus are as follows: The labeled corpus was converted to BIO sequence labeling format, medical concept entities were encoded with synonyms, and data augmentation techniques such as medical synonym replacement and entity boundary perturbation were used to improve the diversity of training samples and the generalization ability of the model.

[0035] A linear classification layer and a CRF layer are added to the medical pre-trained model. The CRF layer is used to model the sequence dependency relationship between labels, and a Dropout layer is set to prevent overfitting and optimize the entity boundary recognition effect.

[0036] A tiered learning rate strategy is adopted, setting differentiated learning rates for different network layers. The bottom layers retain pre-trained knowledge, while the top layers quickly adapt to new tasks. Key parameters such as the base learning rate, batch size, and training cycle are configured.

[0037] The model was trained using a multi-task loss function and the AdamW optimizer. An early stopping mechanism was implemented with entity F1 score as the evaluation metric. The optimal model checkpoint was saved to ensure that the recognition accuracy of the four types of entities met the clinical application standards.

[0038] Furthermore, the set of concept anchors consists of the location indexes of all identified medical concept entities.

[0039] It should be noted that this step involves constructing a medical-specific annotated corpus (containing four core entities: diseases, symptoms, drugs, and examination items), and performing supervised fine-tuning by adding a linear classification layer to a pre-trained medical model (such as PCL-MedBERT). Combined with the cross-entropy loss function, AdamW optimizer, and early stopping mechanism, the medical entity recognition model consistently achieves an F1 score better than 0.90 on the test set, ensuring high accuracy in identifying key medical concept entities. More importantly, defining the identified entity location indexes as a set of concept anchor points breaks the limitation of existing technologies that "entity recognition only outputs a list of entities." This transforms entity information into "medical prior guidance signals" for subsequent attention calculations, avoiding the predicament of "globally undifferentiated data-driven" semantic analysis. This lays the foundation for accurately focusing on core clinical entities and filtering redundant text. Simultaneously, the category limitation of the anchor point set (four core entities) ensures the clinical relevance of the guidance signals, adapting to the domain characteristics of highly concentrated key information in medical documents.

[0040] It should be noted that the concept anchor set is used to provide medical prior guidance signals for subsequent attention calculations, preserving the entity's position index in the original text and associating it with the deep semantic information encoded by Bi-LSTM.

[0041] For example, based on the word vector sequence obtained from S1, the pre-built medical entity recognition model is called for processing: This model is based on the medical pre-trained model PCL-MedBERT and trained on 10,000 outpatient medical records labeled with "disease, symptoms, drugs, and examination items" (during training, the data is converted to BIO labeled format, a CRF layer is added to optimize boundary recognition, and the AdamW optimizer is used to train to an F1 score of 0.93); the model identifies key medical concept entities from the word vector sequence—symptom "polyuria and polydipsia" (sequence positions 4-5), disease "myocardial infarction" (sequence position 10), and drug "aspirin" (sequence position 16), and records the position index of each entity; these position indices are integrated into a concept anchor set {4-5 (symptoms), 10 (disease), 16 (drug)}, providing medical prior guidance signals for subsequent attention calculation.

[0042] S3: Input the word vector sequence into the Bi-LSTM network for encoding to obtain the hidden state sequence.

[0043] Preferably, the specific content of encoding the word vector sequence is as follows: The word vector sequence is input into a bidirectional long short-term memory network.

[0044] The forward LSTM layer of the Bi-LSTM network processes word vectors in sequence to generate a forward hidden state sequence.

[0045] The backward LSTM layer of the Bi-LSTM network processes word vectors in reverse order to generate a backward hidden state sequence.

[0046] For each position in the sequence, the forward hidden state and the backward hidden state of the corresponding position are concatenated to form a complete hidden state containing bidirectional context information.

[0047] The complete hidden states at all locations form a hidden state sequence, which is used for subsequent hierarchical attention calculations.

[0048] It should be noted that the Bi-LSTM network consists of a forward LSTM layer and a backward LSTM layer. The word vector sequence is input into the forward and backward LSTM layers in sequence to obtain the forward hidden state sequence and the backward hidden state sequence, respectively. At each time step, the forward and backward hidden states are concatenated to form a hidden state sequence containing complete contextual information. The dimension of the hidden state can be adjusted according to the task, and is usually set to 128 or 256 dimensions.

[0049] It should be noted that by using a bidirectional long short-term memory network (Bi-LSTM) to process word vector sequences in parallel with the forward LSTM layer and the backward LSTM layer, the positive and negative contextual semantics of the text are captured respectively. Then, the complete hidden state sequence is formed by concatenation. This effectively solves the problem that traditional unidirectional LSTM cannot fully utilize bidirectional semantic information. It is especially suitable for capturing the semantics of bidirectional relationships such as "symptom description-disease diagnosis" and "drug use-treatment effect" in medical documents. At the same time, the hidden state dimension can be flexibly set to 128 or 256 dimensions according to the length of the medical document (such as short outpatient medical records and long inpatient medical records), taking into account both processing efficiency and semantic capacity. Furthermore, by using a sequence filling method adapted to medical scenarios (such as medical-specific placeholders), the semantic interference caused by general zero filling is avoided. The final generated complete hidden state sequence can provide a semantic carrier containing rich bidirectional contextual information for subsequent hierarchical attention calculation, ensuring the semantic integrity of the attention weight calculation.

[0050] For example, the 22 128-dimensional word vector sequences generated by S1 are input into the Bi-LSTM network: the forward LSTM layer processes the word vectors in the order of "patient → Zhang → cause → ... → aspirin", generating 22 64-dimensional forward hidden state sequences; the backward LSTM layer processes them in the reverse order of "aspirin → ... → cause → Zhang → patient", generating 22 64-dimensional backward hidden state sequences; the vectors at each position are concatenated, such as concatenating the forward 64-dimensional vector of position 4 ("excessive thirst" in "excessive urination") with the backward 64-dimensional vector to form a complete 128-dimensional hidden state, and concatenating the forward and backward vectors of position 10 ("myocardial infarction") to form a 128-dimensional hidden state; finally, a hidden state sequence containing 22 128-dimensional vectors is formed, which completely preserves the bidirectional semantic association of "symptom-disease-drug", providing a semantic carrier for hierarchical attention calculation.

[0051] S4: Based on the hidden state sequence and concept anchor set, dynamically calculate the hierarchical attention weights, generate a weighted context vector, and fuse the key hidden states and context vectors to obtain a fused feature vector.

[0052] Preferably, by utilizing a set of concept anchors, attention allocation is dynamically guided for the generated hidden state sequence, thereby achieving a deep fusion of local key information focus and global semantic understanding.

[0053] Specifically, hierarchical attention includes term-level attention for focusing on medical entities and context-level attention for understanding clinical semantics.

[0054] Specifically, based on the hidden state sequence and the concept anchor set, hierarchical attention weights are dynamically calculated to generate a weighted context vector, as detailed below: Based on the concept anchor set, key hidden states corresponding to medical concept entities are extracted from the hidden state sequence. The similarity between each hidden state and the key hidden state is calculated to obtain the term-level attention weight, as shown in the following formula: ; in, For position The final attention weight is determined by the hierarchy; a larger value indicates that the word is more important to medical decisions. As a normalization factor, it ensures that the sum of attention weights at all positions is 1, avoiding imbalance caused by excessive differences in the absolute values ​​of the weights; For position The query vector, determined by position Hidden state Obtained through linear transformation and tanh activation function, used with the bond matrix. Perform similarity calculation; The key matrix is ​​formed by the key hidden state matrix. It is obtained after linear transformation and tanh activation function; The dimension of the key vector is the same as the dimension of the hidden state in the Bi-LSTM. The sigmoid activation function is the standard activation function in the field of deep learning. The length of the input sequence (i.e., the number of words in the medical document); For Bi-LSTM networks at location The complete hidden state vector, containing bidirectional contextual information at that location; The average vector of the key hidden states represents the overall semantic features of all key medical concept entities.

[0055] Based on the entire hidden state sequence, the contribution of each hidden state to the overall clinical semantic understanding is calculated through an attention mechanism, resulting in context-level attention weights, as detailed below: ; In the formula, Position in the hidden state sequence Context-level attention weights; For Bi-LSTM networks at location The complete hidden state vector, containing bidirectional contextual information at that location; The global semantic mean vector of the hidden state sequence represents the core semantic center of the entire medical document; The length of the input sequence (i.e., the number of words in the medical document); The average value of the hidden state vectors of the entire sequence represents the global semantic center of the entire document; The covariance matrix of the hidden state vector describes the distribution of the hidden state vector in the feature space. The bandwidth parameter of the Gaussian kernel function controls the sensitivity of the similarity metric to distance. For Bi-LSTM networks at location The complete hidden state vector, containing bidirectional contextual information at that location; for The transpose of a vector is converted from a column vector to a row vector; for The transpose of a vector is converted from a column vector to a row vector; Let be the Euclidean distance between the hidden state vector at the current position and the global average vector.

[0056] The term-level attention weights and context-level attention weights are weighted and fused to obtain the final hierarchical attention weights, as shown in the following formula: ; In the formula, For the final hierarchical attention weights, the position The final attention weight represents the importance of the word in medical document analysis; The balance parameter is a learnable balance coefficient that controls the relative importance of the two attention weights. For position The final hierarchical attention weights; Position in the hidden state sequence Context-level attention weights.

[0057] The hidden state sequence is weighted and summed using hierarchical attention weights to generate a weighted context vector, as shown in the following formula: ; In the formula, This is the context vector, and the semantic summary vector of the entire medical document. For the final hierarchical attention weights; For Bi-LSTM networks at location The complete hidden state vector contains bidirectional contextual information for that location.

[0058] Furthermore, the key hidden states and context vectors are fused to obtain the fused feature vector, which specifically includes the following steps: Extract the key hidden states from the hidden state sequence that correspond to the position indices in the set of concept anchors.

[0059] The key hidden states are concatenated with the weighted context vector to form a preliminary fusion vector.

[0060] The initial fusion vector is nonlinearly transformed through a fully connected layer to obtain the final fusion feature vector.

[0061] It should be noted that by designing a hierarchical attention mechanism of "terminology-level attention + context-level attention," a dynamic balance between "precise focusing on key local entities" and "deep understanding of global clinical semantics" is achieved: terminology-level attention extracts key hidden states based on concept anchors, and combines similarity calculation and Sigmoid distance constraints to accurately increase the attention weight of core entities such as diseases and symptoms, while filtering out redundant text interference such as "patient basic information"; context-level attention introduces a hidden state covariance matrix to quantify the global semantic distribution, ensuring that the weight allocation takes into account the overall clinical logic (such as the relationship between "symptoms and past medical history"). This approach avoids semantic fragmentation caused by local entity focusing. Simultaneously, it obtains the final hierarchical attention weights through weighted fusion, generates a context vector through weighted summation, and concatenates it with key hidden states. Then, it undergoes nonlinear transformation through a fully connected layer to achieve deep coupling between "local entity features" and "global semantic features." This solves the inefficiency problem of existing technologies where feature fusion only involves surface-level concatenation, significantly enhancing the discriminative power of the fused feature vectors. This provides core support for subsequent classifiers to output high-precision structured information, especially improving the processing capabilities for complex clinical semantic tasks such as "differentiation of similar diseases" and "entity relationship association."

[0062] For example, hierarchical attention is calculated by combining the hidden state sequence of S3 with the concept anchor set of S2: First, the key hidden states corresponding to the anchor positions (4-5, 10, 16) are extracted, and the cosine similarity between the hidden states of other positions and these key states is calculated to obtain term-level attention weights (e.g., the weights of positions 4-5, 10, and 16 are 0.82, 0.78, and 0.75, respectively; the weights of non-entity positions such as "patient" and "visit" are less than 0.2); then, context-level attention weights are calculated based on the entire hidden state sequence, and the influence of each position on "patient visit reason" is quantified by a Gaussian kernel function. The contribution of "medication history" to the global semantics (e.g., the weights of "cause", "past", and "taken" are 0.45, 0.38, and 0.42, respectively); with a balance parameter λ=0.6, the two types of weights are weighted and fused to obtain the final hierarchical attention weights, and the weighted summation of the hidden state sequence generates a 64-dimensional context vector; finally, the key hidden states (positions 4-5, 10, and 16) are extracted and concatenated into a 384-dimensional vector, which is merged with the 64-dimensional context vector to obtain a 448-dimensional preliminary fusion vector. After nonlinear transformation by a fully connected layer, a 256-dimensional fusion feature vector is obtained, which strengthens the association between key entities and global semantics.

[0063] S5: Input the feature vector into the classifier and output structured information to assist clinical diagnostic decision-making.

[0064] Specifically, structured information includes one of the following: disease autocoding results, medical entity identification results, or medical event relationship extraction results.

[0065] Specifically, the classifier is a multilayer perceptron classifier, and its specific processing steps include: The fused feature vectors are input into the multilayer perceptron.

[0066] Nonlinear transformations are performed using hidden layers.

[0067] The softmax function is used in the output layer to generate probability distributions for various types of structured information.

[0068] The final structured information is output based on the probability distribution, including automatic disease coding results, medical entity recognition results, or medical event relationship extraction results.

[0069] It should be noted that a multilayer perceptron classifier with at least one hidden layer is used. By introducing nonlinear transformation capabilities through activation functions and combining them with the Softmax function to generate a probability distribution of structured information, this method can accurately output three types of high-value clinical information: automatic disease coding results, medical entity recognition results, and medical event relationship extraction results. This solves the problems of existing analysis methods having single output information and weak support for clinical decision-making. In particular, the output form of the probability distribution can intuitively reflect the credibility of the results, helping clinicians to quickly judge the reference value of the analysis results. The format design of the structured information (such as disease coding corresponding to the ICD-11 standard and entity relationships presented as triples) can be directly connected to hospital clinical decision support systems and HIS systems without the need for secondary manual processing. This significantly shortens the medical document analysis time (from 30-60 minutes / document to a few seconds / document), reducing the document processing burden on doctors. At the same time, the three types of structured information cover the needs of multiple scenarios such as clinical diagnosis, medical quality control, and clinical research, significantly improving the utilization efficiency of medical data and the auxiliary support capability for clinical decision-making.

[0070] For example, the 256-dimensional fused feature vector obtained from S4 is input into a multilayer perceptron classifier (containing one 128-dimensional hidden layer and a ReLU activation function): the hidden layer performs a nonlinear transformation on the feature vector to highlight the association features of "polyuria and polydipsia - type 2 diabetes" and "myocardial infarction - aspirin"; the output layer generates probability distributions for three types of structured information through a softmax function—automatic disease coding (ICD-11: E11.900 type 2 diabetes probability 0.98, I25.900 ischemic heart disease probability 0.92), medical... Entity recognition results ({symptoms: polydipsia and polyuria (confidence 0.95), disease: myocardial infarction (confidence 0.97), medication: aspirin (confidence 0.96)}), medical event relationship extraction (“patient-suffers-myocardial infarction”, “patient-takes-aspirin”, both with confidence 0.93); the final output is the structured information with the highest probability. Among them, the automatic disease coding results can be directly connected to the hospital's clinical decision-making system. The medical entity and relationship results help doctors quickly sort out the patient's condition, reducing the document analysis time from 30 minutes to 5 seconds.

[0071] In summary, this invention effectively overcomes the problems of insufficient attention to key information and limited semantic understanding in existing technologies by introducing medical conceptual entities as conceptual anchors to guide attention allocation and combining a hierarchical attention mechanism at the term and context levels. This significantly improves the accuracy of entity recognition, disease coding, and relation extraction in medical document analysis. Furthermore, by strengthening the deep integration of key entity features and global semantics, this method generates more discriminative feature representations. Because its attention mechanism is clearly linked to medical concepts, it greatly enhances the transparency and clinical credibility of the model's decision-making process, ultimately achieving accurate and interpretable intelligent analysis of medical documents.

[0072] Example 2, an embodiment of the present invention, provides a Bi-LSTM-based medical document analysis system, comprising: a data preprocessing module for acquiring the medical document to be analyzed, preprocessing it, and vectorizing it into word vectors to obtain a word vector sequence; an entity recognition module for identifying key medical concept entities from the word vector sequence using a medical entity recognition model, and defining their position indices as a set of concept anchor points; a network encoding module for inputting the word vector sequence into a Bi-LSTM network for encoding to obtain a hidden state sequence; a feature fusion module for dynamically calculating hierarchical attention weights based on the hidden state sequence and the set of concept anchor points, generating a weighted context vector, and fusing the key hidden states and the context vector to obtain a fused feature vector; and a classification output module for inputting the feature vector into a classifier and outputting structured information for assisting clinical diagnostic decisions.

[0073] Example 3 is an embodiment of the present invention, which differs from the previous embodiment in that: If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0075] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0076] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0077] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A medical document analysis method based on Bi-LSTM, characterized in that: include: The medical documents to be analyzed are obtained, preprocessed, and vectorized into word vectors to obtain a sequence of word vectors. A medical entity recognition model is used to identify key medical concept entities from the word vector sequence, and their position indices are defined as a set of concept anchor points. The word vector sequence is input into a Bi-LSTM network for encoding to obtain the hidden state sequence; Based on the hidden state sequence and concept anchor set, the hierarchical attention weights are dynamically calculated to generate a weighted context vector, and the key hidden states are fused with the context vector to obtain a fused feature vector. The feature vector is input into a classifier, which outputs structured information to assist in clinical diagnostic decision-making.

2. The medical document analysis method based on Bi-LSTM as described in claim 1, characterized in that: The medical concept entities include diseases, symptoms, drugs, and examination items; The hierarchical attention includes term-level attention for focusing on medical entities and context-level attention for understanding clinical semantics; The structured information includes one of the following: disease autocoding results, medical entity identification results, or medical event relationship extraction results.

3. The medical document analysis method based on Bi-LSTM as described in claim 2, characterized in that: The specific construction steps of the medical entity recognition model are as follows: Acquire text data in the medical field and construct an annotated corpus including medical concept entities; A pre-trained language model from the medical field was selected as the base model. The basic model is trained and adjusted on an annotated corpus to learn the mapping relationship between medical entity boundaries and types; The adjusted model is used as a medical entity recognition model to identify and output medical concept entities and their location indices from word vector sequences.

4. The medical document analysis method based on Bi-LSTM as described in claim 3, characterized in that: The specific details of encoding the word vector sequence are as follows: The word vector sequence is input into a bidirectional long short-term memory network; The forward LSTM layer of the Bi-LSTM network processes word vectors in sequence order to generate a forward hidden state sequence. The backward LSTM layer of the Bi-LSTM network processes word vectors in reverse order to generate a backward hidden state sequence. For each position in the sequence, the forward hidden state and the backward hidden state of the corresponding position are concatenated to form a complete hidden state containing bidirectional context information; The complete hidden states at all locations form a hidden state sequence, which is used for subsequent hierarchical attention calculations.

5. The medical document analysis method based on Bi-LSTM as described in claim 4, characterized in that: Based on the hidden state sequence and the set of concept anchors, the hierarchical attention weights are dynamically calculated to generate a weighted context vector, as detailed below: Based on the concept anchor set, key hidden states corresponding to medical concept entities are extracted from the hidden state sequence, and the similarity between each hidden state and the key hidden state is calculated to obtain term-level attention weights. Based on the entire hidden state sequence, the contribution of each hidden state to the overall clinical semantic understanding is calculated through an attention mechanism to obtain context-level attention weights; The term-level attention weights and context-level attention weights are weighted and fused to obtain the final hierarchical attention weights; The hidden state sequence is weighted and summed using hierarchical attention weights to generate a weighted context vector.

6. The medical document analysis method based on Bi-LSTM as described in claim 5, characterized in that: The process of fusing the key hidden state with the context vector to obtain the fused feature vector includes the following steps: Extract the key hidden states from the hidden state sequence that correspond to the position indices in the concept anchor set; The key hidden states are concatenated with the weighted context vector to form a preliminary fusion vector; The initial fusion vector is nonlinearly transformed through a fully connected layer to obtain the final fusion feature vector.

7. The medical document analysis method based on Bi-LSTM as described in claim 6, characterized in that: The classifier is a multilayer perceptron classifier, and its specific processing steps include: The fused feature vector is input into a multilayer perceptron that includes at least one hidden layer; Nonlinear transformation capability is introduced through activation functions; The softmax function is used in the output layer to generate probability distributions for various types of structured information; The final structured information is output based on the probability distribution, including automatic disease coding results, medical entity recognition results, or medical event relationship extraction results.

8. A medical document analysis system based on Bi-LSTM, based on the medical document analysis method based on Bi-LSTM as described in any one of claims 1 to 7, characterized in that: include, The data preprocessing module is used to acquire the medical documents to be analyzed, preprocess them, and convert them into word vectors to obtain a word vector sequence. The entity recognition module is used to identify key medical concept entities from word vector sequences using a medical entity recognition model, and define their position indices as a set of concept anchor points; The network encoding module is used to input word vector sequences into the Bi-LSTM network for encoding to obtain hidden state sequences; The feature fusion module is used to dynamically calculate hierarchical attention weights based on the hidden state sequence and concept anchor set, generate a weighted context vector, and fuse the key hidden states and context vectors to obtain a fused feature vector. The classification output module is used to input feature vectors into the classifier and output structured information to assist clinical diagnostic decisions.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the medical document analysis method based on Bi-LSTM as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the medical document analysis method based on Bi-LSTM as described in any one of claims 1 to 7.

Images