Feature fusion lung cancer risk prediction method based on electronic health record

Abstract

The invention discloses a feature fusion lung cancer risk prediction method based on electronic health records, and belongs to the technical field of medical data analysis. The method comprises the following steps: extracting structured data features and unstructured text features, splicing structured feature vectors and a word vector matrix, carrying out time sequence modeling by utilizing a GRU gating model, learning the weight and dependency relationship of the spliced vectors, and carrying out time sequence modeling by utilizing a GRU gating model. The input word-level attention pooling layer captures importance and the like of different vectors in the feature matrix through a gating attention mechanism; according to the method, the data information in the electronic health record is more fully utilized through feature level fusion of the structured data and the unstructured data, the available feature dimension in the electronic health record is improved by more than 30%, a deep interactive fusion mechanism between different data types is established, the problem of data islands is solved, and the user experience is improved. Omission of key information is remarkably reduced, and meanwhile, the generalization ability and prediction precision of the model are greatly enhanced through rich data information.

Description

Technical Field

[0001] The present invention relates to the technical field of medical data analysis, and in particular to a lung cancer risk prediction method based on feature fusion of electronic health records. Background Art

[0002] Lung cancer is currently the malignant tumor with the highest morbidity and mortality in China and even the world, placing a heavy burden on patients' families and medical resources. Studies have shown that early screening and intervention of lung cancer can significantly improve patients' postoperative survival rate and quality of life.

[0003] Currently, the important data cornerstone for early lung cancer screening is the patient electronic health records widely used in medical institutions at all levels, which mainly include two types of data: structured data and unstructured text data.

[0004] Among them, structured data is mainly composed of PLCOm2012 model variables, including demographic characteristics, blood test characteristics, smoking status, respiratory disease history, and family cancer history. Existing technologies mostly use generalized linear models (such as logistic regression and Cox proportional hazard model) or ensemble learning algorithms (such as random forest and XGBoost) for feature selection and model training.

[0005] Unstructured data refers to patient information stored in text format, primarily consisting of patient complaints (such as medical history and allergy history), medical records (such as symptom descriptions and course of disease), and LDCT imaging reports (such as nodule location, density, and edge characteristics). Existing technical processes primarily employ two approaches: (1) Rule-driven coding: The disease field is converted into a structured label using the ICD-10 disease coding system. For example, “ground-glass nodule (8 mm) in the right upper lung” is coded as R91.1 (pulmonary nodular lesions). (2) Entity semantic parsing: Using deep learning natural language processing models such as BiLSTM-CRF and BERT to extract text entity relationships, for example, using the attention mechanism to identify the semantic association of "nodule size-location-malignancy probability"; It is worth noting that the existing technology for processing unstructured data still remains at the text parsing level, mainly establishing connections between data by extracting explicit structured information from unstructured text.

[0006] Potential connections are common between different features in electronic health records. However, current technologies rely heavily on a single data source, with different data types using independent data processing pipelines. This leads to widespread data silos between data streams, resulting in loss of interactive information. While existing traditional algorithms fully utilize both structured and unstructured data, they lack methods for fusing features from structured and unstructured text data. These algorithms effectively utilize only a portion of the dataset's information, neglecting the hidden connections between different data and the unique characteristics of different data representations, resulting in weak cross-modal connections.

[0007] Therefore, there is an urgent need in this field for an improved method to enhance the accuracy and clinical interpretability of lung cancer risk prediction. Summary of the Invention

[0008] The purpose of the present invention is to provide a lung cancer risk prediction method based on feature fusion of electronic health records to solve the above problems.

[0009] In order to achieve the above object, the technical solution adopted by the present invention is as follows: A lung cancer risk prediction method based on feature fusion of electronic health records, comprising the following steps: S1: Extract structured data features and obtain structured feature vectors; S2: Extract unstructured text features and obtain word vector matrix; S3: Concatenate the structured feature vector obtained in step S1 with the word vector matrix obtained in step S2 to obtain a concatenated feature vector matrix; S4: Use the GRU gating model for time series modeling, learn the weights and dependencies of the concatenated vectors, and obtain the fused feature vector matrix; S5: Input the fused feature vector matrix obtained in step S4 into the word-level attention pooling layer. The gated attention mechanism is used to capture the importance of different vectors in the feature matrix, aggregate the high-dimensional feature matrix into a low-dimensional feature vector, and obtain the fused feature vector. S6: Input the fused feature vector obtained in step S5 into the fully connected classifier, and use the Softmax activation function to output the probability distribution result of the final lung cancer risk prediction.

[0010] As a preferred technical solution, in step S1, the method for extracting structured data features is: A multi-layer perceptron (MLP) consisting of a fully connected layer and a RELU activation layer is constructed to perform nonlinear mapping on structured data. A structured numerical vector is input, and the original numerical features are mapped to a high-dimensional space using a fully connected projection network. The activation layer enhances the nonlinear expression ability and extracts a structured feature vector of 128 to 1024 dimensions.

[0011] As a preferred technical solution, in step S2, the method for extracting unstructured text features is: We chose to use the pre-trained BERT Chinese model to obtain vector representations of free text. The BERT model consists of at least six layers of stacked transformer encoders. It relies on a multi-head attention mechanism to capture detailed dependencies between words and project textual relationships into a high-dimensional semantic space, converting them into feature vector representations. The BERT model has multiple variants, including stacks of 6, 12, or 24 layers. The preferred pre-trained model used in this paper is a 12-layer stack.

[0012] As a further preferred technical solution, the specific method is: The preset input length of the Bert model is set to the maximum length supported by the Bert model, 512 dimensions. The input of the Bert model is in queue form. The model inputs a word embedding at each time step and outputs the hidden layer state of the current time step as the local semantic representation of the text, expressed as a 768-dimensional vector form, that is, the word vector representation of the text; the word vector output at each time step is obtained to form a vector matrix, which is: Bert output dimension × model preset length, as the extracted unstructured feature matrix.

[0013] As a preferred technical solution, the specific method of step S3 is: The structured feature vector is copied to each time step dimension of the text feature matrix to form an extended vector consistent with the length of the text sequence; it is spliced ​​vector by vector with the unstructured feature matrix (i.e., "word vector matrix") according to the time step to generate a fused feature matrix. The dimension of the fused feature matrix is: (structured feature vector + unstructured feature vector) × fused feature matrix of the preset length of the model.

[0014] As a preferred technical solution, in step S4, The GRU gating model consists of at least two recurrent gating units, which automatically learns to retain important features from sequence information, captures the dynamic evolution of features in the sequence dimension, filters important time step information through the gating mechanism, and outputs a sequence representation containing context memory; The input data of the GRU gating model is queue data. The GRU hidden layer dimension is set to 128 to 1024 dimensions. The vectors in the spliced ​​feature matrix are input in queue order to obtain the hidden layer output vector of the GRU gating model at each time step to form a fused feature vector matrix. The dimension of the fused feature vector matrix is: GRU hidden layer dimension × model preset length.

[0015] As a preferred technical solution, in step S5, the attention pooling layer is composed of a word-level attention network and a weighted pooling network, wherein, The word-level attention network consists of two fully connected layers and a Tanh activation function; the input is the feature vector matrix output by the GRU. The first fully connected layer generates an intermediate representation through linear mapping and the Tanh activation function. The intermediate dimension is smaller than the GRU hidden layer dimension. The second fully connected layer generates an attention score through an unbiased linear transformation. Finally, Softmax normalization is applied to obtain the weight distribution of each time step. During the training process, the word-level attention network will learn the feature weight of each time step and output a weight score vector corresponding to the feature vector of each time step. That is, the MLP structure here is three layers: the first fully connected layer: GRU hidden layer -> intermediate dimension, the second layer: Tanh activation layer (dimension unchanged), the third fully connected layer: intermediate dimension -> 1 dimension (attention score), but the first and second layers are generally combined as one layer; The training method of the weighted pooling network (which is part of the model and not trained separately) is: multiplying the obtained weight score vector (each vector will obtain a weight score, so it is a weight score vector with a length equal to the matrix dimension) with the feature vector of the corresponding time step in the feature matrix output by the GRU to obtain a weighted feature matrix, and then adding the weighted feature vectors one by one to generate a full-text feature representation of a fixed dimension to obtain the final fused feature vector, thereby achieving the technical effect of dynamically focusing on key diagnostic fragments, and finally fusion to obtain a one-dimensional vector with a dimension equal to the hidden layer dimension of the GRU model.

[0016] As a preferred technical solution, in step S6, The cross entropy loss function is used for end-to-end optimization, and the parameters are adjusted through multiple rounds of back propagation. The optimization goal is to minimize the difference between the predicted probability and the true label.

[0017] Through the above method, the present invention clearly solves the core problems of the existing technology, such as data fragmentation, rough text processing, and lack of feature fusion methods. It realizes the deep interactive modeling of structured data and unstructured text for the first time, significantly improving the accuracy and clinical interpretability of lung cancer risk prediction.

[0018] Compared with the method disclosed in CN117542515A, the present invention has at least the following advantages: (1) In the unstructured text part, the present invention uses the Bert model to extract the word vector matrix. Compared with the static word list obtained by the word2vec model, the Bert model introduces a multi-head attention mechanism to capture more detailed contextual relationships between different words and can dynamically project the text into a high-dimensional vector space; (2) The structured features use MLP to obtain feature vectors, which does not require additional design for feature selection. The model can autonomously adjust the importance weights of different features during training, making the model more applicable. (3) The design of the feature fusion part effectively solves the problem of fusing sequence data with one-dimensional features. At the same time, through gated fusion of sequence data, it can effectively capture the data context and the dependencies between different types of features. Moreover, after fusion, the probability distribution result can be directly obtained, and the classification function is used as part of the model, eliminating the need to train an independent classification model.

[0019] Compared with the existing technology, the advantages of the present invention are: the present invention makes more full use of the data information in electronic health records by fusing structured data and unstructured data at the feature level, increases the available feature dimensions in electronic health records by more than 30%, establishes a deep interactive fusion mechanism between different data types, solves the data island problem, and significantly reduces the omission of key information. At the same time, the rich data information greatly enhances the generalization ability and prediction accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is the ROC curve of different models; Figure 2 Schematic diagram of a prediction method according to an embodiment of the present invention; Figure 3 Flowchart of a prediction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the present invention is further described below with reference to specific embodiments.

[0022] Example 1 The privacy dataset used for training in this example comes from the Health Management Center of West China Hospital, Sichuan University. After the inclusion criteria, a total of 5,257 sample records were included in Sichuan Province from 2010 to 2020, including 1,307 cases of lung cancer. The dataset underwent a cleaning and preprocessing process including integration, screening, dimensionality reduction, desensitization, standardization, and deduplication to remove the patients' sensitive privacy information. Missing values ​​in the structured data were grouped according to gender and lung cancer status, and then mean-filled. Group filling was used to reduce data bias caused by gender and disease status. Unstructured text was uniformly integrated and spliced. The preprocessed dataset contains 78 structured features and 1 segment of integrated unstructured medical record text, covering patient samples aged 24-87.

[0023] A feature fusion method for lung cancer risk prediction based on electronic health records, the principle of which is as follows Figure 2As shown, the process is as Figure 3 As shown, the following steps are included: S1: Extract structured data features and obtain structured feature vectors. Specifically: Patient information is stored in numerical form in structured data tables, so each patient's structured electronic health record can be directly represented as a numerical vector. Figure 2 As shown in step 1, a multi-layer perceptron (MLP) consisting of a fully connected layer and a RELU activation layer is constructed to perform nonlinear mapping on the structured data. A structured numerical vector is input, and the original numerical features are mapped to a high-dimensional space using a fully connected projection network. The activation layer enhances the nonlinear expression capability and extracts a 256-dimensional structured feature vector.

[0024] S2: Extract unstructured text features and obtain word vector matrix. Specifically: Unstructured data refers to patient information stored in the form of Chinese text. Each patient sample in the dataset contains multiple segments of unstructured text data. The preprocessing process concatenates these texts into an integrated medical record description. Data in free text form cannot be directly put into computer models for use, so vectorization is required to convert the text into a numerical vector form that the computer can understand. The pre-trained Bert Chinese model is chosen to obtain a vector representation of free text. The Bert model consists of a 12-layer stack of transformer encoders. Relying on a unique multi-head attention mechanism, it can capture detailed dependencies between words and project text relationships into a high-dimensional semantic space to convert them into feature vector representations. The BERT model input preprocessing process will split the Chinese text into words, convert each word into a numerical representation in the form of word embedding, and fill it to a preset uniform length. In this study, the model preset length is set to the maximum length supported by the BERT model, 512 dimensions; Figure 2 The input of the Bert model shown in step 2 is in queue form. The model inputs a word embedding at each time step and outputs the hidden layer state of the current time step as the local semantic representation of the text, expressed as a 768-dimensional vector form, that is, the word vector representation of the text; the word vector output at each time step is obtained to form a vector matrix of Bert output dimension × model preset length, which is used as the extracted unstructured feature matrix.

[0025] S3: Concatenate the structured feature vector obtained in step S1 with the word vector matrix obtained in step S2 to obtain a concatenated feature vector matrix. Specifically: In order to establish an explicit association between structured features and unstructured features, a feature splicing strategy is used to construct a multimodal representation space. Figure 2As shown in step 3, the structured feature vector is copied to each time-step dimension of the text feature matrix, forming an extended vector consistent with the length of the text sequence. It is then concatenated with the unstructured feature matrix vector by vector, time-step by time-step, to generate a fused feature matrix with dimensions of (structured feature vector + unstructured feature vector) × the model's preset length. This design preserves the spatial correspondence of clinical features in the text sequence while providing a unified feature space for subsequent time series modeling.

[0026] S4: Use the GRU gating model for time series modeling, learn the weights and dependencies of the spliced ​​vectors, and obtain the fused feature vector matrix. Specifically: like Figure 2 As shown in step 4, the GRU gating model is used for time series modeling to learn the weights and dependencies of the splicing vectors. The GRU gating model consists of multiple recurrent gating units and can automatically train and learn to retain important features from sequence information, capture the dynamic evolution of features in the sequence dimension, filter important time step information through the gating mechanism, and output a sequence representation containing context memory; The input of the GRU gating model is also queue data. The GRU hidden layer dimension is set to 512 dimensions. The vectors in the spliced ​​feature matrix are input in queue order. The hidden layer output vector of the GRU gating model at each time step is obtained to form the fused feature vector matrix, whose dimension is the GRU hidden layer dimension × the preset model length.

[0027] S5: Input the fused feature vector matrix obtained in step S4 into the word-level attention pooling layer. The gated attention mechanism is used to capture the importance of different vectors in the feature matrix, aggregate the high-dimensional feature matrix into a low-dimensional feature vector, and obtain the fused feature vector. Specifically: like Figure 2 As shown in step 5, the fused feature matrix is ​​input into the word-level attention pooling layer. The attention pooling layer consists of a word-level attention network and a weighted pooling network. The gated attention mechanism is used to capture the importance of different vectors in the feature matrix, and the high-dimensional feature matrix is ​​aggregated into a low-dimensional feature vector, thereby streamlining and concentrating the effective feature information. The word-level attention network consists of two fully connected layers and a Tanh activation function. The input is the feature vector matrix output by the GRU. The first fully connected layer generates an intermediate representation through linear mapping and the Tanh activation function. The intermediate dimension can be smaller than the GRU hidden layer dimension (the optional range is 128-1024 dimensions). The second fully connected layer generates an attention score through an unbiased linear transformation (GRU hidden layer dimension -> intermediate dimension -> 1; that is, the MLP structure here is three layers: the first fully connected layer: GRU hidden layer -> intermediate dimension, the second layer: Tanh activation layer (dimension remains unchanged), the third fully connected layer: intermediate dimension -> 1 (attention score), but the first and second layers are generally combined and regarded as one layer). Finally, Softmax normalization is applied to obtain the weight distribution of each time step. During the training process, the word-level attention network will learn the feature weight of each time step and output the weight score vector corresponding to the feature vector of each time step. The training method of the weighted pooling network is: The obtained weight score is multiplied again with the feature vector of the corresponding time step in the feature matrix output by the GRU to obtain a weighted feature matrix. The weighted feature vectors are then added one by one to generate a full-text feature representation of a fixed dimension to obtain the final fused feature vector, thereby achieving the technical effect of dynamically focusing on key diagnostic fragments. Finally, a one-dimensional vector with a dimension equal to the GRU hidden layer dimension is obtained by fusion.

[0028] S6: The fused feature vector obtained in step S5 is input into the fully connected classifier, and the Softmax activation function is used to output the probability distribution result of the final lung cancer risk prediction. The model uses the cross-entropy loss function (Cross-Entropy Loss) for end-to-end optimization. The parameters are adjusted through multiple rounds of backpropagation. The optimization goal is to minimize the difference between the predicted probability and the true label.

[0029] Example 2 Effect evaluation The model effect is measured by precision, recall, F1 score and AUC-ROC curve.

[0030] Precision is the correct rate of samples identified by the model as having lung cancer, reflecting the model's accuracy in predicting lung cancer patients. Recall is the proportion of lung cancer samples correctly identified by the model to all lung cancer samples, reflecting the model's sensitivity in predicting lung cancer patients. The F1 score is the harmonic mean of precision and recall, which takes a balanced approach to both precision and recall. It can monitor extreme situations that may arise due to sample imbalance and comprehensively reflects the performance of the model. The ROC curve (receiver operating characteristic curve) is a graphical tool used to represent the classification performance of a model. It uses the true positive rate (TPR) and false positive rate (FPR) as the horizontal and vertical coordinates of the image to show the performance of the model at different thresholds. The AUC is defined as the area under the ROC curve and is widely used to evaluate the effectiveness of binary classification models.

[0031] To evaluate the actual performance of the feature fusion model of this embodiment, lung cancer risk prediction models were trained using structured and unstructured data, respectively, as a benchmark for comparing the performance of the model after feature fusion in this embodiment. Based on established models from existing research, four high-performing machine learning algorithms, Random Forest (RF), Support Vector Machine (SVM), Adaboost, and XGboost, were selected for training the model for structured data. For unstructured data, the well-performing benchmark model, Bert+BiLSTM, was selected to construct the classification model. Furthermore, to investigate the impact of extended features and deep feature fusion on model performance, the experiment also tested the performance indicators of extracting only structured and unstructured features, concatenating them, and then training the model using BiLSTM (i.e., "concatenation + BiLSTM"), without further fusion.

[0032] The indicators and AUC-ROC curves of each model finally obtained are shown in Table 1 and Figure 1 shown.

[0033] Table 1 Indicators of different models Precision Recall F1 AUC RF 0.8729 0.3771 0.5267 0.8915 SVM 0.7907 0.4869 0.6027 0.8876 Adaboost 0.7902 0.5394 0.6411 0.8841 XGboost 0.8306 0.6086 0.7025 0.9130 Bert+BiLSTM 0.8476 0.8114 0.8291 0.9420 Concat+BiLSTM 0.8833 0.8078 0.8439 0.9596 CN117542515A 0.85 0.82 0.83 0.95 OURS 0.9030 0.8612 0.8816 0.9723 Figure 1 In Table 1, "OURS" refers to the model of the prediction method of Example 1 of the present invention; "CN117542515A" refers to the various indicators estimated according to the method disclosed in patent application CN117542515A.

[0034] From Table 1 and Figure 1 As can be seen, the overall performance of the lung cancer screening model trained on structured data from a single data source is relatively low. The best performing model is XGboost, achieving a recall of 0.64 and an F1 score of 0.72. On the other hand, the Bert+BiLSTM model performs better on unstructured data, achieving a recall of 0.81 and an F1 score of 0.82.

[0035] Notably, the approach of simply concatenating structured and unstructured data achieved superior results compared to models using only one type of data, with significant improvements across various metrics. This result validates the effectiveness of the strategy of expanding available data types, namely, by integrating complementary information sources, the model can more comprehensively capture key features for lung cancer risk prediction. It also demonstrates the inadequate data utilization of existing models. The feature fusion model proposed in this study achieved breakthrough performance, with a recall rate of 0.86, an F1 score of 0.88, and an AUC of 0.97. These metrics not only surpass the performance of previous baseline models, but also significantly outperform simple data concatenation methods.

[0036] From a clinical perspective, the AUC-ROC curve's near-ideal performance (0.97) indicates that the model possesses near-optimal classification capabilities. Specifically, while maintaining a low false positive rate (FPR), the model maintains a high true positive rate (TPR), which is crucial for early lung cancer screening: it minimizes the risk of missed diagnosis (high recall rate) while also controlling overdiagnosis (reasonable false positive rate). This balanced approach makes the feature fusion model highly practical in clinical deployment and is expected to be a significant technological breakthrough in lung cancer screening. This invention can facilitate early detection and screening of individuals at high risk for lung cancer, effectively assisting patients with early intervention and treatment, and improving their survival rate and quality of life. This method optimizes medical resource allocation and public health benefits. Using structured data and text records alone, the method can be used in primary healthcare institutions to perform risk stratification, prioritizing screening for high-risk individuals. This approach can help healthcare institutions at all levels better advance the development of a tiered healthcare system and effectively improve the efficiency of healthcare resource allocation.

[0037] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A lung cancer risk prediction method based on feature fusion of electronic health records, characterized in that: The steps include: S1: Extract structured data features and obtain structured feature vectors; S2: Extract unstructured text features and obtain word vector matrix; S3: Concatenate the structured feature vector obtained in step S1 with the word vector matrix obtained in step S2 to obtain a concatenated feature vector matrix; S4: Use the GRU gating model for time series modeling, learn the weights and dependencies of the concatenated vectors, and obtain the fused feature vector matrix; S5: Input the fused feature vector matrix obtained in step S4 into the word-level attention pooling layer. The gated attention mechanism is used to capture the importance of different vectors in the feature matrix, aggregate the high-dimensional feature matrix into a low-dimensional feature vector, and obtain the fused feature vector. S6: Input the fused feature vector obtained in step S5 into the fully connected classifier, and use the Softmax activation function to output the probability distribution result of the final lung cancer risk prediction.

2. The method according to claim 1, characterized in that In step S1, the method for extracting structured data features is: A multi-layer perceptron (MLP) consisting of a fully connected layer and a RELU activation layer is constructed to perform nonlinear mapping on structured data. A structured numerical vector is input, and the original numerical features are mapped to a high-dimensional space using a fully connected projection network. The activation layer enhances the nonlinear expression ability and extracts a structured feature vector of 128 to 1024 dimensions.

3. The method according to claim 1, characterized in that In step S2, the method for extracting unstructured text features is: We chose to use the pre-trained BERT Chinese model to obtain vector representations of free text. The BERT model consists of at least 6 layers of transformer encoder stacks. It relies on a multi-head attention mechanism to capture detailed dependencies between words and projects text relationships into a high-dimensional semantic space to convert them into feature vector representations.

4. The method according to claim 3, characterized in that The specific method for extracting unstructured text features is: The preset input length of the Bert model is set to the maximum length supported by the Bert model, 512 dimensions. The input of the Bert model is in queue form. The model inputs a word embedding at each time step and outputs the hidden layer state of the current time step as the local semantic representation of the text, expressed as a 768-dimensional vector form, that is, the word vector representation of the text; the word vector output at each time step is obtained to form a vector matrix, which is: Bert output dimension × model preset length, as the extracted unstructured feature matrix.

5. The method according to claim 1, wherein The specific method of step S3 is: The structured feature vector is copied to each time step dimension of the text feature matrix to form an extended vector consistent with the length of the text sequence; and it is spliced ​​vector by vector with the unstructured feature matrix according to the time step to generate a fused feature matrix. The dimension of the fused feature matrix is: (structured feature vector + unstructured feature vector) × fused feature matrix of the preset length of the model.

6. The method according to claim 1, characterized in that In step S4, The GRU gating model consists of at least two recurrent gating units, which automatically learns to retain important features from sequence information, captures the dynamic evolution of features in the sequence dimension, filters important time step information through the gating mechanism, and outputs a sequence representation containing context memory; The input data of the GRU gating model is queue data. The GRU hidden layer dimension is set to 128 to 1024 dimensions. The vectors in the spliced ​​feature matrix are input in queue order to obtain the hidden layer output vector of the GRU gating model at each time step to form a fused feature vector matrix. The dimension of the fused feature vector matrix is: GRU hidden layer dimension × model preset length.

7. The method according to claim 1, characterized in that In step S5, the attention pooling layer is composed of a word-level attention network and a weighted pooling network, wherein, The word-level attention network consists of two fully connected layers and a Tanh activation function; the input is the feature vector matrix output by the GRU. The first fully connected layer generates an intermediate representation through linear mapping and the Tanh activation function. The intermediate dimension is smaller than the GRU hidden layer dimension. The second fully connected layer generates an attention score through an unbiased linear transformation. Finally, Softmax normalization is applied to obtain the weight distribution of each time step. During the training process, the word-level attention network will learn the feature weight of each time step and output a weight score vector corresponding to the feature vector of each time step. The training method of the weighted pooling network is as follows: the obtained weight score vector is re-multiplied by the feature vector of the corresponding time step in the feature matrix output by the GRU to obtain a weighted feature matrix, and then the weighted feature vectors are added one by one to generate a full-text feature representation of a fixed dimension to obtain a final fused feature vector, thereby achieving the technical effect of dynamically focusing on key diagnostic fragments, and finally fusing to obtain a one-dimensional vector with a dimension equal to the dimension of the GRU hidden layer.

8. The method according to claim 1, characterized in that In step S6, The cross entropy loss function is used for end-to-end optimization, and the parameters are adjusted through multiple rounds of back propagation. The optimization goal is to minimize the difference between the predicted probability and the true label.

Images