An electronic health record-based feature fusion lung cancer risk prediction method

Abstract

The application discloses a feature fusion lung cancer risk prediction method based on electronic health records, and belongs to the technical field of medical data analysis.The method comprises the following steps: extracting structured data features and unstructured text features, splicing a structured feature vector and a word vector matrix, performing time series modeling by using a GRU gate model, learning the weight and dependency relationship of the spliced vector, inputting a word-level attention pooling layer to capture the importance of different vectors in a feature matrix by using a gated attention mechanism, and the like.The application fully utilizes data information in electronic health records by fusing structured data and unstructured data at the feature level, improves the available feature dimension in electronic health records by more than 30%, establishes a deep interaction fusion mechanism between different data types, solves the data island problem, significantly reduces the omission of key information, and greatly enhances the generalization ability and prediction accuracy of the model.

Description

Technical Field

[0001] This invention relates to the field of medical data analysis technology, and in particular to a method for predicting lung cancer risk based on feature fusion of electronic health records. Background Technology

[0002] Lung cancer is currently the leading cause of cancer incidence and mortality in China and even globally, placing a heavy burden on patients' families and medical resources. Studies have shown that early screening and intervention for lung cancer can significantly improve patients' postoperative survival rate and quality of life.

[0003] Currently, the crucial data foundation for early lung cancer screening is the electronic health record of patients, which is widely used in medical institutions at all levels. This record mainly includes two types of data: structured data and unstructured text data.

[0004] The structured data mainly consists of PLCom2012 model variables, including demographic characteristics, blood test characteristics, smoking status, respiratory history and family cancer history. Existing technologies mostly use generalized linear models (such as logistic regression and Cox proportional hazards model) or ensemble learning algorithms (such as random forest and XGBoost) for feature selection and model training.

[0005] Unstructured data is patient information stored in text form, mainly consisting of patient complaints (such as patient medical history, allergy history, etc.), medical records (such as symptom descriptions, disease progress records), and LDCT image reports (such as nodule location, density, edge features, etc.). Current technical processes primarily employ two methods:

[0006] (1) Rule-driven coding: The disease field is converted into a structured label through the ICD-10 disease coding system. For example, “ground-glass nodule in the upper right lung (8mm)” is coded as R91.1 (pulmonary nodular lesion).

[0007] (2) Entity semantic parsing: Deep learning natural language processing models such as BiLSTM-CRF and BERT are used to extract text entity relationships. For example, the semantic association of "nodule size-location-malignancy probability" is identified through the attention mechanism.

[0008] It is worth noting that current technologies for processing unstructured data are still limited to text parsing, mainly establishing connections between data by extracting explicit structured information from unstructured text.

[0009] Potential correlations are prevalent among different features of electronic health records. However, current technologies heavily rely on single data sources, employing independent data processing pipelines for different data types. This results in data silos and the loss of feature interaction information. While existing traditional algorithms fully utilize both structured and unstructured data, they lack exploration of feature fusion methods for structured and unstructured text data. They only effectively utilize a portion of the dataset's information, failing to consider the hidden connections between different data and the characteristics of different data representations, leading to weak cross-modal correlations.

[0010] Therefore, there is an urgent need in this field for an improved method to enhance the accuracy and clinical interpretability of lung cancer risk prediction. Summary of the Invention

[0011] The purpose of this invention is to provide a lung cancer risk prediction method based on feature fusion of electronic health records to solve the above-mentioned problems.

[0012] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0013] A lung cancer risk prediction method based on feature fusion of electronic health records includes the following steps:

[0014] S1: Extract structured data features to obtain structured feature vectors;

[0015] S2: Extract unstructured text features to obtain a word vector matrix;

[0016] S3: Concatenate the structured feature vector obtained in step S1 with the word vector matrix obtained in step S2 to obtain the concatenated feature vector matrix;

[0017] S4: Use the GRU gated model for temporal modeling, learn the weights and dependencies of the concatenated vectors, and obtain the fused feature vector matrix;

[0018] S5: Input the fused feature vector matrix obtained in step S4 into the word-level attention pooling layer. The importance of different vectors in the feature matrix is ​​captured by the gating attention mechanism. The high-dimensional feature matrix is ​​aggregated to form a low-dimensional feature vector, and the fused feature vector is obtained.

[0019] S6: Input the fused feature vector obtained in step S5 into the fully connected classifier, and use the Softmax activation function to output the probability distribution result of the final lung cancer risk prediction.

[0020] As a preferred technical solution, the method for extracting structured data features in step S1 is as follows:

[0021] A multilayer perceptron (MLP) consisting of fully connected layers and ReLU activation layers is constructed to perform nonlinear mapping on structured data. The input is a structured numerical vector, and the original numerical features are mapped to a high-dimensional space using a fully connected projection network. The activation layer enhances the nonlinear expressive power and extracts structured feature vectors of 128 to 1024 dimensions.

[0022] As a preferred technical solution, the method for extracting unstructured text features in step S2 is as follows:

[0023] The pre-trained BERT Chinese model is used to obtain vector representations of free text. The BERT model consists of at least 6 stacked transformer encoder layers, which rely on multi-head attention to capture subtle dependencies between words and project textual relationships into a high-dimensional semantic space to transform them into feature vector representations. The BERT model has several variations, including stacked layers of 6, 12, and 24. The pre-trained model used in this invention is preferably a 12-layer stacked model.

[0024] As a further preferred technical solution, the specific method is as follows:

[0025] The preset length of the BERT model input is set to the maximum supported length of 512 dimensions. The input of the BERT model is in queue form. At each time step, the model inputs a word embedding and outputs the hidden layer state of the current time step as a local semantic representation of the text, represented as a 768-dimensional vector, i.e., the word vector representation of the text. The word vectors output at each time step are obtained to form a vector matrix. The vector matrix is: BERT output dimension × model preset length, which serves as the extracted unstructured feature matrix.

[0026] As a preferred technical solution, the specific method of step S3 is as follows:

[0027] The structured feature vector is copied to each time step dimension of the text feature matrix to form an extended vector with the same length as the text sequence; it is then concatenated with the unstructured feature matrix (i.e., the "word vector matrix") vector by vector at each time step to generate a fused feature matrix. The dimension of the fused feature matrix is: (structured feature vector + unstructured feature vector) × fused feature matrix of the preset length of the model.

[0028] As a preferred technical solution, in step S4...

[0029] The GRU gated model consists of at least two recurrent gated units. It automatically learns to retain important features from sequence information, captures the dynamic evolution of features in the sequence dimension, filters important time step information through the gating mechanism, and outputs a sequence representation containing contextual memory.

[0030] The input data of the GRU gated model is queue data. The GRU hidden layer dimension is set to 128 to 1024. The vectors in the concatenated feature matrix are input in the queue order to obtain the hidden layer output vector of the GRU gated model at each time step, forming a fused feature vector matrix. The dimension of the fused feature vector matrix is: GRU hidden layer dimension × model preset length.

[0031] As a preferred technical solution, in step S5, the attention pooling layer consists of a word-level attention network and a weighted pooling network, wherein...

[0032] The word-level attention network consists of two fully connected layers and a Tanh activation function. The input is the feature vector matrix output by GRU. The first fully connected layer generates an intermediate representation through linear mapping and the Tanh activation function. The intermediate dimension is smaller than the dimension of the GRU hidden layer. The second fully connected layer generates attention scores through unbiased linear transformation. Finally, Softmax normalization is applied to obtain the weight distribution at each time step. During training, the word-level attention network learns the feature weights at each time step and outputs a weight score vector corresponding to the feature vector at each time step.

[0033] That is, the MLP structure here has three layers: the first layer is a fully connected layer: GRU hidden layer -> intermediate dimension; the second layer is a Tanh activation layer (dimension unchanged); and the third layer is a fully connected layer: intermediate dimension -> 1 dimension (attention score). However, the first and second layers are generally merged into one layer.

[0034] The training method of the weighted pooling network (which is part of the model and is not trained separately) is as follows: the obtained weight score vector (each vector will get a weight score, so it is a weight score vector with a length equal to the matrix dimension) is multiplied again with the feature vector of the corresponding time step in the feature matrix output by GRU to obtain a weighted feature matrix. Then, the weighted feature vectors are added one by one to generate a fixed-dimensional full-text-level feature representation to obtain the final fused feature vector, which realizes the technical effect of dynamically focusing key diagnostic segments. Finally, a one-dimensional vector with a dimension equal to the hidden layer dimension of the GRU model is obtained.

[0035] As a preferred technical solution, in step S6...

[0036] End-to-end optimization is performed using the cross-entropy loss function, and parameters are adjusted through multiple rounds of backpropagation. The optimization objective is to minimize the difference between the predicted probability and the true label.

[0037] This invention clearly solves the core problems in the prior art, such as data fragmentation, coarse text processing, and lack of feature fusion methods, through the above-mentioned method. It is the first to realize deep interactive modeling of structured data and unstructured text, which significantly improves the accuracy and clinical interpretability of lung cancer risk prediction.

[0038] Compared with the method disclosed in CN117542515A, the present invention has at least the following advantages:

[0039] (1) In the unstructured text part, the present invention uses the Bert model to extract the word vector matrix. Compared with the static vocabulary obtained by the word2vec model, the Bert model introduces a multi-head attention mechanism to capture more detailed contextual relationships between different words and can dynamically project the text into a high-dimensional vector space.

[0040] (2) Structured features use MLP to obtain feature vectors, eliminating the need for additional feature selection. The model can autonomously adjust the importance weights of different features during training, making the model more widely applicable.

[0041] (3) The design of the feature fusion part effectively solves the problem of fusing sequence data with one-dimensional features. At the same time, by gating the fusion of sequence data, the data context and the dependencies between different types of features can be effectively captured. Furthermore, the probability distribution results can be obtained directly after fusion, and the classification function can be used as part of the model without the need to train an additional independent classification model.

[0042] Compared with the prior art, the advantages of the present invention are as follows: The present invention makes fuller use of the data information in electronic health records by fusing structured and unstructured data at the feature level, increasing the available feature dimensions in electronic health records by more than 30%, establishing a deep interactive fusion mechanism between different data types, solving the problem of data silos, significantly reducing the omission of key information, and at the same time, the rich data information greatly enhances the generalization ability and prediction accuracy of the model. Attached Figure Description

[0043] Figure 1 ROC curves for different models;

[0044] Figure 2 This is a schematic diagram of the prediction method according to an embodiment of the present invention;

[0045] Figure 3 This is a flowchart of the prediction method according to an embodiment of the present invention. Detailed Implementation

[0046] To explain the technical content, objectives, and effects of the present invention in detail, the following description, in conjunction with specific embodiments, further illustrates the content of the present invention.

[0047] Example 1

[0048] The privacy dataset used for training in this embodiment comes from the Health Management Center of West China Hospital, Sichuan University. After exclusion criteria, a total of 5257 sample records from Sichuan Province between 2010 and 2020 were included, of which 1307 were lung cancer cases. The dataset underwent cleaning and preprocessing processes such as integration, screening, dimensionality reduction, desensitization, standardization, and deduplication to remove sensitive patient privacy information. Missing values ​​in the structured data were grouped according to gender and lung cancer status, and then imputed with the mean. Group imputation was performed to reduce data bias caused by gender and disease status. Unstructured text was uniformly integrated and spliced. The preprocessed dataset contains 78 structured features and one integrated unstructured medical record text, covering patient samples aged 24-87 years.

[0049] A lung cancer risk prediction method based on feature fusion of electronic health records, the principle of which is as follows: Figure 2 As shown, the process is as follows: Figure 3 As shown, it includes the following steps:

[0050] S1: Extract structured data features to obtain structured feature vectors, specifically:

[0051] In structured data tables, patient information is stored in numerical form; therefore, each patient's structured electronic health record can be directly represented as a numerical vector. For example... Figure 2 Step 1 shows the construction of a multilayer perceptron (MLP) consisting of fully connected layers and ReLU activation layers to perform nonlinear mapping on structured data. The input is a structured numerical vector, and the original numerical features are mapped to a high-dimensional space using a fully connected projection network. The activation layer enhances the nonlinear expressive power and extracts a 256-dimensional structured feature vector.

[0052] S2: Extract unstructured text features to obtain a word vector matrix, specifically:

[0053] Unstructured data consists of patient information stored in Chinese text format. Each patient sample in the dataset contains multiple segments of unstructured text data. The preprocessing step concatenates these texts to form an integrated medical record description. Since free text data cannot be directly used in computer models, it requires vectorization to convert the text into numerical vectors that computers can understand. A pre-trained BERT Chinese model was chosen to obtain the vector representation of the free text. The BERT model consists of 12 stacked transformer encoder layers and relies on a unique multi-head attention mechanism to capture subtle dependencies between words, projecting these textual relationships into a high-dimensional semantic space and transforming them into feature vector representations.

[0054] The BERT model's input preprocessing involves segmenting the Chinese text into words, converting each word into a numerical representation in word embedding form, and padding it to a preset uniform length. In this study, the preset length is set to the maximum length supported by the BERT model: 512 dimensions. Figure 2 The input to the BERT model shown in step 2 is in the form of a queue. The model takes a word embedding as input at each time step and outputs the hidden layer state of the current time step as a local semantic representation of the text, which is represented as a 768-dimensional vector, that is, the word vector representation of the text. The word vectors output at each time step are obtained to form a vector matrix of BERT output dimension × model preset length, which is used as the extracted unstructured feature matrix.

[0055] S3: Concatenate the structured feature vector obtained in step S1 with the word vector matrix obtained in step S2 to obtain the concatenated feature vector matrix. Specifically:

[0056] To establish explicit associations between structured and unstructured features, a feature concatenation strategy is employed to construct a multimodal representation space. For example... Figure 2 Step 3 involves copying the structured feature vectors to each time step dimension of the text feature matrix, forming an extended vector with the same length as the text sequence. This extended vector is then concatenated with the unstructured feature matrix vector by vector at each time step to generate a fused feature matrix with dimensions of (structured feature vector + unstructured feature vector) × the model's preset length. This design preserves the spatial correspondence of clinical features in the text sequence and provides a unified feature space for subsequent time-series modeling.

[0057] S4: Temporal modeling is performed using a GRU gated model to learn the weights and dependencies of the concatenated vectors, resulting in a fused feature vector matrix. Specifically:

[0058] like Figure 2 As shown in step 4, the GRU gated model is used for time series modeling to learn the weights and dependencies of the concatenated vectors. The GRU gated model consists of multiple recurrent gated units, which can automatically learn from sequence information to retain important features, capture the dynamic evolution of features in the sequence dimension, filter important time step information through the gating mechanism, and output a sequence representation containing contextual memory.

[0059] The input to the GRU gated model is also queued data. The GRU hidden layer dimension is set to 512. The vectors in the concatenated feature matrix are input in the order of the queue to obtain the hidden layer output vector of the GRU gated model at each time step, forming a fused feature vector matrix with a dimension of GRU hidden layer dimension × model preset length.

[0060] S5: Input the fused feature vector matrix obtained in step S4 into the word-level attention pooling layer. The gating attention mechanism captures the importance of different vectors in the feature matrix, aggregating the high-dimensional feature matrix into a low-dimensional feature vector, thus obtaining the fused feature vector. Specifically:

[0061] like Figure 2 Step 5 shows the input of the fused feature matrix into the word-level attention pooling layer. This layer consists of a word-level attention network and a weighted pooling network. Through a gated attention mechanism, it captures the importance of different vectors in the feature matrix, aggregating the high-dimensional feature matrix into low-dimensional feature vectors, thereby concisely condensing effective feature information. Wherein:

[0062] The word-level attention network consists of two fully connected layers and a Tanh activation function. The input is the feature vector matrix output by the GRU. The first fully connected layer generates an intermediate representation through linear mapping and the Tanh activation function; the intermediate dimension can be smaller than the GRU hidden layer dimension (selectable range 128-1024 dimensions). The second fully connected layer generates the attention score through an unbiased linear transformation (GRU hidden layer dimension -> intermediate dimension -> 1; that is, the MLP structure here is three layers: first fully connected layer: GRU hidden layer -> intermediate dimension; second layer: Tanh activation layer (dimensionality unchanged); third fully connected layer: intermediate dimension -> 1 (attention score), but generally the first and second layers are combined into one layer). Finally, Softmax normalization is applied to obtain the weight distribution at each time step. During training, the word-level attention network learns the feature weights at each time step and outputs a weight score vector corresponding to the feature vector at each time step.

[0063] The training method for the weighted pooling network is as follows:

[0064] The obtained weight scores are multiplied again with the feature vectors of the corresponding time steps in the feature matrix output by GRU to obtain a weighted feature matrix. Then, the weighted feature vectors are added one by one to generate a fixed-dimensional full-text feature representation and obtain the final fused feature vector. This achieves the technical effect of dynamically focusing on key diagnostic segments and finally, a one-dimensional vector with a dimension equal to that of the GRU hidden layer is obtained.

[0065] S6: Input the fused feature vector obtained in step S5 into the fully connected classifier, use the Softmax activation function to output the probability distribution of the final lung cancer risk prediction, and use the cross-entropy loss function for end-to-end optimization of the model. Adjust the parameters through multiple rounds of backpropagation. The optimization objective is to minimize the difference between the predicted probability and the true label.

[0066] Example 2

[0067] Effect evaluation

[0068] Model performance is measured by precision, recall, F1 score, and AUC-ROC curve.

[0069] Precision is the percentage of samples correctly identified as having lung cancer by the model, reflecting the model's accuracy in predicting lung cancer patients. Recall is the proportion of lung cancer samples correctly identified by the model out of all lung cancer samples, reflecting the model's sensitivity in predicting lung cancer patients. The F1 score is the harmonic mean of precision and recall, a balanced indicator that considers both precision and recall, monitoring potential extreme cases caused by sample imbalance, and comprehensively reflecting the model's performance. The ROC curve (Receiver Operating Characteristic) is a graphical tool used to represent the classification performance of a model. It uses the true positive rate (TPR) and false positive rate (FPR) as the horizontal and vertical axes of an image to represent the model's performance at different thresholds. AUC is defined as the area under the ROC curve and is widely used for evaluating the performance of binary classification models.

[0070] To evaluate the actual performance of the feature fusion model in this embodiment, lung cancer risk prediction models were trained using both structured and unstructured data, serving as benchmarks to compare the performance of the model after feature fusion in this embodiment. Based on existing mature models, four high-performing machine learning algorithms—Random Forest (RF), SVM, Adaboost, and XGboost—were selected to train the model for structured data. For unstructured data, the well-performing benchmark model "BERT+BiLSTM" was used to construct the classification model. Furthermore, to investigate the impact of extended feature and deep feature fusion on model performance, the experiment also tested the performance metrics of extracting only structured and unstructured features, concatenating them, and then training the model using BiLSTM (i.e., "concatenation + BiLSTM") without further deep fusion.

[0071] The final metrics and AUC-ROC curves for each model are shown in Table 1 and 2. Figure 1 As shown.

[0072] Table 1. Indicators of different models

[0073] Accuracy Recall rate F1 AUC RF 0.8729 0.3771 0.5267 0.8915 SVM 0.7907 0.4869 0.6027 0.8876 Adaboost 0.7902 0.5394 0.6411 0.8841 XGboost 0.8306 0.6086 0.7025 0.9130 Bert+BiLSTM 0.8476 0.8114 0.8291 0.9420 splicing + BiLSTM 0.8833 0.8078 0.8439 0.9596 CN117542515A 0.85 0.82 0.83 0.95 OURS 0.9030 0.8612 0.8816 0.9723

[0074] Figure 1 In Table 1, "OURS" refers to the model of the prediction method of Embodiment 1 of the present invention; "CN117542515A" refers to the various indicators estimated according to the method disclosed in patent application CN117542515A.

[0075] From Table 1 and Figure 1It can be seen that, for single data sources, lung cancer screening models trained on structured data have relatively low overall performance, with XGboost being the best performing model, achieving a recall of 0.64 and an F1 score of 0.72. For unstructured data, BERT+BiLSTM performs better, achieving a recall of 0.81 and an F1 score of 0.82.

[0076] It is noteworthy that methods that simply concatenate structured and unstructured data already achieve better results than models using only one type of data, showing significant improvements across various metrics. This result validates the effectiveness of the strategy of expanding the available data types; that is, by integrating complementary information sources, the model can more comprehensively capture the key features of lung cancer risk prediction, while also demonstrating the inadequacy of existing models in utilizing data. Furthermore, the feature fusion model proposed in this study achieves a breakthrough in performance, with a recall of 0.86, an F1 score of 0.88, and an AUC of 0.97. These metrics not only surpass the performance of previous benchmark models but also significantly outperform simple data concatenation methods.

[0077] From a clinical practice perspective, an AUC-ROC curve close to the ideal value (0.97) indicates that the model possesses near-optimal classification capabilities. Specifically, even with a low false positive rate (FPR), the model maintains a high true positive rate (TPR), which is significant for early lung cancer screening: it minimizes the risk of missed diagnoses (high recall) while controlling overdiagnosis (reasonable false positive rate). This balanced characteristic makes the feature fusion model more practically valuable in clinical deployment and is expected to become a major technological breakthrough in the field of lung cancer screening. This invention can assist in the early detection and screening of high-risk groups for lung cancer, effectively helping patients with early intervention and treatment, and improving patient survival rates and quality of life. Optimizing medical resource allocation and public health value, the method of this invention can perform risk stratification in primary healthcare institutions using only structured data and text records, prioritizing the screening of high-risk groups, helping medical institutions at all levels better promote the construction of a tiered medical system, and effectively improving the allocation efficiency of social medical resources.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lung cancer risk prediction method based on feature fusion of electronic health records, characterized in that, Includes the following steps: S1: Extract structured data features to obtain structured feature vectors; S2: Extract unstructured text features to obtain a word vector matrix; S3: Concatenate the structured feature vector obtained in step S1 with the word vector matrix obtained in step S2 to obtain the concatenated feature vector matrix; S4: Use the GRU gated model for temporal modeling, learn the weights and dependencies of the concatenated vectors, and obtain the fused feature vector matrix; S5: Input the fused feature vector matrix obtained in step S4 into the word-level attention pooling layer. The importance of different vectors in the feature matrix is ​​captured by the gating attention mechanism. The high-dimensional feature matrix is ​​aggregated to form a low-dimensional feature vector, and the fused feature vector is obtained. S6: Input the fused feature vector obtained in step S5 into a fully connected classifier, and use the Softmax activation function to output the final probability distribution of lung cancer risk prediction; where, In step S2, the method for extracting unstructured text features is as follows: We chose to use the pre-trained BERT Chinese model to obtain the vector representation of the free text. The BERT model consists of at least 6 layers of transformer encoders stacked together. It relies on the multi-head attention mechanism to capture the subtle dependencies between words and projects the text relations into a high-dimensional semantic space to transform them into feature vector representations. The specific method for extracting unstructured text features is as follows: The preset length of the BERT model input is set to the maximum supported length of 512 dimensions. The input of the BERT model is in queue form. At each time step, the model inputs a word embedding and outputs the hidden layer state of the current time step as a local semantic representation of the text, represented as a 768-dimensional vector, i.e., the word vector representation of the text. The word vectors output at each time step are obtained to form a vector matrix. The vector matrix is: BERT output dimension × model preset length, which serves as the extracted unstructured feature matrix. In step S5, the attention pooling layer consists of a word-level attention network and a weighted pooling network, wherein... The word-level attention network consists of two fully connected layers and a Tanh activation function. The input is the feature vector matrix output by GRU. The first fully connected layer generates an intermediate representation through linear mapping and the Tanh activation function. The intermediate dimension is smaller than the dimension of the GRU hidden layer. The second fully connected layer generates attention scores through unbiased linear transformation. Finally, Softmax normalization is applied to obtain the weight distribution at each time step. During training, the word-level attention network learns the feature weights at each time step and outputs a weight score vector corresponding to the feature vector at each time step. The training method of the weighted pooling network is as follows: the obtained weight score vector is multiplied again with the feature vector of the corresponding time step in the feature matrix output by GRU to obtain a weighted feature matrix. Then, the weighted feature vectors are added one by one to generate a fixed-dimensional full-text-level feature representation to obtain the final fused feature vector, thereby achieving the technical effect of dynamically focusing on key diagnostic segments. Finally, a one-dimensional vector with a dimension equal to the dimension of the GRU hidden layer is obtained by fusion.

2. The method according to claim 1, characterized in that, In step S1, the method for extracting structured data features is as follows: A multilayer perceptron (MLP) consisting of fully connected layers and ReLU activation layers is constructed to perform nonlinear mapping on structured data. The input is a structured numerical vector, and the original numerical features are mapped to a high-dimensional space using a fully connected projection network. The activation layer enhances the nonlinear expressive power and extracts structured feature vectors of 128 to 1024 dimensions.

3. The method according to claim 1, characterized in that, The specific method for step S3 is as follows: The structured feature vector is copied to each time step dimension of the text feature matrix to form an extended vector with the same length as the text sequence; it is then concatenated with the unstructured feature matrix vector by vector at each time step to generate a fused feature matrix. The dimension of the fused feature matrix is: (structured feature vector + unstructured feature vector) × fused feature matrix of preset model length.

4. The method according to claim 1, characterized in that, In step S4, The GRU gated model consists of at least two recurrent gated units. It automatically learns to retain important features from sequence information, captures the dynamic evolution of features in the sequence dimension, filters important time step information through the gating mechanism, and outputs a sequence representation containing contextual memory. The input data of the GRU gated model is queue data. The GRU hidden layer dimension is set to 128 to 1024. The vectors in the concatenated feature matrix are input in the queue order to obtain the hidden layer output vector of the GRU gated model at each time step, forming a fused feature vector matrix. The dimension of the fused feature vector matrix is: GRU hidden layer dimension × model preset length.

5. The method according to claim 1, characterized in that, In step S6, End-to-end optimization is performed using the cross-entropy loss function, and parameters are adjusted through multiple rounds of backpropagation. The optimization objective is to minimize the difference between the predicted probability and the true label.

Images