Respiratory system risk prediction method and system based on graph neural network

Abstract

The invention relates to the technical field of respiratory system risk prediction, and provides a respiratory system risk prediction method and system based on a graph neural network, and the method comprises the steps: collecting the multi-modal medical data of a patient, and constructing a multilayer heterogeneous graph based on the multi-modal medical data; constructing a weighted adjacency matrix and a node feature vector through the multi-layer heterogeneous graph; matrix product operation and convolution operation are carried out based on the weighted adjacent matrix and the node feature vector, splicing combination with historical moment state information is carried out, graph state representation is obtained, weighted aggregation of time dimensions is carried out, and time sequence attention features are obtained; performing coding processing based on the clinical examination data to obtain multi-modal fusion features; and inputting the multi-modal fusion features into a risk classifier for classification calculation to obtain a respiratory system risk level prediction result, generating a risk assessment report, and outputting respiratory risk early warning information. The accuracy and clinical practicability of respiratory system risk prediction are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of respiratory system risk prediction, and in particular to a respiratory system risk prediction method and system based on a graph neural network. BACKGROUND

[0002] Respiratory system risk prediction refers to a medical technology for evaluating and warning the possibility of future respiratory system diseases or disease exacerbation of a patient by analyzing multi-dimensional information such as clinical data, imaging data, laboratory indicators, and lung function test results. Compared with traditional single-index evaluation or simple scoring systems, modern respiratory system risk prediction usually needs to consider clinical examination data, chest imaging data, blood gas analysis, lung function tests and other multi-modal medical data. These data have significant differences in data structure, time characteristics, semantic expression and clinical significance. With the fusion of such multi-modal data, the risk prediction system can maximize the complementary advantages of different data sources, such as capturing structural lesions, functional abnormalities and physiological indicator changes in complex cases, and has stronger prediction accuracy, better clinical applicability and wider disease coverage. However, multi-modal respiratory system risk prediction also brings additional challenges, the key of which is how to effectively fuse heterogeneous medical data, handle time-varying dynamics and achieve interpretable risk assessment results.

[0003] In the prior art, respiratory system risk prediction mainly uses traditional machine learning methods, static feature fusion strategies and linear regression models to achieve basic risk assessment and warning functions. However, traditional methods are difficult to handle complex nonlinear relationships and high-dimensional feature interactions, static fusion cannot adapt to dynamic changes in physiological state, linear models are difficult to capture complex association patterns between multi-modal data, and lack modeling ability for time-dependent relationships and clinical interpretability, which makes existing systems perform poorly when faced with complex cases, especially in handling multi-modal data fusion, time-varying dynamic modeling and personalized risk assessment, and it is difficult to achieve accurate and reliable respiratory system risk prediction. SUMMARY

[0004] Therefore, the present application proposes a respiratory system risk prediction method and system based on a graph neural network, which solves the problems that the prior art is difficult to handle complex nonlinear relationships and high-dimensional feature interactions, uses static fusion which cannot adapt to dynamic changes in physiological state, uses linear models which are difficult to capture complex association patterns between multi-modal data, and lack modeling ability for time-dependent relationships and clinical interpretability, and performs poorly in handling multi-modal data fusion, time-varying dynamic modeling and personalized risk assessment, and it is difficult to achieve accurate and reliable respiratory system risk prediction.

[0005] The technical scheme of the present application is implemented as follows: In a first aspect, the present application provides a respiratory system risk prediction method based on a graph neural network, comprising the following steps: Collecting multi-modal medical data of a patient, and constructing a multi-layer heterogeneous graph based on the multi-modal medical data; Constructing a weighted adjacency matrix and a node feature vector through clinical feature similarity, image feature similarity, and time series correlation between nodes in the multi-layer heterogeneous graph; Performing matrix multiplication and convolution operations based on the weighted adjacency matrix and the node feature vector, and combining with historical time state information to obtain a graph state representation at the current time, and performing weighted aggregation in the time dimension to obtain a time series attention feature; Encoding and processing the clinical examination data through a clinical feature encoder to obtain a hidden representation of the clinical examination data, and combining with the time series attention feature for splicing and combination and latent space mapping to obtain a common latent feature, and performing distribution alignment processing on the common latent feature to obtain a multi-modal fusion feature; Inputting the multi-modal fusion feature into a risk classifier for classification calculation to obtain a respiratory system risk level prediction result, generating a risk assessment report, and outputting respiratory risk warning information.

[0006] Based on the above technical scheme, preferably, the encoding and processing of the clinical examination data through the clinical feature encoder to obtain the hidden representation of the clinical examination data, and the splicing and combination with the time series attention feature for splicing and combination and latent space mapping to obtain the common latent feature, and the distribution alignment processing on the common latent feature to obtain the multi-modal fusion feature, comprise: Constructing a multi-layer perceptron clinical feature encoder to perform nonlinear transformation encoding on the clinical examination data, constructing a convolutional neural network image feature encoder to perform deep feature extraction encoding on the imaging data, and constructing a long short-term memory network time series feature encoder to perform time series modeling encoding on the laboratory index and lung function detection data, to obtain dimension-aligned clinical hidden representation, image hidden representation, and time series hidden representation, respectively; Adaptively splicing and combining the clinical hidden representation, the image hidden representation, the time series hidden representation, and the time series attention feature, inputting them into a multi-modal variational autoencoder for probability distribution modeling and latent space mapping to obtain a common latent feature with uncertainty quantization; constructing a generator-discriminator adversarial training network, and performing cross-modal distribution alignment processing on the common latent feature through min-max game optimization to obtain a distribution-consistent multi-modal fusion feature.

[0007] On the basis of the above technical scheme, preferably, the multi-modal fusion features are input into a risk classifier for classification calculation to obtain a respiratory system risk level prediction result, generate a risk assessment report, and output respiratory risk early warning information, including: A multi-layer classification network is constructed to perform nonlinear transformation on the multi-modal fusion features, and a probability distribution of three risk levels is output through a Softmax activation function, and a respiratory system risk level prediction result with credibility is obtained by combining confidence evaluation and uncertainty quantification; the prediction results of multiple classifiers are fused through an ensemble learning method; Based on the gradient back propagation technology, the contribution of the input features to the prediction result is calculated, and a multi-dimensional risk factor importance ranking is generated by combining graph attention weights and node importance scores; a medical knowledge graph mapping module is constructed to convert the importance ranking into a clinically understandable risk assessment report; according to the risk level, important risk factors and historical trends of individual patients, personalized risk early warning information including prevention suggestions and monitoring indicators is generated.

[0008] On the basis of the above technical scheme, preferably, the multi-layer classification network adopts a three-layer fully connected structure, residual connection and layer normalization are added between each layer, and an ELU activation function is used to enhance gradient propagation; an uncertainty quantification module is introduced to estimate the uncertainty of the prediction by using the Monte Carlo sampling method, and a confidence interval is provided for each risk level prediction; an integrated classifier is constructed, including a deep neural network classifier, a gradient boosting classifier and a support vector machine classifier, and the prediction results of multiple classifiers are fused through a weighted voting mechanism; a dynamic threshold adjustment mechanism is designed to adaptively adjust the classification decision boundary according to the clinical cost and benefit of different risk levels; A multi-level feature importance calculation module is constructed, including global importance, local importance and time sequence importance; the marginal contribution of each feature to the prediction result is quantified by using the Shapley value calculation method; a medical knowledge graph is constructed, including disease-symptom relationship, examination-index relationship and treatment-effect relationship, and the calculated importance score is mapped to clinical semantics; a personalized early warning rule engine is designed to generate differentiated prevention suggestions according to the patient's age, gender, medical history and current risk state.

[0009] On the basis of the above technical scheme, preferably, the multi-modal medical data of the patient is collected, and a multi-layer heterogeneous graph is constructed based on the multi-modal medical data, including: Clinical examination data, imaging data, laboratory indicators and lung function detection data of the patient are collected, the multi-modal medical data is subjected to data cleaning and standardization processing to obtain a standardized multi-modal medical data set; The data elements in the standardized multi-modal medical data set are respectively mapped into patient nodes, feature nodes, time nodes, device nodes and disease nodes according to node type mapping rules, the connection relationships of patient-feature edges, patient-time edges, feature-device edges, patient-disease edges and time-time edges are defined according to medical semantic relationships, and a multi-layer heterogeneous graph structure is constructed.

[0010] On the basis of the above technical scheme, preferably, the constructing of the weighted adjacency matrix and the node feature vector through the clinical feature similarity, the image feature similarity and the time sequence correlation between the nodes in the multi-layer heterogeneous graph comprises: The clinical feature similarity, the image feature similarity and the time sequence correlation between the nodes in the multi-layer heterogeneous graph are calculated, the similarity is adaptively weighted and distributed through a multi-modal attention mechanism to obtain a fusion similarity matrix, and a weighted adjacency matrix is constructed based on the fusion similarity matrix; A multi-modal canonical correlation analysis model is constructed, the clinical examination data, the imaging data, the laboratory index and the lung function detection data are respectively projected into a common low-dimensional latent space, the semantic consistency of the features of each mode is optimized through a cross-modal alignment loss function, and a cross-modal aligned node feature vector is obtained.

[0011] On the basis of the above technical scheme, preferably, the matrix multiplication operation and the convolution operation are performed based on the weighted adjacency matrix and the node feature vector, and the historical time state information is spliced and combined to obtain a graph state representation at a current time, and the time dimension weighted aggregation is performed on the graph state representation to obtain a time sequence attention feature, which comprises: The weighted adjacency matrix and the node feature vector are subjected to multi-layer graph convolution aggregation operation to obtain multi-scale spatial aggregation features, a multi-resolution time convolution filter bank is constructed, the multi-scale spatial aggregation features are subjected to parallel convolution operation at different time scales based on the multi-resolution time convolution filter bank, and spatio-temporal fusion features are obtained through feature fusion; The spatio-temporal fusion features and the state information of a plurality of historical times are weightedly spliced and combined, three gating processing of selective forgetting, incremental updating and conditional output are performed through an adaptive gating unit network to obtain an enhanced graph state representation at a current time, a multi-level query-key-value mapping mechanism is constructed, the importance weights of different time steps and different levels are calculated through hierarchical scaling dot product attention, and multi-dimensional weighted aggregation is performed on the enhanced graph state representation to obtain hierarchical time sequence attention features.

[0012] In a second aspect, the present application also provides a respiratory system risk prediction system based on a graph neural network, which comprises: The system comprises: a heterogeneous graph construction module, configured to collect multi-modal medical data of a patient, and construct a multi-layer heterogeneous graph based on the multi-modal medical data; a feature weighting mapping module, configured to construct a weighted adjacency matrix and a node feature vector based on clinical feature similarity, image feature similarity and time sequence correlation between nodes in the multi-layer heterogeneous graph; a time sequence feature aggregation module, configured to perform matrix multiplication operation and convolution operation based on the weighted adjacency matrix and the node feature vector, and splice and combine with historical time state information to obtain a graph state representation at a current time, and perform weighted aggregation in a time dimension on the graph state representation to obtain a time sequence attention feature; a data encoding splicing module, configured to perform encoding processing on clinical examination data by a clinical feature encoder to obtain a hidden representation of the clinical examination data, splice and combine with the time sequence attention feature, and perform latent space mapping to obtain a common latent feature, and perform distribution alignment processing on the common latent feature to obtain a multi-modal fusion feature; a respiratory risk prediction module, configured to input the multi-modal fusion feature into a risk classifier to perform classification calculation, obtain a respiratory system risk level prediction result, generate a risk assessment report, and output respiratory risk early warning information.

[0013] In a third aspect, the present application further provides an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; The processor, the memory and the communication interface can communicate with each other through the bus, the memory stores program instructions executable by the processor, and the processor invokes the program instructions to implement the steps of the method.

[0014] In a fourth aspect, the present application further provides a computer readable storage medium, which stores computer instructions, and the computer instructions enable a computer to implement the steps of the method.

[0015] The method and system for predicting respiratory system risk based on graph neural network have the following advantages over the prior art: (1) The multi-modal medical data of clinical examination, imaging, laboratory index and lung function are integrated by constructing a multi-layer heterogeneous graph structure, the cross-modal data alignment is realized by using attention mechanism and canonical correlation analysis, the dynamic time sequence dependency is captured by combining spatio-temporal convolution filter and gated unit network, and deep multi-modal feature fusion is realized by using variational autoencoder and adversarial training network, thereby improving the accuracy and clinical practicability of respiratory system risk prediction. (2) Through the multi-modal attention mechanism, the similarity between different modalities such as clinical, imaging and time sequence is adaptively calculated and fused, the importance of each modality is dynamically adjusted according to the data characteristics, the heterogeneous medical data is projected to a unified latent space through a multi-modal canonical correlation analysis model, and the semantic consistency is optimized by using a cross-modal alignment loss function, thereby improving the fusion quality of multi-modal medical data and the effectiveness of feature representation; (3) By constructing three independent attention modules of clinical, imaging and time sequence, the complementary algorithms of cosine similarity and Euclidean distance, deep feature extraction and structural similarity, dynamic time warping and Pearson correlation coefficient are used respectively, the internal similarity features and correlation patterns of different modal data are captured, and the multi-head attention network is used for intelligent weighted fusion of the three kinds of similarities, the comprehensive evaluation of multi-dimensional similarity is realized, and the comprehensiveness of the relationship modeling between nodes is improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 A flow chart of a respiratory system risk prediction method based on a graph neural network. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0019] Please refer to Figure 1 The present application provides a respiratory system risk prediction method based on a graph neural network, comprising the following steps: Collecting multi-modal medical data of a patient, the multi-modal medical data comprising clinical examination data, imaging data, laboratory indicators and lung function test data, mapping the multi-modal medical data into patient nodes, feature nodes, time nodes, device nodes and disease nodes through node type definition, establishing a multi-type edge connection relationship between nodes, and obtaining a multi-layer heterogeneous graph; The clinical feature similarity, the image feature similarity and the time sequence correlation between nodes in the multi-layer heterogeneous graph are calculated by a multi-modal similarity calculation method, the similarities are combined by weighting based on an attention mechanism, adaptive weights of each edge of the multi-layer heterogeneous graph are obtained, and a weighted adjacency matrix is constructed based on the adaptive weights; the different modal features are mapped and transformed by canonical correlation analysis to obtain cross-modal aligned node feature vectors; The weighted adjacency matrix and the node feature vectors are subjected to matrix multiplication operation to obtain spatial aggregation features; a time convolution filter is constructed to perform convolution operation on the spatial aggregation features in the time dimension to obtain spatio-temporal fusion features; the spatio-temporal fusion features and historical time state information are spliced and combined, and a three-gating processing of forgetting-updating-outputting is performed by a gating unit network to obtain a graph state representation at the current time; a query vector, a key vector and a value vector are constructed, the importance weights of different time steps are calculated by scaled dot-product attention, the graph state representation is weighted aggregated in the time dimension to obtain a time sequence attention feature; The clinical examination data are encoded by a clinical feature encoder, the imaging data are encoded by an image feature encoder, and the laboratory index and lung function detection data are encoded by a time sequence feature encoder to obtain a clinical hidden representation, an image hidden representation and a time sequence hidden representation, respectively; the clinical hidden representation, the image hidden representation, the time sequence hidden representation and the time sequence attention feature are spliced and combined, and input into a variational autoencoder for latent space mapping to obtain a common latent feature; the common latent feature is subjected to distribution alignment processing by an adversarial training network to obtain a multi-modal fusion feature; The multi-modal fusion feature is input into a risk classifier for classification calculation to obtain a respiratory system risk level prediction result, the risk level includes low risk, medium risk and high risk; a risk factor importance ranking is generated by graph attention weight calculation to obtain an interpretable risk assessment report; and personalized risk warning information is output based on the risk level prediction result and the risk assessment report.

[0020] Specifically, the embodiment integrates multi-modal medical data of clinical examination, imaging, laboratory index and lung function by constructing a multi-layer heterogeneous graph structure, realizes cross-modal data alignment by using an attention mechanism and canonical correlation analysis, captures dynamic time sequence dependency by combining a spatio-temporal convolution filter and a gating unit network, realizes deep multi-modal feature fusion by a variational autoencoder and an adversarial training network, and improves the accuracy and clinical practicability of respiratory system risk prediction.

[0021] The multi-modal medical data of the patient is collected, and a multi-layer heterogeneous graph is constructed based on the multi-modal medical data, the multi-modal medical data including clinical examination data, imaging data, laboratory index and lung function detection data, comprising: collecting clinical examination data, imaging data, laboratory indicators and lung function test data of a patient, performing data cleaning and standardization processing on the multi-modal medical data to obtain a standardized multi-modal medical data set; In a specific embodiment, the clinical examination data is subjected to missing value filling and outlier detection, and the Z-score standardization method is used to convert numerical features into a standard normal distribution; the imaging data is subjected to size normalization and pixel value standardization processing, and the image size is uniformly adjusted to a preset resolution; the laboratory indicators are normalized according to the normal reference range, and the indicator values are mapped to the [0, 1] interval; the lung function test data is stratified standardized according to age and gender, and standardized values corrected for age and gender are obtained.

[0022] The data elements in the standardized multi-modal medical data set are respectively mapped into patient nodes, feature nodes, time nodes, device nodes and disease nodes according to node type mapping rules, and the connection relationships of patient-feature edges, patient-time edges, feature-device edges, patient-disease edges and time-time edges are defined according to medical semantic relationships, and a multi-layer heterogeneous graph structure is constructed.

[0023] In a specific embodiment, each patient individual is mapped into a patient node, and the node attributes include patient ID, age, gender and basic disease information; the medical feature indicators are mapped into feature nodes, and the node attributes include feature type, numerical range and clinical significance; the time stamp of data collection is mapped into a time node, and the node attributes include time information and time interval; the medical device information is mapped into a device node, and the node attributes include device type, accuracy level and detection parameters; the respiratory system disease categories are mapped into disease nodes, and the node attributes include disease coding, severity and pathogenesis.

[0024] Specifically, the present embodiment ensures the quality consistency and comparability of data from different sources by performing data cleaning, standardization and missing value processing on multi-modal medical data, effectively converts heterogeneous medical data into graph structure representation based on node type mapping rules, establishes semantic association relationships between patients, features, time, devices and diseases, not only solves the problem that traditional methods cannot effectively process multi-source heterogeneous medical data, but also lays a high-quality data foundation for graph neural network analysis, and improves the data input quality and structured representation ability of the respiratory system risk prediction model.

[0025] The construction of the weighted adjacency matrix and the node feature vector through the clinical feature similarity, the imaging feature similarity and the time sequence correlation between the nodes in the multi-layer heterogeneous graph includes: The similarity of clinical features, image features, and temporal correlation between nodes in the multi-layer heterogeneous graph is calculated. The similarity is adaptively weighted using a multimodal attention mechanism to obtain a fused similarity matrix. A weighted adjacency matrix is ​​then constructed based on the fused similarity matrix.

[0026] In one specific embodiment, the multimodal attention mechanism includes: A clinical feature attention module is constructed to obtain clinical feature similarity by calculating the cosine similarity and Euclidean distance similarity of clinical indicators between nodes; an image feature attention module is constructed to obtain image feature similarity by deep feature extraction and structural similarity measurement; a temporal correlation attention module is constructed to obtain temporal correlation by calculating the dynamic time warping algorithm and Pearson correlation coefficient; the clinical feature similarity, image feature similarity, and temporal correlation are input into a multi-head attention network for weighted fusion to obtain a comprehensive similarity score.

[0027] In one specific embodiment, the formula for calculating the comprehensive similarity score is: ; in, For nodes With nodes The overall similarity score between them For nodes With nodes The similarity of clinical characteristics between them was calculated using cosine similarity and Euclidean distance. For nodes With nodes The image feature similarity between the two images is obtained through deep feature extraction and structural similarity measurement. For nodes With nodes The temporal correlation similarity between them was calculated using the dynamic time warping algorithm and the Pearson correlation coefficient. The adaptive weighting coefficient for clinical feature similarity has a value range of [value range missing]. , The adaptive weighting coefficient for image feature similarity has a value range of [value range missing]. , The adaptive weighting coefficient for temporal relevance similarity has a value range of [value range missing]. ,and .

[0028] In one specific embodiment, the update formula for the adaptive weights is: ; in, For the first Node at the next iteration adaptive weight between nodes , for the first iteration, the weight value between nodes , for the first iteration, the comprehensive similarity score between nodes , for the first iteration, the comprehensive similarity score between nodes , for the first iteration, the comprehensive similarity score between nodes , for the first iteration, the comprehensive similarity score between nodes , for the first iteration, the comprehensive similarity score between nodes ,

[0029]

[0030] In an embodiment, the cross-modal alignment loss function is calculated as follows: ; wherein, is the cross-modal alignment loss function value, used to optimize the semantic consistency of different modalities in the latent space, is the total number of medical data modalities, including clinical, imaging, laboratory indicators and lung function test data, is the number of modalities, used to calculate the alignment loss between different modalities, is the projection matrix of the i-th modality, used to map high-dimensional features to -dimensional latent space, is the original feature matrix of the i-th modality, containing all feature vectors of the modality, is the square of the Frobenius norm, used to measure the difference between the projected features of different modalities, is the sparse regularization parameter, used to control the sparsity of the projection matrix and the complexity of the model, is the L1 norm sparse constraint, used to ensure the sparsity and interpretability of the projection matrix. In an embodiment, the multi-modal canonical correlation analysis model comprises:

[0031] In an embodiment, the multi-modal canonical correlation analysis model comprises: ​The clinical modality projection matrix and the image modality projection matrix are constructed, and the high-dimensional clinical features and image features are respectively mapped to d the same latent space of dimensionality d The time sequence modality projection matrix is constructed, and the time sequence features are mapped to the same latent space of dimensionality

[0032] Specifically, the embodiment constructs K a layer graph convolution network, and multiple aggregation functions (mean aggregation, maximum aggregation and attention aggregation) are used to realize multi-scale spatial feature aggregation, and residual connection and layer normalization are combined to effectively prevent gradient disappearance and overfitting. Different time scales of dynamic patterns are processed in parallel through a multi-resolution time convolution filter set (short-term, medium-term and long-term filters), and a gated fusion mechanism is used to realize adaptive combination. The adaptive gating unit network is used for three gating processing of selective forgetting, incremental updating and conditional output, and the multi-level query-key-value mapping mechanism and hierarchical scaling dot product attention calculation are combined to capture the spatio-temporal dependency and multi-dimensional feature importance of medical data, and improve the time sequence modeling capability and feature representation quality of respiratory system risk prediction.

[0033] In a specific embodiment, the adaptive gating update formula of the adaptive gating unit network is: ; ; ; wherein, is the selective forgetting gate output at time , used to control the retention degree of historical information, and the value range is , is the incremental updating gate output at time , used to control the updating amplitude of current information, and the value range is , is the conditional output gate output at time , used to control the output intensity of graph state representation, and the value range is , is a Sigmoid activation function, used to limit the output to the interval [0, 1], is a learnable weight matrix of the forgetting gate, used for feature transformation, is a learnable weight matrix of the updating gate, used for feature transformation, is a learnable weight matrix of the output gate, used for feature transformation, is the hidden state vector of time step t, representing the historical graph state information, is the input feature vector of time step t, representing the spatio-temporal fusion feature of the current time step, is the input feature vector of time step t, representing the spatio-temporal fusion feature of the current time step, is the concatenation operation for connecting multiple vectors by dimension, is the correlation function for calculating the correlation between the historical state and the current input, is the difference function for calculating the difference between the historical state and the current input, is the confidence function for evaluating the credibility of the current input, , , , are the bias vectors of the forget gate, update gate and output gate, respectively.

[0034] The matrix multiplication operation and convolution operation are performed based on the weighted adjacency matrix and the node feature vector, and the historical time step state information is spliced and combined to obtain the graph state representation of the current time step. The graph state representation is weighted aggregated in the time dimension to obtain the time sequence attention feature, including: The multi-layer graph convolution aggregation operation is performed on the weighted adjacency matrix and the node feature vector to obtain multi-scale spatial aggregation features. A multi-resolution time convolution filter bank is constructed, and the multi-scale spatial aggregation features are subjected to parallel convolution operation in different time scales based on the multi-resolution time convolution filter bank. The spatio-temporal fusion feature is obtained through feature fusion.

[0035] In a specific embodiment, the multi-layer graph convolution aggregation operation includes: A K-layer graph convolution network is constructed, each layer of which aggregates neighborhood node information through different aggregation functions, including mean aggregation, maximum aggregation and attention aggregation. Residual connection and layer normalization are introduced after each convolution to prevent gradient vanishing and overfitting. A multi-resolution time convolution filter is constructed, including short-term filter, medium-term filter and long-term filter, which capture dynamic patterns of different time spans respectively. The time features of different resolutions are adaptively combined through a gating fusion mechanism to obtain comprehensive spatio-temporal fusion features.

[0036] In a specific embodiment, the calculation formula of the multi-layer graph convolution aggregation is: ; wherein, is the node feature matrix of the i-th layer, including the feature representation of all nodes in the layer, is the node feature matrix of the i-th layer, which is the input feature of the current layer, is the node feature matrix of the i-th layer, which is the input feature of the current layer, is the node feature matrix of the i-th layer, which is the input feature of the current layer, The activation function is non-linear; in this embodiment, it is the ReLU activation function. This represents the total number of relation types, corresponding to different edge types in the multi-level heterogeneous graph (such as clinical relations, imaging relations, temporal relations, etc.). For relational type indexes, the value range is 1 to... , For the first The first in the layer Learnable weight matrices for various relationships, used for feature transformation. For the first An adjacency matrix for a given relationship describes the connections between nodes under that relationship. For the first Aggregation functions for various relationships, including mean aggregation, maximum aggregation, and attention aggregation. This is the product of the adjacency matrix and the node feature matrix, used to aggregate neighborhood information. For residual connectivity terms, to prevent gradient vanishing and maintain information flow. This is the index of the current network layer, with a value ranging from 0 to... ( (Total number of floors).

[0037] The spatiotemporal fusion features are weighted and concatenated with state information from multiple historical moments. An adaptive gating unit network is used for selective forgetting, incremental updating, and conditional output—a three-gating process—to obtain the enhanced graph state representation at the current moment. A multi-level query-key-value mapping mechanism is constructed, and importance weights for different time steps and levels are calculated using hierarchical scaling dot product attention. This multi-dimensional weighted aggregation of the enhanced graph state representation yields hierarchical temporal attention features.

[0038] In one specific embodiment, the formula for calculating the importance weights at different time steps and at different levels is as follows: ; in, For the first Layer nodes For nodes Attention weights, representing the node's attention weights. For nodes The degree of importance, It is an exponential function used for softmax normalization calculation. The LeakyReLU activation function allows small gradients for negative values. For the first The attention weight matrix of the layer is used to calculate the attention score. For the first Layer nodes eigenvectors, is the feature vector of the node in the first layer, is the feature vector of the node in the first layer, is the feature vector of the node in the first layer, is the feature vector of the node in the first layer, is the feature vector of the node in the first layer, is the feature vector of the node in the first layer, is the feature vector of the node in the first layer, is the feature vector of the node in the first layer, is the feature vector of the node in the first layer, is the edge feature vector between the node and the node, containing relationship type and weight information, is the edge feature vector between the node and the node, is the edge feature vector between the node and the node, is the edge feature vector between the node and the node, is the edge feature vector between the node and the node, is the edge feature vector between the node and the node, is the edge feature vector between the node and the node, is the edge feature vector between the node and the node, is the edge feature vector between the node and the node, is the edge feature vector between the node and the node, is the edge feature vector between the node and the node,

[0039] In a specific embodiment, the adaptive gating unit network comprises: a historical state importance evaluation module, which determines the weight distribution of historical information by calculating the correlation between current features and historical states; a selective forgetting gate is designed to determine the retention degree of historical information according to the timeliness and relevance of medical features; an incremental update gate is designed to determine the amplitude of state update through the difference analysis of current time information and historical information; a conditional output gate is designed to control the output intensity of graph state representation based on the confidence of risk prediction and feature importance; a multi-level attention mechanism is constructed, including node-level attention, graph-level attention and time-series-level attention, to realize multi-dimensional feature importance modeling.

[0040] Specifically, the embodiment constructs three specialized encoders, namely multi-layer perceptron, convolutional neural network and long short-term memory network, to extract deep features respectively according to the characteristics of clinical, image and time-series data, and combines batch normalization, Dropout regularization and spatial attention mechanism to effectively improve the quality of feature representation; Through the adaptive weight learning module, the fusion weight of multi-modal features is dynamically adjusted, and combined with the probability distribution modeling of multi-modal variational autoencoder and the distribution alignment mechanism of adversarial training network, not only high-quality cross-modal feature fusion and uncertainty quantification are realized, but also the distribution difference between modalities is effectively eliminated, improving the accuracy and reliability of respiratory system risk prediction.

[0041] The clinical examination data is encoded by the clinical feature encoder to obtain a hidden representation of the clinical examination data, and is spliced and combined with the time sequence attention feature and mapped to a latent space to obtain a common latent feature. The common latent feature is processed by distribution alignment to obtain a multi-modal fusion feature, including: The multi-layer perception clinical feature encoder is constructed to perform nonlinear transformation and encoding on the clinical examination data. The convolutional neural network image feature encoder is constructed to perform deep feature extraction and encoding on the imaging data. The long short-term memory network time sequence feature encoder is constructed to perform time sequence modeling and encoding on the laboratory index and lung function detection data, to obtain dimension-aligned clinical hidden representation, image hidden representation, and time sequence hidden representation, respectively.

[0042] In an embodiment, the multi-modal encoding processing includes: The clinical feature encoder adopts a three-layer fully connected network structure, each layer is added with batch normalization and Dropout regularization, and the activation function adopts a Swish function. The image feature encoder adopts a residual convolutional network structure, including a multi-scale feature extraction module and a spatial attention mechanism, and finally obtains a fixed dimension feature vector through global average pooling. The time sequence feature encoder adopts a bidirectional LSTM structure, which models the importance of different time steps in combination with a time attention mechanism, and uses the hidden state of the last time step as the time sequence feature representation. The hidden representations of the three modalities are unified to the same feature space dimension through a feature dimension transformation layer.

[0043] The clinical hidden representation, image hidden representation, and time sequence hidden representation are adaptively weighted and spliced with the time sequence attention feature, input into a multi-modal variational autoencoder for probability distribution modeling and latent space mapping to obtain a common latent feature with uncertainty quantization. A generator-discriminator adversarial training network is constructed to perform cross-modal distribution alignment processing on the common latent feature through a minimax game optimization to obtain a distribution-consistent multi-modal fusion feature.

[0044] In an embodiment, the multi-modal fusion processing includes: An adaptive weight learning module is constructed to dynamically allocate splicing weights by calculating the correlation of each modality feature and the time sequence attention feature. The multi-modal variational autoencoder includes an encoder network, a reparameterization layer, and a decoder network. The encoder maps the multi-modal feature to mean and variance parameters, samples the latent feature through the reparameterization technique, and the decoder reconstructs the feature. The adversarial training network includes a feature generator and a modality discriminator. The generator learns to generate fusion features without modality differences, and the discriminator learns to distinguish different modalities. Through adversarial training, a modality-invariant feature representation is achieved.

[0045] Specifically, the embodiment adaptively calculates and fuses the similarities among different modalities such as clinical, image and time sequence through a multi-modal attention mechanism, dynamically adjusts the importance of each modality according to the data characteristics, projects the heterogeneous medical data into a unified latent space through a multi-modal canonical correlation analysis model, optimizes the semantic consistency by using a cross-modal alignment loss function, and improves the fusion quality of multi-modal medical data and the effectiveness of feature representation.

[0046] The multi-modal fusion features are input into a risk classifier for classification calculation to obtain a respiratory system risk level prediction result, generate a risk assessment report, and output respiratory risk early warning information, including: A multi-layer classification network is constructed to perform nonlinear transformation on the multi-modal fusion features, and a Softmax activation function is used to output the probability distribution of three risk levels, and a respiratory system risk level prediction result with credibility is obtained by combining confidence evaluation and uncertainty quantification; the prediction results of multiple classifiers are fused by an ensemble learning method; The contribution of input features to the prediction result is calculated based on a gradient back propagation technique, a graph attention weight and a node importance score are combined to generate a multi-dimensional risk factor importance ranking; a medical knowledge graph mapping module is constructed to convert the importance ranking into a clinically understandable risk assessment report; individual patient risk levels, important risk factors and historical trends are used to generate personalized risk warning information including prevention suggestions and monitoring indicators.

[0047] The risk classification prediction includes: The multi-layer classification network adopts a three-layer fully connected structure, residual connections and layer normalization are added between each layer, and an ELU activation function is used to enhance gradient propagation; an uncertainty quantification module is introduced to estimate the uncertainty of the prediction by using a Monte Carlo sampling method, and a confidence interval is provided for each risk level prediction; an ensemble classifier is constructed, including a deep neural network classifier, a gradient boosting classifier and a support vector machine classifier, and the prediction results of multiple classifiers are fused by a weighted voting mechanism; a dynamic threshold adjustment mechanism is designed to adaptively adjust the classification decision boundary according to the clinical cost and benefit of different risk levels; The explainability analysis and early warning generation includes: A multi-level feature importance calculation module is constructed, including global importance, local importance and time sequence importance; the marginal contribution of each feature to the prediction result is quantified by using a Shapley value calculation method; a medical knowledge graph is constructed, including disease-symptom relationships, examination-index relationships and treatment-effect relationships, and the calculated importance scores are mapped to clinical semantics; a personalized early warning rule engine is designed to generate differentiated prevention suggestions according to the patient's age, gender, medical history and current risk state.

[0048] Specifically, the embodiment constructs three independent attention modules of clinic, image and time sequence, respectively adopts cosine similarity and Euclidean distance, deep feature extraction and structural similarity, dynamic time warping and Pearson correlation coefficient complementary algorithm, captures the internal similarity features and correlation patterns of different modal data, and utilizes a multi-head attention network to intelligently weight and fuse the three similarities, realizes multi-dimensional similarity comprehensive evaluation, and improves the comprehensiveness of the relationship modeling between nodes.

[0049] The application also provides a respiratory system risk prediction system based on a graph neural network, the system comprising: A heterogeneous graph construction module is configured to collect multi-modal medical data of a patient, wherein the multi-modal medical data comprises clinical examination data, imaging data, laboratory indicators and lung function detection data, the multi-modal medical data is mapped into patient nodes, feature nodes, time nodes, device nodes and disease nodes through node type definition, a multi-type edge connection relationship between the nodes is established, and a multi-layer heterogeneous graph is obtained. A feature weighting mapping module is configured to calculate clinical feature similarity, imaging feature similarity and time sequence correlation between the nodes in the multi-layer heterogeneous graph through a multi-modal similarity calculation method, combine the similarities based on an attention mechanism, obtain adaptive weights of each edge of the multi-layer heterogeneous graph, construct a weighted adjacency matrix based on the adaptive weights, and map and transform different modal features through canonical correlation analysis to obtain cross-modal aligned node feature vectors. A spatio-temporal feature fusion module is configured to perform matrix multiplication operation on the weighted adjacency matrix and the node feature vectors to obtain spatial aggregation features, perform convolution operation on the spatial aggregation features in the time dimension through a time convolution filter to obtain spatio-temporal fusion features, splice and combine the spatio-temporal fusion features and historical time state information, perform three gating processing of forgetting, updating and outputting through a gating unit network to obtain a graph state representation at a current time, construct a query vector, a key vector and a value vector, calculate the importance weights of different time steps through scaled dot-product attention, and perform weighted aggregation in the time dimension on the graph state representation to obtain a time sequence attention feature. A data encoding and splicing module is configured to perform encoding processing on the clinical examination data through a clinical feature encoder, perform encoding processing on the imaging data through an imaging feature encoder, and perform encoding processing on the laboratory indicators and lung function detection data through a time sequence feature encoder to obtain clinical hidden representation, imaging hidden representation and time sequence hidden representation, respectively; the clinical hidden representation, the imaging hidden representation, the time sequence hidden representation and the time sequence attention feature are spliced and combined, input into a variational autoencoder for latent space mapping to obtain common latent features; the common latent features are subjected to distribution alignment processing through an adversarial training network to obtain multi-modal fusion features. The respiratory risk prediction module is used for inputting the multi-modal fusion features into a risk classifier for classification calculation to obtain a respiratory system risk level prediction result, the risk level including low risk, medium risk and high risk; a risk factor importance ranking is generated through graph attention weight calculation to obtain an interpretable risk assessment report; and personalized risk warning information is output based on the risk level prediction result and the risk assessment report.

[0050] The application further discloses an electronic device, comprising at least one processor, at least one memory, a communication interface and a bus; wherein the processor, the memory and the communication interface complete communication with each other through the bus; the memory stores program instructions executable by the processor, and the processor invokes the program instructions to realize the respiratory system risk prediction method based on a graph neural network.

[0051] The application further discloses a computer readable storage medium, which stores computer instructions, and the computer instructions make the computer realize all or part of steps of the respiratory system risk prediction method based on a graph neural network.

[0052] The above merely describes preferred embodiments of the application and is not intended to limit the application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A respiratory system risk prediction method based on a graph neural network, characterized in that, The method comprises the following steps: Collecting multi-modal medical data of a patient, constructing a multi-layer heterogeneous graph based on the multi-modal medical data, wherein the multi-modal medical data comprises clinical examination data, imaging data, laboratory indicators and lung function test data; Constructing a weighted adjacency matrix and a node feature vector through clinical feature similarity, imaging feature similarity and time correlation between nodes in the multi-layer heterogeneous graph; Performing matrix multiplication and convolution operations based on the weighted adjacency matrix and the node feature vector, and combining with historical time state information to obtain a graph state representation at the current time, and performing weighted aggregation in the time dimension to obtain a time attention feature; Encoding the clinical examination data through a clinical feature encoder to obtain a hidden representation of the clinical examination data, combining the hidden representation with the time attention feature, and performing latent space mapping to obtain a common latent feature, and performing distribution alignment processing on the common latent feature to obtain a multi-modal fusion feature; Inputting the multi-modal fusion feature into a risk classifier for classification calculation to obtain a respiratory system risk level prediction result, generating a risk assessment report, and outputting respiratory risk early warning information.

2. The respiratory system risk prediction method based on a graph neural network according to claim 1, wherein, The encoding processing of the clinical examination data through the clinical feature encoder to obtain the hidden representation of the clinical examination data, the combination of the hidden representation with the time attention feature, and the latent space mapping to obtain the common latent feature, and the distribution alignment processing of the common latent feature to obtain the multi-modal fusion feature, comprise: Constructing a multi-layer perception clinical feature encoder to perform nonlinear transformation coding on the clinical examination data, constructing a convolutional neural network image feature encoder to perform deep feature extraction coding on the imaging data, and constructing a long short-term memory network time sequence feature encoder to perform time sequence modeling coding on the laboratory indicators and lung function test data, to obtain dimension-aligned clinical hidden representation, image hidden representation and time sequence hidden representation, respectively; Adaptive weight splicing combination of the clinical hidden representation, the image hidden representation, the time sequence hidden representation and the time attention feature, inputting into a multi-modal variational autoencoder for probability distribution modeling and latent space mapping to obtain a common latent feature with uncertainty quantization, constructing a generator-discriminator adversarial training network, and performing cross-modal distribution alignment processing on the common latent feature through min-max game optimization to obtain a distribution-consistent multi-modal fusion feature.

3. The respiratory system risk prediction method based on graph neural network according to claim 1, wherein, The inputting of the multi-modal fusion feature into the risk classifier for classification calculation to obtain the respiratory system risk level prediction result, the generation of the risk assessment report, and the outputting of the respiratory risk early warning information, comprise: Constructing a multi-layer classification network to perform nonlinear transformation on the multi-modal fusion feature, outputting a probability distribution of three risk levels through a Softmax activation function, combining confidence evaluation and uncertainty quantization to obtain a respiratory system risk level prediction result with credibility, and fusing prediction results of multiple classifiers through an ensemble learning method. The contribution of input features to the prediction result is calculated based on the gradient back propagation method, and a multi-dimensional risk factor importance ranking is generated by combining the graph attention weight and the node importance score. A medical knowledge graph mapping module is constructed to convert the importance ranking into a clinically understandable risk assessment report. According to the individual patient's risk level, important risk factors and historical trends, personalized risk warning information including prevention suggestions and monitoring indicators is generated.

4. The respiratory system risk prediction method based on graph neural network according to claim 3, wherein, The multi-layer classification network adopts a three-layer fully connected structure, with residual connections and layer normalization added between each layer, and an ELU activation function used to enhance gradient propagation. An uncertainty quantification module is introduced to estimate the uncertainty of the prediction using the Monte Carlo sampling method, providing a confidence interval for each risk level prediction. An integrated classifier is constructed, including a deep neural network classifier, a gradient boosting classifier and a support vector machine classifier, which fuse the prediction results of multiple classifiers through a weighted voting mechanism. A dynamic threshold adjustment mechanism is designed to adaptively adjust the classification decision boundary based on the clinical cost and benefit of different risk levels. A multi-level feature importance calculation module is constructed, including global importance, local importance and time sequence importance. The marginal contribution of each feature to the prediction result is quantified by the Shapley value calculation method. A medical knowledge graph is constructed, including disease-symptom relationships, examination-index relationships and treatment-effect relationships, and the calculated importance scores are mapped to clinical semantics. A personalized early warning rule engine is designed to generate differentiated prevention suggestions based on the patient's age, gender, medical history and current risk state.

5. The respiratory system risk prediction method based on graph neural network of claim 1, wherein, The multi-modal medical data of the patient is collected, and a multi-layer heterogeneous graph is constructed based on the multi-modal medical data, including: Clinical examination data, imaging data, laboratory indicators and lung function detection data of the patient are collected, and the multi-modal medical data is subjected to data cleaning and standardization processing to obtain a standardized multi-modal medical data set; The data elements in the standardized multi-modal medical data set are respectively mapped to patient nodes, feature nodes, time nodes, device nodes and disease nodes according to node type mapping rules, and the connection relationships of patient-feature edges, patient-time edges, feature-device edges, patient-disease edges and time-time edges are defined according to medical semantic relationships, to construct a multi-layer heterogeneous graph structure.

6. The respiratory system risk prediction method based on graph neural network of claim 1, wherein, The clinical feature similarity, imaging feature similarity and time sequence correlation between nodes in the multi-layer heterogeneous graph are calculated, and the similarity is adaptively weighted by a multi-modal attention mechanism to obtain a fusion similarity matrix, and a weighted adjacency matrix is constructed based on the fusion similarity matrix. A multi-modal canonical correlation analysis model is constructed, and the clinical examination data, imaging data, laboratory indicators and lung function detection data are projected into a common low-dimensional latent space, and the semantic consistency of the features of each modality is optimized by a cross-modal alignment loss function to obtain cross-modal aligned node feature vectors. ​ 7. The respiratory system risk prediction method based on graph neural network of claim 1, wherein, The matrix multiplication operation and convolution operation are performed on the weighted adjacency matrix and the node feature vector, and the historical time state information is spliced and combined to obtain a graph state representation at the current time. The graph state representation is weighted aggregated in the time dimension to obtain a time sequence attention feature, including: The multi-layer graph convolution aggregation operation is performed on the weighted adjacency matrix and the node feature vector to obtain a multi-scale spatial aggregation feature. A multi-resolution time convolution filter bank is constructed, and the multi-scale spatial aggregation feature is subjected to parallel convolution operation in different time scales based on the multi-resolution time convolution filter bank. The spatio-temporal fusion feature is obtained through feature fusion; The spatio-temporal fusion feature and the state information of multiple historical times are weighted spliced and combined, and the three gating processing of selective forgetting, incremental updating and conditional output is performed through an adaptive gating unit network to obtain an enhanced graph state representation at the current time. A multi-level query-key-value mapping mechanism is constructed, and the importance weight of different time steps and different levels is calculated through hierarchical scaling dot product attention. The enhanced graph state representation is weighted aggregated in multiple dimensions to obtain a hierarchical time sequence attention feature.

8. A graph neural network-based respiratory system risk prediction system configured to perform a graph neural network-based respiratory system risk prediction method according to any one of claims 1-7. The system comprises: A heterogeneous graph construction module configured to collect multi-modal medical data of a patient, and construct a multi-layer heterogeneous graph based on the multi-modal medical data; A feature weighting mapping module configured to construct a weighted adjacency matrix and a node feature vector based on clinical feature similarity, image feature similarity and time sequence correlation between nodes in the multi-layer heterogeneous graph; A time sequence feature aggregation module configured to perform matrix multiplication operation and convolution operation on the weighted adjacency matrix and the node feature vector, and splice and combine the historical time state information to obtain a graph state representation at the current time. The graph state representation is weighted aggregated in the time dimension to obtain a time sequence attention feature; A data encoding splicing module configured to perform encoding processing on clinical examination data through a clinical feature encoder to obtain a hidden representation of the clinical examination data, splice and combine the hidden representation with the time sequence attention feature, and map the hidden representation to a latent space to obtain a common latent feature. The common latent feature is subjected to distribution alignment processing to obtain a multi-modal fusion feature; A respiratory risk prediction module configured to input the multi-modal fusion feature into a risk classifier for classification calculation to obtain a respiratory system risk level prediction result, generate a risk assessment report, and output respiratory risk warning information.

9. An electronic device, comprising: Comprise: At least one processor, at least one memory, a communication interface and a bus; Wherein, the processor, memory, communication interface complete mutual communication through the bus, the memory stores program instructions executable by the processor, the processor calls the program instructions to realize the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, and the computer instructions enable the computer to implement the method of any one of claims 1-7.

Images