A medical data analysis method and system

Abstract

The application discloses a medical data analysis method and system, and relates to the technical field of data analysis, which comprises the following steps: acquiring multi-modal medical data of a patient, recording the existing state and missing reason of each mode, and forming a data set with a missing reason label; through a missing mechanism perception coding network, extracting a content representation vector for the mode in the existing state, and generating a missing embedding vector rich in missing semantics for the mode in the missing state; calculating the availability coefficient of each mode through a missing gate fusion network, and generating a pure multi-modal fusion representation; taking the multi-modal fusion representation and the availability coefficient as conditions, outputting a prediction value and a confidence interval determined by the missing degree through a missing conditional uncertainty estimation network; and generating a prediction evaluation report with missing influence explanation information based on the prediction value and the confidence interval. The application aims to solve the problem of missing reason analysis and signalization utilization of non-random missing modes.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, and in particular to a medical data analysis method and system. Background Technology

[0002] In multimodal medical data analysis, modalities such as imaging, laboratory tests, and medical records are often missing due to clinical decisions rather than random occurrences: critically ill patients cannot undergo CT scans due to emergency intubation, and patients with mild symptoms do not undergo gene sequencing due to low-risk assessments. These "non-random missing" data themselves carry strong prognostic information—missing CT scans often indicate a rapid deterioration of the condition, and missing gene data often suggests a benign clinical diagnosis.

[0003] Current technologies generally treat modal missingness as "data defects" that need to be eliminated, and employ three types of methods to handle it: first, zero-value or mean imputation; second, generative models (such as GANs and diffusion models) to complete pseudo-modalities; and third, using existing modalities to predict the embedding representation of missing modalities. These methods are all based on the assumption that "missing data is noise and should be restored," ignoring the diagnostic value inherent in the missing data mechanism itself, and even causing fatal biases: misjudging high-risk patients who have not had CT scans as having normal images, leading to a systematic underestimation of prognostic risks.

[0004] Furthermore, the completion operation introduces hallucinatory features into the input layer, polluting subsequent fusion modules and causing the model to fail entirely when key modalities are missing. While existing gating mechanisms can mask missing modalities, their gating relies solely on a Boolean flag indicating whether a modality exists, failing to perceive the underlying clinical reasons for the absence. Therefore, they cannot distinguish between the vastly different clinical scenarios of "inability to perform examination due to critical illness" and "need no examination due to low risk." Thus, there is an urgent need for an analytical method capable of analyzing the causes of missing modalities and directly encoding the missing behavior into decision signals. Summary of the Invention

[0005] This invention provides a medical data analysis method, comprising: Acquire patients' multimodal medical data and record the existence status and reasons for missing data for each modality to form a dataset with missing data reason labels; By using a missing mechanism-aware encoding network, content representation vectors are extracted for modalities in the present state, and missing embedding vectors rich in missing semantics are generated for modalities in the missing state. Based on the content representation vector and missing embedding vector of each modality, the availability coefficient of each modality is calculated through a missing gating fusion network, and a clean multimodal fusion representation is generated. Using multimodal fusion representation and availability coefficient as conditions, a missing condition uncertainty estimation network is used to output the predicted value and its confidence interval determined by the degree of missingness. Based on the predicted values ​​and confidence intervals, a prediction and evaluation report with explanations of the impact of missing values ​​is generated.

[0006] The aforementioned medical data analysis method, through a missing mechanism-aware encoding network, extracts content representation vectors for modalities in an existent state and generates missing embedding vectors rich in missing semantics for modalities in a missing state, including: For modalities that are in existence, their raw data is input into the content encoding branch, and the content representation vector is extracted by the dedicated encoder of each modality. For modalities in a missing state, a missing embedding generation function is constructed, which generates missing embedding vectors rich in missing semantics through learnable nonlinear transformations.

[0007] The aforementioned medical data analysis method calculates the availability coefficient of each modality using a missing embedding vector and a missing-gated fusion network, based on the content representation vector and missing embedding vector of each modality, and generates a clean multimodal fusion representation, including: A first-level gating network is constructed based on the content representation vector and missing embedding vector of each modality, and the availability coefficient of each modality is calculated. A second-level gating network is constructed, and the availability coefficients of the first-level gating output are corrected by feedback through the inter-modal compensation matrix to obtain the final effective weights of each mode; The multimodal attention interactions are modulated using the final effective weights to generate a pure multimodal fusion representation.

[0008] The aforementioned medical data analysis method, using multimodal fusion representation and availability coefficients as conditions, outputs predicted values ​​and their confidence intervals determined by the degree of missing data through a missing condition uncertainty estimation network, including: Construct a subnetwork for estimating the predicted mean, taking multimodal fusion representation as input and outputting the predicted value; A missing conditional variance estimation subnetwork is constructed, with multimodal fusion representation and the final effective weights of each modality as joint inputs, and the output is the uncertainty variance determined by the degree of missingness; Based on the combination of the predicted values ​​and the total predicted variance, a confidence interval containing information on missing attribution uncertainty is generated. The predictive mean estimation subnetwork, the missing conditional variance estimation subnetwork, the missing mechanism perception coding network, and the missing gating fusion module are connected in series to form a complete end-to-end trainable system.

[0009] The aforementioned medical data analysis method, based on the predicted value and confidence interval, generates a predictive assessment report with accompanying explanations of missing information, including: A multi-task joint loss function is constructed, which combines the supervision loss of the prediction task with the semantic regularization loss of missing embeddings to drive the collaborative optimization of the parameters of each module in the whole system. All modules are trained jointly in an end-to-end manner, and all learnable parameters are updated synchronously through unified gradient backpropagation. The trained system is deployed in a medical data analysis environment to perform end-to-end inference on non-random missing multimodal data of newly added patients.

[0010] The aforementioned medical data analysis method constructs a multi-task joint loss function, combining the supervision loss of the prediction task with the semantic regularization loss of missing embeddings to drive the collaborative optimization of parameters of each module in the entire system, including: Construct a task loss function to measure the difference between the model's predictions and the true labels; We construct a semantic regularization loss function for missing embeddings to enhance the semantic structure of missing embedding vectors.

[0011] The aforementioned medical data analysis method employs an end-to-end approach to jointly train all modules, synchronously updating all learnable parameters through unified gradient backpropagation, including: Perform batch splitting on datasets with missing reason labels; In each training batch, the complete forward propagation path is executed; Calculate the total loss value based on the predicted value and the confidence interval; The entire training dataset is iterated through multiple rounds until the total loss converges to a preset threshold or reaches the upper limit of the specified training rounds, thus obtaining a complete prediction system after training.

[0012] A medical data analysis system, comprising: The missing cause labeling module is used to acquire patients' multimodal medical data and record the existence status and missing causes of each modality, forming a dataset with missing cause labels; The missing semantic encoding module is used to extract content representation vectors for modalities in the presence state and generate missing embedding vectors rich in missing semantics for modalities in the missing state through the missing mechanism-aware encoding network. The missing-gated fusion module is used to calculate the availability coefficient of each modality through the missing-gated fusion network based on the content representation vector and missing embedding vector of each modality, and generate a clean multimodal fusion representation. The missing uncertainty estimation module is used to output predicted values ​​and their confidence intervals determined by the degree of missingness through a missing conditional uncertainty estimation network, based on multimodal fusion representation and availability coefficient. The prediction and evaluation module is used to generate a prediction and evaluation report with explanations of missing impacts based on the predicted values ​​and confidence intervals.

[0013] The beneficial effects achieved by this invention are as follows: Modal missing data is transformed from defects requiring elimination into key diagnostic clues. Through missing context annotation and missing mechanism-aware encoding, the "cause of missing data" (e.g., emergency intubation, low-risk undetected) is explicitly encoded into semantically rich missing data embedding vectors, enabling the model to distinguish between entirely different clinical situations. Simultaneously, a missing data gating mechanism dynamically blocks noise channels based on missing semantics rather than Boolean labels, avoiding predictive contamination. The final output includes the confidence changes caused by missing data, providing safer and more transparent support for clinical decision-making. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0015] Figure 1 This is a flowchart of a medical data analysis method provided in Embodiment 1 of this application.

[0016] Figure 2 This is a schematic diagram of a medical data analysis system provided in Embodiment 2 of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1 like Figure 1 As shown, Embodiment 1 of this application provides a medical data analysis method, including: S1: Acquire patients' multimodal medical data and record the existence status and reasons for missing data for each modality to form a dataset with missing data label; Multimodal data is collected from medical information systems, and when a modality is missing, the cause of the missing data is extracted from clinical records and labeled to construct a missing cause label dataset.

[0019] Specifically, patient image data, structured test data, text medical record data, and vital sign time series data are acquired in batches from electronic medical record systems, image archiving and communication systems, and laboratory information systems through standardized interfaces. The above data are divided into different modalities, such as image modality, structured test modality, text medical record modality, and vital sign time series modality.

[0020] For each patient's medical visit, the existence status of each of the above modalities is checked one by one; for modalities that exist, their data and existence timestamps are recorded; for modalities that are missing, the medical records and medical orders of that visit are analyzed to extract the clinical reason description that caused the missing modality, and the timestamp when the missing was confirmed is recorded.

[0021] Specifically, a text classification method based on a pre-trained language model is used to parse the textual medical record data of the corresponding patients, especially medical orders and progress notes, to identify descriptive statements related to the failure to perform examinations. For example, if the progress note contains the text "The patient suddenly experienced respiratory failure, requiring emergency endotracheal intubation, and a CT scan was not performed in time," the reason for the missing imaging modality is marked as "emergency resuscitation without examination." If the medical order system shows that the doctor has issued a CT scan request, but the progress note states that the patient signed a refusal of examination consent form, the reason for the missing modality is marked as "patient refused examination." If, throughout the entire treatment process, the doctor did not issue any imaging examination requests, and the discharge summary clearly states "based on clinical manifestations and laboratory test results, organic lesions are ruled out, and imaging examinations are not required," the reason for the missing modality is marked as "low-risk clinical condition without examination." For the missing modalities, the same method is used to mark their missing reasons, such as "unable to draw blood during resuscitation," "extremely short hospital stay with no progress notes," and "non-critical patients not continuously monitored."

[0022] The extracted missing cause descriptions are then mapped to a pre-constructed missing cause coding system to obtain standardized missing cause coding labels. The missing cause coding system includes three levels of coding, where the first level of coding identifies the major category of missing cause, specifically including: C01 for medical priority reasons, C02 for clinical decision reasons, C03 for patient-related reasons, and C04 for systemic reasons.

[0023] Under the primary coding level, specific reasons for missing information are further subdivided through secondary and tertiary coding levels. For example, the missing reason "emergency resuscitation not examined" is mapped and stored as the code "C01.1.01". This coding sequence corresponds to the primary category "medical priority", the secondary subcategory "emergency resuscitation", and the tertiary specific reason "emergency intubation not examined". The missing reason for "clinically low-risk undetected" is mapped and stored as the code "C02.1.01", corresponding to the first-level category "clinical decision", the second-level subcategory "low-risk judgment", and the third-level specific reason "outpatient mild case undetected". By standardizing the coding of all missing reasons, the same clinical missing situation has a unified machine representation in the dataset.

[0024] The existence status of each modality, the missing cause coding label, the missing timestamp, and the clinical context snapshot containing the admission route, admission diagnosis, and treatment outcome are combined into a structured record. All records of all patients are summarized to form a dataset with missing cause labels.

[0025] Each data record includes at least the following: a "Patient ID" to identify the patient, a "Visit Event ID" to identify the specific medical event, a "Modal Name" to identify the modality to which the data belongs, a "Presence Status" to record the collection status, a "Missing Reason Label" to record the specific reason when the presence status is missing, a "Missing Timestamp" to record the time the missing information was confirmed, and a "Context Snapshot" extracted from the core medical record information of the event. The "Context Snapshot" field further includes key information such as the admission route, departmental diagnosis, and treatment outcome of this visit, providing necessary background support for semantic encoding of missing information in subsequent steps.

[0026] S2: Through the missing mechanism-aware encoding network, content representation vectors are extracted for modalities in the present state, and missing embedding vectors rich in missing semantics are generated for modalities in the missing state. The missing mechanism-aware coding network consists of a content coding branch and a missing coding branch, and the outputs of both branches are vectors of the same dimension. The content encoding branch handles existing modalities, compressing the raw data into fixed-dimensional semantic vectors. Different dedicated encoders are deployed internally depending on the modality type. The missing data encoding branch handles missing data modalities, taking as input the missing cause code, the missing timestamp, and the patient's global state vector.

[0027] The process involves using a missing mechanism-aware encoding network to extract content representation vectors for modalities in the present state and to generate missing embedding vectors rich in missing semantics for modalities in the missing state. This includes the following sub-steps: S21: For a modality that is in an existing state, its raw data is input into the content encoding branch, and the content representation vector is extracted by the dedicated encoder of each modality. Specifically, select the corresponding encoder based on the modality type.

[0028] For image modalities, a pre-trained visual Transformer network is used to divide the image into fixed-size image blocks. After linear projection and position encoding, the blocks are input into a multi-layer self-attention network, and the global average pooling result is used as the content representation vector.

[0029] For the structured testing modality, the values ​​of each testing item are standardized and then mapped to a fixed-dimensional vector through a multilayer perceptron consisting of fully connected layers.

[0030] For the text medical record modality, a pre-trained language model is used to convert the text sequence into an embedding representation, and the pooled representation of the sequence is taken as the content representation vector.

[0031] For the time-series modality of vital signs, the time-series data is input into a time-series encoder composed of a long short-term memory network, and the hidden state of the last time step is taken as the content representation vector.

[0032] All of the above representation vectors are projected onto dimension d to ensure alignment with the missing embedding vector space.

[0033] S22: For a mode in a missing state, construct a missing embedding generation function and generate a missing embedding vector rich in missing semantics through a learnable nonlinear transformation.

[0034] Specifically, the missing behavior itself is encoded into a semantic vector that can participate in subsequent calculations through a missing embedding generation function. The specific calculation formula is as follows: ,in, This represents the generated missing embedding vector, with dimension . Consistent with the content representation vector; Encode the missing reason label; For learnable embedding lookup operations, discrete encoded labels are mapped to dense vectors, so that semantically similar missing causes are close to each other in the vector space. Missing timestamp; The location encoding function converts the absolute time of the missing event or the time difference relative to the admission time into a periodic sine-cosine encoding vector, enabling the model to perceive the location of the missing event on the patient's disease timeline. The patient's global state vector is obtained by attention pooling aggregation of the content representation vectors of all existing modalities of the patient before the current missing event occurs, and is used to characterize the patient's condition background when the missing event occurs. and Here are the weight matrix and bias vector of the first fully connected layer; It is a non-linear activation function; and The weight matrix and bias vector of the second fully connected layer are used to project the hidden layer representation onto the dimension. ; This represents a vector concatenation operation, which joins multiple vectors end-to-end along the feature dimension.

[0035] The patient's global state vector is obtained as follows: Content representation vectors of all modalities present in the patient before the current missing event occur are collected, and these vectors are fed into an attention pooling layer. The attention pooling layer uses the patient's demographic feature embedding vector as the query vector, and the content representation vectors of each present modality as the key and value vectors. It calculates the aggregation weights of each present modality using a scaled dot product attention mechanism, and then sums the value vectors using weighted summation to obtain the patient's global state vector s.

[0036] The missing embedding generation function is used to concatenate the vectors of the missing cause, the missing timing, and the patient's state, and then perform nonlinear fusion through a two-layer fully connected network, so that the missing embedding vector can express the interaction effect between the three types of information.

[0037] For example, when the cause of the missing data is failure to detect emergency intubation, the missing data occurs in the very early stage of admission, and the patient's overall condition shows severe hypoxemia, the missing data embedding vector generated by the interaction of these three factors will carry a strong critical warning signal. When the missing reason is that a minor outpatient condition was not detected, the missing condition occurs at the scheduled appointment time, and the patient's overall status shows that all indicators are normal, the missing embedding vector generated by the interaction of these three factors carries a low-risk signal.

[0038] S3: Based on the content representation vector and missing embedding vector of each modality, calculate the availability coefficient of each modality through the missing gating fusion network, and generate a pure multimodal fusion representation; The missing-gated fusion network includes availability gating and modality compensation gating. It takes the content representation vector and missing embedding vector of each modality as input, and generates a pure multimodal fusion representation through availability judgment driven by missing semantics and intermodal functional compensation modeling.

[0039] The process involves calculating the availability coefficient of each modality using a missing embedding vector and a missing-gated fusion network, and generating a clean multimodal fusion representation. This includes the following sub-steps: S31: Construct a first-level gating network based on the content representation vector and missing embedding vector of each modality, and calculate the availability coefficient of each modality; Specifically, the vectors corresponding to all modalities are collected into a sequence X. For modalities in an existent state, their vectors are content representation vectors; for modalities in a missing state, their vectors are missing embedding vectors. Each vector in the input sequence is fed into a first-level gating network, and its availability coefficient is calculated. This first-level gating network consists of a fully connected layer and... The activation function is constructed, and its core formula is as follows: ,in, For the first A vector for each modality can be either a content representation vector or a missing embedding vector; and Here are the weight matrix and bias vector of the gated hidden layer; It is a non-linear activation function; The gated projection vector; For scalar bias; The function compresses the output to the (0, 1) interval to obtain the availability coefficient. .

[0040] This gating network takes the modality vector itself as input. When the input is a content representation vector, because it carries real-world modality information, the gating network outputs higher values ​​after training. When the input is a missing embedding vector, the gating network outputs differentiated values ​​based on the three semantic elements encoded in the vector: the cause of the missing data, the timing of the missing data, and the patient's state. The value is to block the coefficient that is close to zero due to forced loss of output caused by emergency rescue, and to output a lower coefficient due to active abandonment caused by low risk judgment. However, the prognostic signal carried by both in the missing embedding vector has been explicitly encoded and retained in step S2.

[0041] S32: Construct a second-level gating network, and use the inter-modal compensation matrix to correct the availability coefficients of the first-level gating output to obtain the final effective weights of each mode; Existing gating methods output a one-dimensional weight vector, which only independently judges the participation level of each mode, without modeling the functional substitution and compensation relationship between modes.

[0042] Therefore, this invention proposes an intermodal compensation gating mechanism, the core formula of which is: ,in, It is modal The final effective weight after two levels of gating correction; and Modal and modality The first level of availability coefficient; and This is the corresponding mode vector; This represents the concatenation operation of two vectors; and The learnable compensation weight matrix and bias; The activation function maps the compensation intensity to the (-1, 1) interval, allowing for both positive compensation and negative inhibition; Representing modes For modes The compensation strength, which is obtained by concatenating two modal vectors and performing a nonlinear transformation, is capable of learning the functional substitution relationship between modes.

[0043] The above-mentioned compensation gating mechanism enables semantic-driven active compensation. For example, when an image modality is missing and the cause of the missing modality is emergency rescue, if the concatenation of the vector of the test modality (the structured test modality in existence) and the image missing embedding vector is processed by the inter-modal compensation gating mechanism, the final weight of the test modality will be significantly improved. This improvement is driven by the semantic computation of inter-modal functional substitution.

[0044] It also preserves the signal path of the missing mode and the initial coefficients of the missing mode. Although it approaches zero, the formula... This allows the compensation behaviors of other modalities to still be recorded and flow into the fusion process, thereby preserving the semantic signals of the missing event itself.

[0045] Furthermore, when there is no functional substitution relationship between the two modes, the compensation network outputs a compensation strength close to zero after training, and the formula automatically degenerates into... This avoids introducing redundant calculations.

[0046] The functional substitution relationship indicates the extent to which information from one modality can compensate for the information loss caused by the absence of another modality; S33: Modulate the multimodal attention interaction with the final effective weights to generate a pure multimodal fusion representation.

[0047] Specifically, the input sequence X is linearly projected to generate a query matrix, a key matrix, and a value matrix. The original attention weights are calculated using a standard scaled dot product attention mechanism. The final effective weights of each modality are then applied to these original attention weights, modulating them so that modalities with lower final effective weights are suppressed during attention interactions, while the attention weights of modalities strengthened through inter-modal compensation are amplified accordingly, resulting in more thorough information interaction among the remaining modalities. Finally, the modulated attention weights are weighted and summed with the value matrix, then passed through a feedforward network and residual connections to output a clean multimodal fusion representation.

[0048] S4: Using multimodal fusion representation and availability coefficient as conditions, the predicted value and its confidence interval determined by the degree of missing information are output through the missing condition uncertainty estimation network; The process, which uses multimodal fusion representation and availability coefficients as conditions, employs a missing condition uncertainty estimation network to output predicted values ​​and their confidence intervals determined by the degree of missing information. This includes the following sub-steps: S41: Construct a subnetwork for estimating the predicted mean, taking the multimodal fusion representation as input and outputting the predicted value; A pure multimodal fusion representation is fed into a prediction mean estimation subnetwork. This subnetwork consists of multiple fully connected layers stacked together, with nonlinear activation functions used to transform between layers. The last layer outputs a predicted value that matches the prediction task.

[0049] The input dimension of the sub-network is consistent with the dimension d of the multimodal fusion representation; the hidden layer can be set to 2 to 4 layers, and the number of neurons in each layer is arranged in descending order; the output layer dimension is determined according to the specific prediction task. If it is a binary disease risk prediction, it outputs a 1-dimensional probability value; if it is a multi-class prediction, it outputs a multi-dimensional probability distribution corresponding to the number of categories; if it is a survival time regression prediction, it outputs a 1-dimensional continuous value.

[0050] The weight matrix and bias vector of each fully connected layer are automatically learned and determined by backpropagation of the task loss during training. Depending on the prediction task, the predicted value can be the disease risk probability, expected survival time, or treatment response score. The prediction mean estimation subnetwork relies only on the joint representation of the currently available modalities after gated fusion, and its output reflects the best point estimate based on the available information.

[0051] S42: Construct a missing conditional variance estimation subnetwork, with multimodal fusion representation and the final effective weights of each modality as joint input, and output the uncertainty variance determined by the degree of missing data; Specifically, the core of this step is to propose a missing conditional variance estimation subnetwork, which enables the final output confidence interval to explicitly separate the uncertainty components caused by the missing modes, rather than mixing various uncertainties into an untraceable overall confidence value.

[0052] This subnetwork consists of two parallel variance output branches, which are configured differently in terms of their respective input characteristics.

[0053] The first branch is for estimating the inherent uncertainty of the data. This branch takes the multimodal fusion representation as input, and the dimension of the multimodal fusion representation is d. The branch contains 2 to 3 fully connected layers, with the number of neurons in each layer decreasing in order. The output of the last layer is processed by an absolute value function or a square function to ensure non-negativity, resulting in the first variance component.

[0054] The first variance component is used to capture noise in the data itself and uncertainty in model cognition. It is independent of whether a mode is missing or not, and it still exists even when all modes are complete, reflecting the inherent fuzziness of the medical prediction task itself.

[0055] The second branch is for estimating the uncertainty caused by missing values. This branch takes an N-dimensional vector composed of the final effective weights of each modality as input, where N is the total number of modalities. Since the dimension N of this input vector is much smaller than the dimension d of the fused representation, the network size of the second branch is configured with 1 to 2 fully connected layers, and the hidden layer width is narrower than that of the first branch to adapt to its low-dimensional input characteristics. The output of the last layer is processed by an absolute value function or a square function to ensure non-negativity, resulting in the second variance component.

[0056] Each element in the N-dimensional vector composed of the final effective weights of each modality directly reflects the actual participation degree of the corresponding modality in the fusion process. When a modality is missing, its final effective weight approaches zero. This information is captured by the uncertainty estimation branch caused by the missing modality and mapped to a correspondingly increased second variance component. When multiple key modalities are missing simultaneously, the second variance component will be significantly amplified, widening the total confidence interval and explicitly indicating that the current prediction is highly uncertain due to the lack of key checks. When all modalities are present and the final effective weights are all high, the second variance component approaches zero, and the total confidence is determined solely by the inherent uncertainty of the data.

[0057] The first and second variance components are combined to obtain the total prediction variance. The combination method can be direct summation or weighted summation, so that the total prediction variance can be traced and decomposed into two dimensions: the inherent source of uncertainty and the source of uncertainty caused by missing data.

[0058] When the total variance is large, you can directly check the contribution of each of the two components to determine whether the unreliability of the prediction is due to the complexity of the data itself or to the lack of too many key checks.

[0059] S43: Generate a confidence interval that includes information on missing attribution uncertainty based on the combination of the predicted value and the total predicted variance; Specifically, a confidence interval is constructed with the predicted value as the center and the total prediction variance as the radius. The width of this confidence interval is directly affected by the degree of modality loss; the more severe the loss, the wider the interval, indicating a decrease in the reliability of the prediction. Simultaneously, this step provides the first and second variance components as additional outputs to downstream steps, enabling the prediction evaluation report generated in step S5 to separately display the two sources of prediction uncertainty. This helps determine whether supplementary examinations are needed to reduce uncertainty caused by loss, or whether the current prediction error mainly stems from the complexity of the disease itself and inherent noise in the data.

[0060] S44: Connect the prediction mean estimation subnetwork, the missing conditional variance estimation subnetwork, the missing mechanism perception coding network, and the missing gating fusion module into a complete end-to-end trainable system.

[0061] S5: Based on the predicted values ​​and confidence intervals, generate a prediction and evaluation report with explanations of the impact of missing values.

[0062] The process of generating a prediction and evaluation report with explanations of missing impacts based on the predicted values ​​and confidence intervals includes the following sub-steps: S51: Construct a multi-task joint loss function that combines the supervision loss of the prediction task with the semantic regularization loss of missing embeddings to drive the collaborative optimization of parameters of each module in the entire system.

[0063] The process involves constructing a multi-task joint loss function that combines the supervision loss of the prediction task with the semantic regularization loss of missing embeddings to drive the collaborative optimization of parameters across all modules of the system. This includes the following sub-steps: S511: Construct a task loss function to measure the difference between the model's predictions and the true labels; Specifically, the appropriate loss function is selected based on the specific type of prediction task. For classification tasks such as disease risk prediction, the cross-entropy loss function is used to calculate the difference between the predicted probability distribution output by the model and the true class label on a sample-by-sample basis. For regression tasks such as survival time prediction, the mean squared error loss or the Cox proportional hazards loss function is used to measure the deviation between the predicted value and the actual survival time.

[0064] The task loss function directly affects the predicted value output in step S4 and is the dominant term of the entire system's optimization objective. It drives all learnable parameters of the missing mechanism perception coding network, the missing gated fusion module, the missing condition uncertainty estimation network, and the prediction output layer to be updated in the direction of improving prediction accuracy.

[0065] S512: Construct a semantic regularization loss function for missing embeddings to enhance the semantic structure of missing embedding vectors.

[0066] Specifically, a contrastive regularization loss is introduced for the missing embedding vectors output by the missing encoding branch in step S2. Specifically, missing embedding vectors generated by the same missing cause encoding under different patients and contexts are considered positive sample pairs, while missing embedding vectors corresponding to different missing cause encodings are considered negative sample pairs. During training, by reducing the distance between positive sample pairs and increasing the distance between negative sample pairs, the missing embedding vectors are encouraged to form a clustered distribution in the vector space corresponding to the semantics of the missing cause. For example, missing embedding vectors encoded as "undetected intubation in emergency department" and "undetected in emergency surgery" are close to each other in the feature space because they belong to the same rescue scenario of forced omission, while the missing embedding vector encoded as "undetected in mild outpatient cases" is farther away from the former two because it belongs to a low-risk, voluntary abandonment clinical scenario. This regularization loss ensures that the parameters of the fully connected layer in the missing embedding lookup table and the missing embedding generation function in step S2 are not only indirectly driven by the final prediction task loss but also subject to explicit semantic structure constraints, accelerating the semantic differentiation of missing embedding vectors and ensuring that the missing mechanism perception encoding network learns to distinguish the clinical meaning of different missing causes more quickly and stably.

[0067] The task loss and the missing embedding semantic regularization loss are weighted and combined to form the total loss. A preset balancing coefficient is introduced during the combination. By adjusting the value of this coefficient, a trade-off can be achieved between optimizing prediction accuracy and preserving the semantic structure of missing embeddings, preventing a single optimization objective from becoming too dominant and causing the other objective to degenerate.

[0068] S52: Train all modules jointly in an end-to-end manner, and update all learnable parameters synchronously through unified gradient backpropagation.

[0069] The process involves jointly training all modules in an end-to-end manner, synchronously updating all learnable parameters through unified gradient backpropagation, and includes the following sub-steps: S521: Perform batch splitting on datasets with missing cause labels; Specifically, each batch contains complete multimodal records of several patients, covering different types of missing causes, different numbers of missing modalities, and samples where missing events occurred at different times in the course of the disease, in order to ensure sample diversity and distribution balance during the training process.

[0070] S522: Execute the complete forward propagation path in each training batch; The batch samples are sequentially passed through the missing mechanism perception encoding network in step S2. Content representation vectors are extracted for existing modalities, and missing embedding vectors are generated for missing modalities. All modal vectors are fed into the missing gating fusion module in step S3. Availability coefficients are obtained through the first-level availability gating, and then feedback correction is performed by the inter-modal compensation gating mechanism to obtain the final effective weights. These weights are used to modulate attention interaction to generate multimodal fusion representations. The multimodal fusion representations and the final effective weights are fed into the missing conditional uncertainty estimation network in step S4. The predicted mean estimation subnetwork outputs the predicted value, and the missing conditional variance estimation subnetwork outputs the confidence interval jointly determined by the inherent uncertainty of the data and the uncertainty caused by missing data.

[0071] S523: Calculate the total loss value based on the predicted value and the confidence interval; Using an automatic differentiation mechanism, the gradients of the parameters of each module are calculated in reverse from the total loss value, and all trainable parameters in the prediction output layer, the missing conditional uncertainty estimation network, the missing gated fusion module, and the missing mechanism perceptual coding network are updated in sequence.

[0072] The parameters jointly and synchronously optimized in a unified gradient update step specifically include: the learnable embedding vector corresponding to the embedding lookup operation in the missing encoding branch, the weights and biases of the fully connected layer in the missing embedding generation function, the parameters of the availability gated network in the missing gated fusion module, the parameters of the inter-modal compensation mechanism, the parameters of the two variance estimation branches in the missing conditional uncertainty estimation network, the parameters of the prediction mean estimation sub-network, and the parameters of each modality-specific encoder in the content encoding branch. All of these are jointly and synchronously optimized in this unified gradient update step.

[0073] S524: Traverse the entire training dataset for multiple rounds until the total loss converges to a preset threshold or reaches the specified training round limit, thus obtaining a complete prediction system after training.

[0074] S53: Deploy the trained system in a medical data analysis environment to perform end-to-end inference on non-random missing multimodal data of newly added patients.

[0075] Specifically, for new patients, step S1 is performed to label the causes of missing data, resulting in structured records labeled with these causes. These records are then sequentially passed through a trained missing mechanism perception encoding network, a missing data gating fusion module, and a missing condition uncertainty estimation network to output predicted values, confidence intervals, and the uncertainty components caused by the missing data. The final generated prediction assessment report includes explanations of the impact of missing data; this report contains a predicted prognosis and confidence range for the patient; it clearly identifies the components of the uncertainty caused by the missing specific modalities, the causes of each missing modality, and the directional meaning of the missing event itself for risk prediction.

[0076] When the absence of a key modality leads to a confidence level below the clinically acceptable level, the report can prompt physicians to conduct additional tests to reduce uncertainty, thereby providing clinical decision support based on the perception of missing information.

[0077] Example 2 like Figure 2 As shown, Embodiment 2 of this application provides a medical data analysis system, including: Missing Reason Labeling Module 21: Used to acquire patients' multimodal medical data and record the existence status and missing reasons of each modality, forming a dataset with missing reason labels; Missing semantic encoding module 22: It is used to extract content representation vectors for modalities in the existence state through the missing mechanism perception encoding network, and generate missing embedding vectors rich in missing semantics for modalities in the missing state. Missing Gated Fusion Module 23: It is used to calculate the availability coefficient of each modality through the missing gated fusion network based on the content representation vector and missing embedding vector of each modality, and generate a pure multimodal fusion representation. Missing uncertainty estimation module 24: Used to output predicted values ​​and their confidence intervals determined by the degree of missingness through a missing conditional uncertainty estimation network, based on multimodal fusion representation and availability coefficient. Prediction and evaluation module 25: Used to generate a prediction and evaluation report with explanations of missing impact based on the predicted value and confidence interval.

[0078] Corresponding to the above embodiments, the present invention provides a computer storage medium, including: at least one memory and at least one processor; The memory is used to store one or more program instructions; A processor is used to run one or more program instructions to perform a medical data analysis method.

[0079] Corresponding to the above embodiments, this embodiment of the invention provides a computer-readable storage medium containing one or more program instructions, which are executed by a processor to provide a medical data analysis method.

[0080] The embodiments disclosed in this invention provide a computer-readable storage medium storing computer program instructions that, when executed on a computer, cause the computer to perform the aforementioned medical data analysis method.

[0081] In this embodiment of the invention, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0082] The various methods, steps, and logic diagrams disclosed in the embodiments of this invention can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor reads information from the storage medium and, in conjunction with its hardware, completes the steps of the above methods.

[0083] The storage medium can be memory, such as volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0084] Among them, non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory.

[0085] Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (Synchlink DRAM, SLDRAM), and direct memory bus RAM (DRRAM).

[0086] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0087] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using a combination of hardware and software. When applied as software, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0088] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A medical data analysis method, characterized in that, include: Acquire patients' multimodal medical data and record the existence status and reasons for missing data for each modality to form a dataset with missing data reason labels; By using a missing mechanism-aware encoding network, content representation vectors are extracted for modalities in the present state, and missing embedding vectors rich in missing semantics are generated for modalities in the missing state. Based on the content representation vector and missing embedding vector of each modality, the availability coefficient of each modality is calculated through a missing gating fusion network, and a clean multimodal fusion representation is generated. Using multimodal fusion representation and availability coefficient as conditions, a missing condition uncertainty estimation network is used to output the predicted value and its confidence interval determined by the degree of missingness. Based on the predicted values ​​and confidence intervals, a prediction and evaluation report with explanations of the impact of missing values ​​is generated.

2. The medical data analysis method according to claim 1, characterized in that, By employing a missing mechanism-aware encoding network, content representation vectors are extracted for modalities in the present state, and for modalities in the missing state, missing embedding vectors rich in missing semantics are generated, including: For modalities that are in existence, their raw data is input into the content encoding branch, and the content representation vector is extracted by the dedicated encoder of each modality. For modalities in a missing state, a missing embedding generation function is constructed, which generates missing embedding vectors rich in missing semantics through learnable nonlinear transformations.

3. The medical data analysis method according to claim 1, characterized in that, Based on the content representation vector and missing embedding vector of each modality, the availability coefficient of each modality is calculated through a missing-gated fusion network, and a clean multimodal fusion representation is generated, including: A first-level gating network is constructed based on the content representation vector and missing embedding vector of each modality, and the availability coefficient of each modality is calculated. A second-level gating network is constructed, and the availability coefficients of the first-level gating output are corrected by feedback through the inter-modal compensation matrix to obtain the final effective weights of each mode; The multimodal attention interactions are modulated using the final effective weights to generate a pure multimodal fusion representation.

4. The medical data analysis method according to claim 1, characterized in that, Using multimodal fusion representation and availability coefficients as conditions, a missing condition uncertainty estimation network outputs predicted values ​​and their confidence intervals determined by the degree of missing information, including: Construct a subnetwork for estimating the predicted mean, taking multimodal fusion representation as input and outputting the predicted value; A missing conditional variance estimation subnetwork is constructed, with multimodal fusion representation and the final effective weights of each modality as joint inputs, and the output is the uncertainty variance determined by the degree of missingness; Based on the combination of the predicted values ​​and the total predicted variance, a confidence interval containing information on missing attribution uncertainty is generated. The predictive mean estimation subnetwork, the missing conditional variance estimation subnetwork, the missing mechanism perception coding network, and the missing gating fusion module are connected in series to form a complete end-to-end trainable system.

5. A medical data analysis method according to claim 1, characterized in that, Based on the predicted values ​​and confidence intervals, a prediction and evaluation report with explanations of the impact of missing information is generated, including: A multi-task joint loss function is constructed, which combines the supervision loss of the prediction task with the semantic regularization loss of missing embeddings to drive the collaborative optimization of the parameters of each module in the whole system. All modules are trained jointly in an end-to-end manner, and all learnable parameters are updated synchronously through unified gradient backpropagation. The trained system is deployed in a medical data analysis environment to perform end-to-end inference on non-random missing multimodal data of newly added patients.

6. A medical data analysis method according to claim 5, characterized in that, A multi-task joint loss function is constructed, combining the supervision loss of the prediction task with the semantic regularization loss of missing embeddings to drive the collaborative optimization of parameters of various modules in the entire system, including: Construct a task loss function to measure the difference between the model's predictions and the true labels; We construct a semantic regularization loss function for missing embeddings to enhance the semantic structure of missing embedding vectors.

7. A medical data analysis method according to claim 5, characterized in that, All modules are trained jointly in an end-to-end manner, and all learnable parameters are updated synchronously through unified gradient backpropagation, including: Perform batch splitting on datasets with missing reason labels; In each training batch, the complete forward propagation path is executed; Calculate the total loss value based on the predicted value and the confidence interval; The entire training dataset is iterated through multiple rounds until the total loss converges to a preset threshold or reaches the upper limit of the specified training rounds, thus obtaining a complete prediction system after training.

8. A medical data analysis system, characterized in that, include: The missing cause labeling module is used to acquire patients' multimodal medical data and record the existence status and missing causes of each modality, forming a dataset with missing cause labels; The missing semantic encoding module is used to extract content representation vectors for modalities in the presence state and generate missing embedding vectors rich in missing semantics for modalities in the missing state through the missing mechanism-aware encoding network. The missing-gated fusion module is used to calculate the availability coefficient of each modality through the missing-gated fusion network based on the content representation vector and missing embedding vector of each modality, and generate a clean multimodal fusion representation. The missing uncertainty estimation module is used to output predicted values ​​and their confidence intervals determined by the degree of missingness through a missing conditional uncertainty estimation network, based on multimodal fusion representation and availability coefficient. The missing uncertainty estimation module is used to generate a prediction and evaluation report with explanations of the missing impact based on the predicted value and confidence interval.

9. A computer-readable storage medium, characterized in that, It includes one or more program instructions, which are executed by a processor as described in any one of claims 1-7, for a medical data analysis method.

Images