A physiological function impairment risk prediction method, device and equipment

Abstract

The application relates to a physiological function damage risk prediction method, device and equipment, comprising: acquiring multi-source clinical data and inputting a risk prediction model, the data comprising structured physiological time series data, clinical diagnosis and treatment text data and time information; encoding the structured physiological time series data and the observation time thereof to obtain structured time series representation, and encoding the clinical diagnosis and treatment text data and the recording time of the text unit to obtain a text time-aware representation; constructing a cross-modal time difference matrix based on the difference between the observation time and the recording time and mapping the cross-modal time difference matrix into a cross-modal time bias term; performing bidirectional fusion based on the two types of representations by using a bidirectional cross-attention mechanism, correcting the fusion attention weight by using the cross-modal time bias term; obtaining a cross-modal joint representation based on the features after bidirectional fusion, and outputting a function damage risk probability. The application solves the problem of how to accurately process the time difference between the structured time series dynamic and the clinical diagnosis and treatment text semantics while retaining the structured time series dynamic and the clinical diagnosis and treatment text semantics.

Description

Technical Field

[0001] This application relates to the field of assistive medical technology, and in particular to a method, device and equipment for predicting the risk of physiological function impairment. Background Technology

[0002] Acute kidney injury (AKI) is a highly prevalent and fatal critical illness in intensive care units (ICUs), characterized by a rapid and precipitous decline in kidney function within a short period. Current clinical diagnosis relies on indicators such as serum creatinine and urine output thresholds, which suffers from significant diagnostic lag: by the time these indicators become significantly abnormal, irreversible substantial kidney damage has usually already occurred. With the advancements in structured electronic medical records and artificial intelligence technologies, building intelligent early warning models for AKI progression risk based on multi-source clinical data from the ICU (electronic medical records, vital sign monitoring data, and clinical text records) has become a core research direction in the field of intelligent diagnosis and treatment of critical care illnesses.

[0003] Overall, existing technologies have evolved from rule-based scoring and statistical models to machine learning-based early warning systems based on structured electronic medical records, and further to multimodal intelligent early warning systems based on the fusion of clinical diagnostic texts and structured time-series data. However, in the scenario of assisting in early warning of acute renal function deterioration risk in the ICU, structured physiological data and clinical diagnostic text data exhibit significant asynchrony: the sampling frequencies of physiological indicators, test results, and urine output records are inconsistent, and clinical diagnostic texts are often generated intermittently based on examinations, consultations, or changes in disease progression. For the prediction of renal function impairment or other similar physiological function impairments, how to accurately handle the temporal differences between the two while preserving the dynamics of structured time-series data and the semantics of clinical diagnostic texts remains a problem that existing technologies need to further address. Summary of the Invention

[0004] The purpose of this application is to provide a method for predicting the risk of physiological function impairment, which addresses the problem of accurately handling the temporal differences between structured temporal dynamics and clinical diagnostic texts while preserving their semantics.

[0005] In a first aspect, embodiments of this application provide a method for predicting the risk of physiological function impairment, the method comprising: Acquire multi-source clinical data and input it into the risk prediction model. The multi-source clinical data includes structured physiological time-series data and observation time, clinical diagnosis and treatment text data and the recording time of its internal text units; The structured time series representation is obtained by encoding the structured physiological time series data and observation time using a risk prediction model. The risk prediction model encodes the clinical diagnosis and treatment text data and the recording time to obtain a text time-aware representation. Based on the difference between the observation time and the recording time, a cross-modal time difference matrix is ​​constructed using a risk prediction model, and the cross-modal time difference matrix is ​​mapped to the cross-modal time bias term corresponding to the observation time and the text unit. Based on the structured temporal representation and textual time-aware representation, the risk prediction model employs a bidirectional cross-attention mechanism for bidirectional fusion. During the fusion process, the cross-modal temporal bias term is used to correct the fusion attention weights. A cross-modal joint representation is obtained based on the bidirectional fusion features through a risk prediction model, and the probability of functional impairment risk is output based on the cross-modal joint representation.

[0006] In some possible embodiments, the method further includes: Based on the cross-modal joint representation, the risk prediction model predicts and outputs the trajectory of key indicators in the future. Determine the probability of trajectory deterioration risk in the trajectory mapping of the key indicators; Evaluate the consistency between the probability of functional impairment risk and the probability of trajectory deterioration risk.

[0007] In some possible embodiments, the physiological impairment is kidney damage, and the key indicators include serum creatinine and urine output.

[0008] In some possible embodiments, the encoding based on the structured physiological time-series data and observation time to obtain a structured time-series representation includes: The physiological variables in the structured physiological time series data are determined, and the structured time validity characterization is performed at any observation time. The structured time validity characterization includes the observed value, the missing mask, and the time delay. The structured time validity representations for different observation times are encoded separately and arranged in chronological order to obtain the structured time series representation; Wherein, the observation value is a true or filled observation value, the missing mask is used to indicate whether the observation value is true, and the time delay is used to indicate the time since the last true observation value.

[0009] In some possible embodiments, the encoding based on the clinical diagnosis and treatment text data and the recording time to obtain a text time-aware representation includes: For any given text unit, determine the time interval between that text unit and other text units; A multidimensional polynomial time base is constructed based on the time interval, and a mapping transformation is performed on the multidimensional polynomial time base to generate a time bias weight matrix. Using the time bias weight matrix as a time bias term, all text units are encoded using a time-aware self-attention encoding method with a time bias term to obtain a time-aware representation of the text.

[0010] In some possible embodiments, constructing the cross-modal time difference matrix based on the difference between the observation time and the recording time includes: The difference between any observation time and any recording time is used as the cross-modal difference to obtain the cross-modal time difference matrix; or The absolute value of the difference between any observation time and any recording time is used as the cross-modal difference to obtain the cross-modal time difference matrix.

[0011] In some possible embodiments, mapping the cross-modal time difference matrix to the cross-modal time bias term corresponding to the observation time and the text unit includes: After normalizing the cross-modal differences in the cross-modal time difference matrix, a time difference feature vector corresponding to the cross-modal differences is constructed. The time difference feature vector is mapped using a mapping function to obtain the cross-modal time bias term corresponding to any observation time and any text unit; The time difference feature vector includes a constant term, a directional time difference, a time distance, and a time decay index. The time decay index is used to represent the degree of time proximity as the time distance increases.

[0012] In some possible embodiments, the bidirectional cross-attention mechanism is used for bidirectional fusion, and the fusion attention weights are corrected using the cross-modal temporal bias term during the fusion process, including: In the direction of structured temporal query text, the structured temporal representation is used as the query end, the text time-aware representation is used as the key value end, the first attention score is determined by the attention mechanism, and the first attention score is corrected by the cross-modal temporal bias term. The modified first attention score is normalized to obtain the first fusion attention weight. The first fusion attention weight is used to weight the text time-aware representation and then fused with the structured temporal representation. In the direction of structured temporal text query, the text time-aware representation is used as the query end, the structured temporal representation is used as the key value end, the second attention score is determined by the attention mechanism, and the second attention score is corrected by the cross-modal temporal bias term. The corrected second attention score is normalized to obtain the second fusion attention weight. The structured temporal representation is then weighted using the second fusion attention weight and fused with the text time-aware representation.

[0013] In some possible embodiments, the method further includes: Based on the cross-modal time difference between any observation time and any text unit and the fusion attention weight, the evidence association score between the observation time and the text unit is determined. Based on the evidence association score, select and output evidence pairs formed by observation time and text unit that meet the pairing requirements.

[0014] In some possible embodiments, the method further includes: Acquire training samples, which include multi-source clinical data samples, labeled risk tags, and key indicator trajectory tags; The training samples are input into the risk prediction model to be trained, and the model outputs the probability of functional impairment risk and the prediction trajectory of key indicators. The loss function is determined based on the difference between the predicted probability of functional impairment risk and the labeled risk tag, and the difference between the predicted trajectory of the key indicator and the label of the key indicator trajectory. The model parameters of the risk prediction model are adjusted using the loss function.

[0015] In some possible embodiments, the method further includes: Using the trajectory deterioration risk mapping function, the predicted trajectory of the key indicator is mapped to the predicted probability of trajectory deterioration risk; The degree of consistency between the trajectory deterioration risk and the functional impairment risk is determined by a distance metric function, and the model parameters are adjusted with the goal of meeting the consistency requirements.

[0016] In some possible embodiments, the training samples further include unimodal samples, which only include structured physiological time-series data, observation time, labeled risk tags, and key indicator trajectory labels; the method further includes: Consistency loss is determined based on the differences in the predicted probabilities of functional impairment risks obtained using the multimodal samples and the predicted trajectories of key indicators obtained using the single-modal samples. The model parameters of the risk prediction model are adjusted using the consistency loss.

[0017] Secondly, embodiments of this application provide a device for predicting the risk of physiological function impairment, the device comprising: The data input module is used to acquire multi-source clinical data and input it into the risk prediction model. The multi-source clinical data includes structured physiological time-series data and observation time, clinical diagnosis and treatment text data and the recording time of its internal text units. The first encoding module is used to encode the structured physiological time-series data and observation time using a risk prediction model to obtain a structured time-series representation. The second encoding module is used to encode the clinical diagnosis and treatment text data and the recording time through the risk prediction model to obtain a text time-aware representation. The time bias module is used to construct a cross-modal time difference matrix based on the difference between the observation time and the recording time through a risk prediction model, and to map the cross-modal time difference matrix into cross-modal time bias terms corresponding to the observation time and the text unit. The bidirectional fusion module is used to perform bidirectional fusion based on the structured temporal representation and the text time-aware representation through the risk prediction model, using a bidirectional cross-attention mechanism. During the fusion process, the cross-modal time bias term is used to correct the fusion attention weights. The risk prediction module is used to obtain a cross-modal joint representation based on the bidirectional fusion features through the risk prediction model, and output the probability of functional impairment risk based on the cross-modal joint representation.

[0018] Thirdly, another embodiment of this application also provides a physiological function impairment risk prediction device, including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any physiological function impairment risk prediction method provided in the embodiments of this application.

[0019] Fourthly, another embodiment of this application also provides a computer storage medium storing a computer program for causing a computer to execute any of the physiological function impairment risk prediction methods provided in the embodiments of this application.

[0020] The physiological function impairment risk prediction method provided in this application can achieve explicit time alignment and bidirectional information fusion between structured physiological time sequence and clinical diagnosis and treatment text under the conditions of irregular sampling of multi-source clinical data in ICU and asynchronous generation of clinical diagnosis and treatment text, thereby improving the time sensitivity, stability and physiological rationality of the auxiliary early warning of acute renal function deterioration risk.

[0021] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart of the method for predicting the risk of physiological function impairment provided in the embodiments of this application; Figure 2 The framework diagram corresponding to the risk prediction model used in the physiological function impairment risk prediction method provided in this application embodiment; Figure 3 Detailed flowchart of the physiological function impairment risk prediction method provided in the embodiments of this application; Figure 4 A structural diagram of the physiological function impairment risk prediction device provided in the embodiments of this application; Figure 5 A structural diagram of the physiological function impairment risk prediction device provided in the embodiments of this application. Detailed Implementation

[0024] To further illustrate the technical solutions provided in the embodiments of this application, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of this application provide method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on conventional or non-inventive effort. For steps that do not logically have a necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of this application. In actual processing or when the control device executes the method, it may be executed sequentially or in parallel according to the method shown in the embodiments or drawings.

[0025] Existing technologies for predicting or providing early warning of the risk of acute kidney injury or other similar physiological impairments can be broadly categorized into the following three types: The first category is methods based on clinical rules and statistical models. These methods typically establish scoring tables, regression models, or risk stratification models based on clinical guidelines, expert experience, or fixed risk factors. Their advantages include simplicity of implementation, strong interpretability, and applicability to risk assessment in specific populations or clinical scenarios. However, these methods rely heavily on pre-defined variables and fixed rules, making it difficult to reflect the continuously changing condition of ICU patients.

[0026] The second category is machine learning or deep learning methods based on structured electronic medical record data. These methods typically organize ICU patients' vital signs, test results, urine output, medications, and treatment records within the observation window into structured or temporal features, and then use machine learning models or temporal neural networks to output the risk of AKI over a future period. Compared to traditional rule-based methods, this type of method can utilize more clinical variables and learn complex nonlinear relationships, making it suitable for dynamic risk prediction.

[0027] The third category is based on the fusion of clinical diagnostic texts and multimodal electronic medical records. Clinical diagnostic texts such as radiology reports, progress notes, nursing records, and consultation records for ICU patients contain a wealth of clinical semantic information that is difficult to express directly using structured indicators, such as infection, obstruction, volume status, perfusion abnormalities, disease progression, and treatment adjustments. Therefore, existing methods have begun to extract clinical diagnostic text information using natural language processing models or multimodal deep learning models and combine it with structured physiological time-series data for risk prediction. This type of method can, to some extent, supplement the clinical contextual information beyond structured data.

[0028] While existing methods for predicting or assisting in the early warning of acute kidney injury risk can utilize clinical data for risk modeling from different perspectives, they still have the following shortcomings from the perspective of actual application needs in the ICU: First, existing methods are insufficient in representing the irregular sampling and temporal validity of ICU data. Different physiological indicators of ICU patients have different sampling frequencies; some indicators are recorded frequently, while others are measured at longer intervals or on demand. Existing methods typically organize data into feature sequences within fixed time windows or at a uniform time granularity, which can easily lead to the misinterpretation of values ​​that have not been updated for a long time as currently valid observations, and may also weaken risk signals that appear in a short period of time.

[0029] Second, existing multimodal methods are insufficient in modeling the asynchronous relationship between structured physiological time series and clinical diagnostic texts. Although existing solutions can incorporate clinical diagnostic texts, they mostly employ methods such as structured text extraction, overall text encoding, or post-feature fusion, making it difficult to retain the explicit correspondence between text recording time and physiological observation time.

[0030] It is evident that the methods in related technologies focus more on "whether a certain type of clinical information appears in the text" and less on "which physiological time point is most relevant to the text information", thus failing to solve the cross-modal time mismatch problem of asynchronous data in the ICU.

[0031] In summary, the main shortcomings of existing technologies can be summarized as follows: first, insufficient sampling of irregular ICU structured data and inadequate representation of temporal validity; second, lack of a fine-grained cross-modal temporal alignment mechanism for the asynchronous recording relationship between structured physiological time series and clinical diagnostic texts. Therefore, a physiological function impairment risk prediction method is needed to achieve asynchronous cross-modal temporal alignment for auxiliary early warning of acute renal function deterioration risk in the ICU. This method should more accurately model the temporal correspondence between structured physiological time series and clinical diagnostic text semantics, and improve the physiological rationality and stability of risk output.

[0032] like Figure 1 As shown in the embodiment of this application, a method for predicting the risk of physiological function impairment is provided, the method comprising: Step 101: Obtain multi-source clinical data and input it into the risk prediction model. Multi-source clinical data includes structured physiological time-series data and observation time, clinical diagnosis and treatment text data and the recording time of its internal text units; This application's embodiments utilize a pre-trained risk prediction model to predict the risk of physiological function impairment, such as... Figure 2 As shown, the risk prediction model includes an input layer, a core module layer, and an output layer. The input layer inputs asynchronous multi-source clinical data from the ICU patient observation window into the core module layer for processing. The asynchronous multi-source clinical data includes at least structured physiological time-series data, clinical diagnosis and treatment text data, and corresponding time information.

[0033] The structured physiological time-series data includes physiological data that changes over time, such as vital signs, test results, and urine output records; the clinical diagnosis and treatment text data includes unstructured clinical documents such as radiology reports, medical records, nursing records, and consultation records; and the time information includes the observation time of the structured physiological time-series data and the recording time of the text units within the clinical diagnosis and treatment text data.

[0034] Step 102: The risk prediction model is used to encode the structured physiological time series data and observation time to obtain the structured time series representation; like Figure 2 As shown, the core module layer includes an asynchronous clinical data input and time stamping module, an intra-branch time characterization encoding module, a cross-modal time difference matrix construction module, and a time bias bidirectional cross-fusion module.

[0035] The asynchronous clinical data input and time stamping module is used to uniformly organize structured physiological time-series data and clinical diagnosis and treatment text data from different sources, and retain the time information corresponding to each data item. This is part of the data processing process. This module does not simply force the data from different modalities to the same time point, but retains the observation time of the structured physiological time-series data and the recording time of the text units within the clinical diagnosis and treatment text data, providing a foundation for subsequent cross-modal time alignment.

[0036] The intra-branch time representation encoding module includes a structured physiological time series branch and a clinical diagnosis and treatment text branch. The structured physiological time series branch is used to encode based on structured physiological time series data and observation time to obtain a structured time series representation. The clinical diagnosis and treatment text branch is used to encode based on clinical diagnosis and treatment text data and recording time to obtain a text time-aware representation.

[0037] Step 103: Encode the text based on clinical diagnosis and treatment text data and recording time using a risk prediction model to obtain a text time-aware representation; In this embodiment, after sampling clinical diagnosis and treatment text data, text units are divided based on text fragments or words in the clinical diagnosis and treatment text data, and the recording time bound to the text units is determined. Thus, based on the text units and recording time, a text time-aware semantic representation is constructed through clinical diagnosis and treatment text branches to solve the problems of irregular data sampling and time expression within a single modality.

[0038] Step 104: Construct a cross-modal time difference matrix based on the difference between observation time and recording time using a risk prediction model, and map the cross-modal time difference matrix to the cross-modal time bias terms corresponding to the observation time and text unit. A cross-modal time difference matrix construction module is used to construct a cross-modal time difference matrix between the observation time of structured physiological time series data and the recording time of text units. This matrix is ​​used to represent the proximity or lag relationship between the observation time of any structured physiological time series data and the recording time of any clinical diagnosis and treatment text segment.

[0039] Step 105: Based on structured temporal representation and textual time-aware representation, the risk prediction model is used to perform bidirectional fusion using a bidirectional cross-attention mechanism. During the fusion process, the cross-modal temporal bias term is used to correct the fusion attention weights. Structured temporal representation and text-time-aware representation are two distinct features. A bidirectional cross-fusion module with temporal bias is used, employing a bidirectional cross-attention mechanism for the fusion process. This includes two fusion directions: structured temporal query text direction and text query structured temporal direction. In the structured temporal query text direction, the structured temporal representation is used as the query endpoint, and clinical diagnostic text data is used as the key-value endpoint. The clinical diagnostic text data is then fused into the structured temporal representation according to its corresponding weights. In the text query structured temporal direction, the clinical diagnostic text data is used as the query endpoint, and the structured temporal representation is used as the key-value endpoint. The structured temporal representation is then fused into the clinical diagnostic text data according to its corresponding weights. Finally, the two representations are concatenated and fused to obtain the joint representation after bidirectional fusion.

[0040] By utilizing a time-biased bidirectional cross-fusion module, the cross-modal time difference matrix is ​​mapped to a cross-modal time bias term, and this cross-modal time bias term is added to the bidirectional cross-attention calculation process. The cross-modal time bias term is used to correct the fusion attention weight, so that the fusion attention weight between structured physiological time series and clinical diagnosis and treatment text is simultaneously affected by content relevance and time correspondence.

[0041] Step 106: Obtain cross-modal joint representation based on bidirectional fusion features through the risk prediction model, and output functional impairment risk probability based on cross-modal joint representation. Specifically, the functional impairment risk probability can be output through the output layer.

[0042] In this application embodiment, physiological functional impairment can be, but is not limited to, kidney injury. Therefore, a risk prediction model can be used to output the probability of acute kidney function deterioration (AKI), enabling auxiliary early warning for AKI risk in the ICU. This addresses the problems of irregular sampling of structured physiological data, asynchronous recording of clinical diagnostic text, and the lack of explicit temporal correspondence during the fusion of these two methods in existing AKI risk prediction methods. This application embodiment reflects the temporal validity of physiological variables in structured physiological time-series data through structured temporal representation, and reflects the temporal-aware semantics of text units in clinical diagnostic text data through text time-aware representation. Furthermore, a cross-modal time alignment relationship is established based on the time difference between observation and recording times. This allows the model to consider not only the relevance of feature content but also the temporal proximity or lag relationship between the two when fusing structured physiological time-series data and clinical diagnostic text data. Under the conditions of irregular sampling of multi-source clinical data in the ICU and asynchronous generation of clinical diagnostic text, explicit time alignment and bidirectional information fusion between structured physiological time-series data and clinical diagnostic text can be achieved, thereby improving the time sensitivity, stability, and physiological rationality of the AKI risk auxiliary early warning.

[0043] Existing methods for predicting or assisting in the early warning of acute kidney injury (AKI) risk lack consistency constraints between AKI risk models and the trajectory of key indicators of renal function. These key indicators, used to map whether renal function is deteriorating, can include, but are not limited to, serum creatinine and urine output. Clinical assessment of AKI is closely related to increases in serum creatinine and decreases in urine output, but most models primarily output risk probabilities or risk levels without explicitly constraining the risk output to be consistent with future trends in serum creatinine and urine output. This may lead to models over-reliance on certain statistically relevant features, resulting in risk outcomes inconsistent with the actual evolution of renal function, thus affecting the physiological rationality and clinical reliability of the early warning results.

[0044] In some possible embodiments, a risk prediction model is used to predict and output the trajectory of key indicators in the future based on cross-modal joint characterization. The trajectory of key indicators is related to whether physiological function is impaired. The probability of trajectory deterioration risk mapped by the key indicator trajectory is determined. Specifically, the probability of trajectory deterioration risk can be determined by using a preset trajectory deterioration risk mapping function, or the key indicator trajectory can be output and the probability of trajectory deterioration risk indicated by clinical personnel can be obtained. The consistency between the probability of functional impairment risk and the probability of trajectory deterioration risk is evaluated.

[0045] In some possible embodiments, the physiological impairment is kidney damage, with key indicators including serum creatinine and urine output. The output layer is further used to output the serum creatinine trajectory and urine output trajectory.

[0046] like Figure 2 As shown, the core module layer also includes a physiological function trajectory consistency early warning output module. This module outputs the probability of functional impairment risk and key indicator trajectories based on cross-modal joint characterization. For example, it outputs the probability of acute renal function deterioration risk to assist in early warning. It also outputs serum creatinine and urine output trajectories and constrains the trend of functional impairment risk probability to be consistent with the trend of key indicator trajectory changes. Through this module, the model output no longer only outputs isolated functional impairment risk probabilities, but also cross-validates with core physiological trajectories related to physiological function deterioration, thereby improving the physiological rationality and clinical credibility of the auxiliary early warning results.

[0047] This application's embodiments introduce a consistency constraint between the output of acute renal function deterioration risk probability and changes in serum creatinine trajectory and urine volume trajectory, making the auxiliary early warning results more consistent with the evolution of renal function, thereby improving the time sensitivity, stability, and physiological rationality of the auxiliary early warning of acute renal function deterioration risk in ICU scenarios.

[0048] like Figure 3 As shown, taking kidney injury as an example of physiological functional impairment, and combining the calling process of each module in the risk prediction model, the detailed flow of the physiological functional impairment risk prediction method of this application embodiment is given, specifically including: Step 301: Call the asynchronous clinical data input and time stamping module to perform data preprocessing: obtain structured physiological time series data, clinical diagnosis and treatment text data and corresponding time information in the ICU patient observation window, and establish time stamps for various types of data: retain the actual observation time of structured physiological time series data, and retain the actual recording time of the text unit of clinical diagnosis and treatment text data.

[0049] Step 302: Invoke the intra-branch time representation encoding module to perform feature encoding: perform branch encoding on structured physiological time series data and clinical diagnosis and treatment text data respectively. Specifically, for structured physiological time series data, generate a structured time series representation that reflects the observed value, missing state and the time since the last valid observation; for clinical diagnosis and treatment text data, generate a text time-aware representation that includes the semantics of text units and the recording time relationship of text units.

[0050] Step 303: Call the cross-modal time difference matrix construction module to construct the cross-modal time difference matrix: This module constructs the cross-modal time difference matrix based on the difference between the observation time and the recording time. Through this matrix, the temporal proximity or lag relationship between the observation time point of any structured physiological time series data and any text unit can be clearly defined.

[0051] Step 304: Call the time bias bidirectional cross-fusion module to perform bidirectional fusion: This module maps the cross-modal time difference matrix into a cross-modal time bias term and adds the cross-modal time bias term to the bidirectional cross-attention fusion process; thus, the structured physiological time series branch can query clinical diagnosis and treatment texts based on semantic relevance and temporal relationship, and the clinical diagnosis and treatment text branch can also query the structured physiological time series in reverse, thereby forming a bidirectional complementary cross-modal joint representation.

[0052] Step 305: The physiological function trajectory consistency early warning output module is invoked to output the acute renal function deterioration risk probability, serum creatinine, and urine output trajectories. This module outputs the acute renal function deterioration risk probability based on the bidirectional fusion cross-modal joint representation to assist in early warning, and simultaneously outputs the predicted trajectories of serum creatinine and urine output. Through consistency constraints between the acute renal function deterioration risk probability output and the serum creatinine and urine output, the model output results are made consistent with the physiological evolution of acute renal function deterioration.

[0053] In some possible embodiments, the method provided in this application further includes: determining the evidence association score between the observation time and the text unit based on the cross-modal time difference and fusion attention weight corresponding to any observation time and any text unit; and selecting and outputting evidence pairings between observation times and text units that meet the pairing requirements based on the evidence association score.

[0054] Specifically, the evidence association score of the evidence pairing formed by structured physiological time-series data and clinical diagnosis and treatment text data can be calculated using the bidirectional attention score, cross-modal time difference, and fusion attention weight corresponding to any observation time and any text unit. The evidence pairings are then sorted according to their evidence association scores, and the top K evidence pairings are selected as auxiliary information. The aforementioned cross-modal evidence pairings can include the observation time of the structured physiological time-series data, the relevant physiological variables or summaries of physiological changes, the text units of the clinical diagnosis and treatment text data, the recording time of the text units, the time difference between the two, and the corresponding evidence association score.

[0055] This approach enables the output of the probability of acute renal function deterioration, while simultaneously providing cross-modal temporal evidence related to the risk output. This evidence can assist clinicians in tracing key physiological changes and related clinical diagnostic and treatment text information that the model focuses on, thereby facilitating interpretation and outcome traceability.

[0056] The following describes the possible implementation methods of the physiological function impairment risk prediction method in this application, based on the architecture and module calling process of the risk prediction model provided in this implementation. This mainly includes asynchronous clinical data input and time stamping, time-validity encoding of structured physiological time-series branches, time-aware semantic encoding of clinical diagnosis and treatment text branches, construction of cross-modal time difference matrix, generation of cross-modal time bias, bidirectional cross-fusion of time bias, generation of cross-modal joint representation, consistent early warning output of physiological function trajectory, and training of the risk prediction model. Without departing from the core idea of ​​this application, the encoder, cross-modal fusion network, and output network involved in the implementation can be implemented using equivalent substitutions.

[0057] 1) Asynchronous clinical data input and time stamping

[0058] Acquire multi-source clinical data of ICU patients within a preset observation window. This multi-source clinical data includes structured physiological time-series data, clinical diagnosis and treatment text data, and corresponding time information. The structured physiological time-series data can include data on changes over time such as vital signs, laboratory tests, urine output records, medication records, and fluid intake and output records. The actual observation time is retained for each piece of structured physiological data.

[0059] Clinical diagnosis and treatment text data can include unstructured clinical documents such as radiology reports, medical records, nursing records, consultation records, and examination reports. Each clinical diagnosis and treatment text retains its actual recording time; when a clinical diagnosis and treatment text is segmented into text units such as text fragments or words, the recording time bound to the text unit is obtained based on the time when the text fragment or word was generated.

[0060] This application does not force the structured physiological time-series data and clinical diagnosis and treatment text data to the same time point. Instead, it retains the observation time of the structured physiological variables and the recording time of the clinical diagnosis and treatment texts respectively. This processing method can provide the original time basis for the subsequent construction of the cross-modal time difference matrix.

[0061] 2) Time-effectiveness coding of structured physiological time-series branches

[0062] The embodiments of this application construct a structured time series representation based on the observed values, missing masks, and time delays of physiological variables in structured physiological time series data.

[0063] In some possible embodiments, structured time-series data and observation time are encoded to obtain a structured time-series representation, including: Identify physiological variables in structured physiological time-series data and perform structured time validity representation at any observation time. The structured time validity representation includes observations, missing masks, and time delays. The structured time validity representations of different observation times are encoded separately and arranged in chronological order to obtain the structured time series representation; In this system, observed values ​​represent the values ​​of physiological variables at the current observation time, whether true or filled. When a true observed value is missing at a certain observation time, the missing position can be filled with forward-looking physiological variable values, zero-filling, or the mean of the physiological variable within the observation time window. A missing value mask indicates whether the observed value is true, and a time lag indicates the time elapsed since the last true observed value. The validity of an observed value at any observation time is characterized by both the missing value mask and the time lag. For any observation time, an observed value vector is obtained based on the observed values ​​of different physiological variables, a missing value mask vector is obtained based on the missing values ​​of different physiological variables, and a time lag value vector is obtained based on the time lag values ​​of different physiological variables.

[0064] Let the first The vector of observation values ​​for each observation time is The missing mask vector is The time delay vector is The structured temporal effectiveness representation is obtained through feature concatenation, and the specific formula is as follows:

[0065] In the formula, This is the feature splicing operation operator.

[0066] Regarding the first For each physiological variable, the time lag of a single variable is calculated using a recursive method. The specific segmented calculation formula is as follows:

[0067] In the formula, t=1 represents the first observation time. This indicates the time interval between two adjacent observation times. For example, the time interval is uniformly expressed in hours. Indicates the first The observation mask state of each physiological variable at the previous observation time, among which =1 indicates a true observation. =0 indicates missing values. The time lag can be calculated in hours and can also be normalized according to the observation window length or a preset time scale. Through this time lag feature, the model can effectively distinguish between recent true observations and long-term unupdated imputed values ​​in time series data, accurately representing the time validity attributes of structured physiological time series data.

[0068] In some possible embodiments, the structured temporal validity representations of each observation time are input into the encoder corresponding to the structured physiological time series branch to obtain a structured time series representation sequence arranged in chronological order, as follows:

[0069] In the formula, This represents the encoder corresponding to the structured physiological timeline branch. This represents the structured time series representation of the output. This indicates the total number of time points in the structured physiological time series data within the observation window.

[0070] The encoder described above can employ any temporal coding model, such as a Gate Recurrent Unit (GRU), a Long Short-Term Memory (LSTM) network, a temporal convolutional network, or a temporal Transformer. One possible implementation is to use a unidirectional GRU for temporal modeling. The reason for using a unidirectional structure is that the physiological function deterioration risk early warning task can only utilize data before the prediction time and cannot incorporate information after the prediction time. Through a unidirectional GRU, the dynamic trends of ICU physiological indicators over time can be captured while maintaining causal constraints.

[0071] 3) Time-aware semantic coding of clinical diagnosis and treatment text branches

[0072] This application embodiment preprocesses and segments the original clinical diagnosis and treatment text data into words and sentences to obtain an ordered sequence of text units such as text fragments or word units, and binds each text unit to a corresponding real record time.

[0073] In some possible embodiments, text time-aware representations are obtained by encoding based on clinical diagnosis and treatment text data and recording time, including: For any given text unit, determine the time interval between that text unit and other text units; let the first... The recording time for each text unit is , No. The recording time for each text unit is The formula for calculating the internal time interval between two text units is:

[0074] A multidimensional polynomial time basis is constructed based on time intervals, and a mapping transformation is performed on the multidimensional polynomial time basis to generate a time bias weight matrix; based on the above... The text unit and the first The time intervals of each text unit are normalized to construct a corresponding multidimensional polynomial time basis. Normalization methods can include normalizing the total duration of the observation window or normalizing the maximum time interval, mapping the time intervals to the [0,1] interval. The normalized intra-text time interval is denoted as... The specific expression for the multidimensional polynomial time basis is:

[0075] In the formula, d represents the order of the time base, which is typically 3 or 4 and can be adjusted according to the scenario; Using the time bias weight matrix as the time bias term, and employing a time-aware self-attention encoding method with a time bias term, all text units are encoded to obtain a time-aware representation of the text.

[0076] The clinical diagnosis text branch in this application includes multiple attention heads. Combined with learnable model parameters, a multidimensional polynomial time base is mapped and transformed to generate a first attention head adapted to the multiple attention heads. The text unit and the first Time bias weights for each text unit:

[0077] In the formula, Indicates the attention head number. This represents the learnable parameter vector corresponding to the h-th attention head. This represents the Sigmoid activation function. Each attention head independently learns a set of temporal bias weights for different text units, forming a single-head temporal bias weight matrix. .

[0078] The clinical diagnosis and treatment text branch uses the time bias weight matrix as a time bias term and employs a time-aware self-attention encoding method with a time bias term to encode all text units, achieving semantic-temporal joint encoding. The time bias weights of different text units are sequentially incorporated into the self-attention score calculation, and the text semantic association weights are corrected to obtain the text time-aware representation. The specific calculation method is as follows:

[0079] The process begins by converting clinical diagnosis and treatment text data into corresponding text vectors, and then obtaining the query matrix through three learnable linear transformations. Key matrix Sum matrix The time bias weight matrix is ​​used according to the above formula. Self-attention encoding is performed to obtain a time-aware representation of the text. The feature dimension representing the time-aware representation of text, such as 128 dimensions, To prevent numerical overflow and ensure computational stability, this constant is typically set to a minimum value, as shown in the formula above. , , The subscript N is used to identify branches in the clinical diagnosis and treatment text. The subscript N is used to represent the dimension of the corresponding text vector.

[0080] The above encoding method can simultaneously uncover both semantic and temporal relationships within the text, preventing the model from classifying text units with large time spans and no temporal correlation as strongly correlated, thus improving the temporal rationality of the text encoding. After completing the time-aware semantic encoding, a fixed-dimensional time-aware representation of the text is output:

[0081] In the formula, In other words, according to the above The text encoder uses a computational method. Represents a sequence of text units. Indicates the number of text unit sequences. This represents the time-aware representation of the final output text.

[0082] 4) Construction of cross-modal time difference matrix

[0083] After completing the independent coding of the structured physiological time-series branch and the clinical diagnosis and treatment text branch, a cross-modal time difference matrix is ​​constructed to address the time misalignment problem of the two types of asynchronous heteromodal data, thereby quantifying the time-series differences between physiological observations and clinical text records.

[0084] This application embodiment constructs a cross-modal time difference matrix based on the difference between observation time and recording time, including: The difference between any observation time and any recording time is used as the cross-modal difference to obtain the cross-modal time difference matrix; or The absolute value of the difference between any observation time and any recording time is used as the cross-modal difference to obtain the cross-modal time difference matrix.

[0085] Suppose that in the structured physiological time series data, the first... Each observation time is The recording time of the j-th text unit in the clinical diagnosis and treatment text data is... The cross-modal difference between the two for:

[0086] In another possible implementation, the temporal distance, characterized by the absolute time difference, can be used as the cross-modal difference. The calculation formula is:

[0087] A cross-modal time difference matrix is ​​formed by pairwise combinations of all observation time points and the recording time of all text units. :

[0088] In the formula, This matrix is ​​used to accurately describe the temporal proximity and time lag relationship between the recording time of any physiological observation and any clinical text. Unlike traditional single-modal positional encoding and intra-text temporal encoding, this matrix is ​​specifically designed for physiological-text cross-modal fusion scenarios, effectively solving the fusion mismatch problem caused by asynchronous recording times of the two types of data.

[0089] 5) Generation of cross-modal time bias

[0090] This application embodiment will use the cross-modal time difference matrix Mapped to observation time and cross-modal time bias term corresponding to text unit The cross-modal temporal bias term is used to correct the fusion attention weights obtained subsequently based on the bidirectional cross-attention scores, where h represents the attention head number.

[0091] In some possible embodiments, the cross-modal time difference matrix is ​​mapped to cross-modal time bias terms corresponding to the observation time and text unit, including: After normalizing the cross-modal differences in the cross-modal time difference matrix, a time difference feature vector corresponding to the cross-modal differences is constructed. By using a mapping function to map the time difference feature vector, we can obtain the cross-modal time bias term corresponding to any observation time and any text unit. The time difference feature vector includes a constant term, a directional time difference, a time distance, and a time decay exponent. The time decay exponent is used to represent the degree of time proximity that decays as the time distance increases.

[0092] Specifically, for the i-th observation time and the j-th text unit, the cross-modal time difference between the two is first calculated. The values ​​are uniformly converted to hourly units and normalized according to the observation window length or a preset time scale to obtain normalized cross-modal differences:

[0093] in, This indicates the length of the observation window or the preset normalized time scale.

[0094] Then, based on the normalized cross-modal differences, a time difference feature vector corresponding to the cross-modal differences is constructed:

[0095] in, The time difference feature vector is defined by the preset time decay scale parameter. It also includes a constant term 1 and a directional time difference. Time distance and time decay index .in, Used to indicate the chronological order between observation time and recording time. Used to indicate the time distance between the two. It is used to indicate the degree of time proximity that decays as time distance increases.

[0096] In some possible embodiments, the mapping function that maps the time difference feature vector to the cross-modal time bias term is a mapping function that includes multiple attention heads. For the h-th attention head, the corresponding learnable parameter vector is used. By performing a linear mapping on the time difference eigenvectors, we obtain the cross-modal time bias term. :

[0097] In the formula, This represents the cross-modal time bias term corresponding to the i-th observation time and the j-th text unit under the h-th attention head; This represents a mapping function.

[0098] Through this mapping method, the model can learn different temporal bias patterns in different attention heads, so that the cross-modal fusion attention weights are not only affected by the content relevance between the structured temporal representation and the text time-aware representation, but also by their temporal direction and temporal distance.

[0099] In other alternative implementations, the mapping function described above can also be implemented using a multilayer perceptron, a piecewise function, a monotonically decaying function, or a lookup table-based learnable temporal bias function. As long as it can convert the cross-modal difference into a cross-modal temporal bias term for correcting the fusion attention weights, it is an equivalent implementation of the embodiments of this application.

[0100] 6) Time-biased bidirectional cross-fusion

[0101] The structured timing characterization in this application is obtained in accordance with the above method. and text time-aware representation Subsequently, a bidirectional cross-attention mechanism is adopted to achieve bidirectional interactive fusion of bimodal information. At the same time, a cross-modal temporal bias term is introduced to correct the attention score and thus correct the fusion attention weight, taking into account both content matching and temporal matching constraints.

[0102] In some possible embodiments, a bidirectional cross-attention mechanism is employed for bidirectional fusion. During the fusion process, cross-modal temporal bias terms are used to correct the fusion attention weights, including: In the direction of structured temporal query text, the structured temporal representation is used as the query end, and the text time-aware representation is used as the key-value end. An attention mechanism is used to determine the first attention score, and a cross-modal temporal bias term is used to correct the first attention score. By calculating the first attention score, physiological state matching and association with clinical diagnosis and treatment information are achieved. The formula for calculating the first attention score is:

[0103] In the formula, Let h be the query vector for the i-th observation time in the h-th attention head. Let j be the key vector of the j-th text unit. The feature vector dimension of a single attention head. For the cross-modal time bias term corresponding to the i-th observation time and the j-th text unit in the h-th attention head; The modified first attention score is Softmax normalized to obtain the first fusion attention weight. This first fusion attention weight is then used to weight the text time-aware representation and fuse it with the structured temporal representation, thus integrating the structured temporal representation of the clinical diagnostic text data semantics. The specific calculation method for the first fusion attention weight is as follows:

[0104] In the direction of structured temporal text query, the text time-aware representation is used as the query end, and the structured temporal representation is used as the key-value end. An attention mechanism is used to determine the second attention score, and a cross-modal temporal bias term is used to correct the second attention score. In the direction of structured temporal text query, reverse matching of diagnostic texts with corresponding physiological states is achieved. The formula for calculating the second attention score is:

[0105] In the formula, Let be the query vector of the j-th text unit in the h-th attention head. Let be the key vector for the i-th observation time.

[0106] The corrected second attention score is normalized using Softmax to obtain the second fusion attention weight. This second fusion attention weight is then used to weight the structured temporal representation and fuse it with the textual time-aware representation, resulting in a textual time-aware representation that incorporates physiological temporal dynamic features. The specific formula for calculating the second fusion attention weight is as follows:

[0107] The bidirectional cross-fusion mechanism provided in this application has significant advantages over traditional cross-attention: traditional cross-attention only calculates the fusion weight based on the content similarity between text and physiological data, while the fusion attention weight of this application simultaneously couples the content matching weight and the temporal bias weight, which can effectively solve the problem of temporal mismatch of asynchronous clinical data and allow the modality fusion result to simultaneously satisfy semantic relevance and temporal rationality.

[0108] 7) Cross-modal joint characterization generation

[0109] After completing the bidirectional cross-fusion, the embodiments of this application obtain two types of optimized representations: structured temporal representations that fuse the semantics of clinical diagnosis and treatment text data. And text time-aware representation that integrates physiological temporal dynamic features. .

[0110] Pooling and feature concatenation are applied to the two types of optimized representations to unify bimodal features of different sequence lengths into a fixed-dimensional cross-modal joint representation. The specific formula is as follows:

[0111] In the formula, For pooling aggregation operations, mean pooling, max pooling, masked mean pooling, attention pooling, and combinations thereof can be employed. This aggregation method enables deep fusion of multi-source, heterogeneous features, generating cross-modal joint representations adapted for subsequent multi-task prediction. .

[0112] In an optional implementation, the embodiments of this application can also generate cross-modal evidence pairings based on the cross-modal time difference and fusion attention weights corresponding to any observation time and any text unit, as described in the above implementation description, which will not be elaborated here.

[0113] 8) Early warning output of physiological function trajectory consistency

[0114] Cross-modal joint characterization The input physiological function trajectory consistency early warning output module includes multiple output mapping networks, which simultaneously complete the prediction of the probability of physiological function deterioration and the regression prediction of the time-series trajectory of key indicators. Taking kidney injury as an example, the main outputs are as follows: Output of the probability of acute renal function deterioration: ; Serum creatinine prediction trajectory output: ; Urine volume prediction trajectory output: ; in, This represents the probability of acute renal function deterioration. Predicting the trajectory based on serum creatinine. For urine output prediction trajectory, , , These represent the learnable output mapping networks for the corresponding tasks. This is the Sigmoid activation function.

[0115] 9) Risk prediction model training

[0116] The risk prediction model provided in this application is pre-trained using corresponding training samples. In some possible embodiments, training samples are obtained, including multi-source clinical data samples, labeled risk labels, and key indicator trajectory labels. The training samples are input into the risk prediction model to be trained, and the model outputs the predicted probability of functional impairment risk and the predicted trajectory of key indicators. A loss function is determined based on the difference between the predicted probability of functional impairment risk and the labeled risk labels, the difference between the predicted trajectory of key indicators and the key indicator trajectory labels, and the model parameters of the risk prediction model are adjusted using the loss function.

[0117] To further improve the accuracy of the prediction results of the risk prediction model, during the model training phase, the method provided in this application embodiment further includes: using a trajectory deterioration risk mapping function to map the predicted trajectory of key indicators to the predicted probability of trajectory deterioration risk; determining the degree of consistency between the trajectory deterioration risk and the probability of functional impairment risk through a distance metric function, and adjusting the model parameters with the goal of meeting the consistency requirements.

[0118] Taking kidney injury as an example of physiological functional impairment, to reduce the risk that the model's output, relying solely on statistically relevant features, might be inconsistent with the evolution of kidney function, this application constructs a multi-task joint loss function and introduces a consistency constraint between the probability of acute kidney function deterioration and the trend of kidney function trajectory changes. The overall training objective function is:

[0119] In the formula, The loss for predicting the risk of acute renal failure is determined based on the difference between the probability of acute renal failure output by the model and the labeled risk label. The loss of the serum creatinine prediction trajectory is determined by comparing the predicted serum creatinine trajectory output by the model with the labeled serum creatinine trajectory. The urine volume trajectory prediction loss is calculated based on the urine volume prediction trajectory output by the model and the labeled urine volume trajectory. To determine the degree of consistency between the predicted probability of trajectory deterioration risk and the probability of acute renal function deterioration using a distance metric function, a consistency loss is determined based on the degree of consistency. , , These are the adjustable weighting coefficients for each loss term.

[0120] The above-mentioned trajectory deterioration risk mapping function can be used to determine the trajectory deterioration risk prediction probability in the following way: by pre-determining the baseline of the key indicator trajectory, the predicted trajectory of the key indicator is compared with the baseline of the key indicator trajectory to determine the change value, and the change value of the key indicator related to the physiological function damage is retained to obtain the change trend of the key indicator related to the physiological function damage, and the trajectory deterioration risk prediction probability is obtained based on the change trend of the key indicator.

[0121] Specifically, the trajectory deterioration risk prediction probability is constructed using the trajectory deterioration risk mapping function, based on the predicted trajectory of serum creatinine and urine output, so that the acute kidney injury risk probability output by the model is consistent with the kidney function deterioration trend reflected by the predicted trajectory.

[0122] In the formula, To predict the probability of trajectory deterioration risk, This is a trajectory deterioration risk mapping function used to generate trajectory risk based on the upward trend of serum creatinine and the downward trend of urine output.

[0123] In one specific implementation, the end value of the observation window, the mean value of the observation window, or the patient's stable reference value corresponding to the key indicator are used as the baseline of the key indicator trajectory. The change in serum creatinine relative to the serum creatinine trajectory baseline within the observation window is calculated respectively. And the magnitude of change in urine volume relative to the baseline of the urine volume trajectory. .in, A value greater than 0 indicates that serum creatinine has increased from the baseline of the serum creatinine trajectory. <0 indicates a decrease in urine volume compared to the baseline of the urine volume trajectory. The predicted probability of trajectory deterioration can be expressed as:

[0124] in, Used to preserve effective trends of change associated with deteriorating renal function; This indicates the portion of the increase in serum creatinine. This indicates the portion of urine output that has decreased. , These are the non-negative weight parameters of the model. These are the bias parameters of the model.

[0125] Furthermore, the risk of trajectory deterioration is determined using a distance metric function. Probability of functional impairment The degree of consistency is considered to satisfy the training convergence requirement when the degree of consistency is met. In some possible embodiments, the distance metric function is... Using mean square error form:

[0126] In other possible embodiments, the distance metric function can also be in the form of cross-entropy or KL divergence. It can be assumed that functions of other constraints on trajectory deterioration risk and functional impairment risk can be used as distance metric functions, thereby adapting to the needs of different training scenarios.

[0127] This consistency constraint forces the model's risk output to match the actual trend of kidney function deterioration, eliminates unfounded abnormal predictions, and significantly improves the physiological rationality and clinical credibility of the early warning results.

[0128] To further optimize the risk prediction model, improve its robustness in complex real ICU scenarios, and adapt to scenarios with delayed updates, temporary omissions, and unstable quality of clinical text, this application embodiment adds a text omission consistency constraint strategy during the model training phase.

[0129] In some possible embodiments, the training samples mentioned above also include unimodal samples, which only include structured physiological time-series data, observation time, labeled risk labels, and key indicator trajectory labels; the method also includes: determining consistency loss based on the difference in the predicted probability of functional impairment risk obtained using multimodal samples and the difference in the predicted trajectory of key indicators; and adjusting the model parameters of the risk prediction model using the consistency loss.

[0130] Taking kidney injury as an example of physiological functional impairment, two types of training samples are constructed during the training process: one type is a full-modal input sample consisting of structured physiological time-series data and clinical diagnosis and treatment text data, and the other type is a single-modal input sample that retains only the structured physiological data and masks the clinical diagnosis and treatment text data. Let the output of the acute kidney injury risk probability corresponding to the full-modal input be . Key indicator trajectory output is The output of the acute kidney injury risk probability corresponding to the single-modal input is: Key indicator trajectory output is The consistency loss formula for missing text is:

[0131] In the formula, The weight coefficients for trajectory consistency constraints are adjusted based on consistency impairment during training. This training method can reduce the model's dependence on clinical diagnosis and treatment text modalities, ensuring that even in scenarios with missing text, the model can still output stable and accurate early warning results based on structured physiological time-series data.

[0132] Using the physiological function impairment risk prediction method provided in the embodiments of this application, taking kidney injury as an example, the following beneficial effects are achieved: 1) Improve the rationality of multi-source asynchronous data fusion in the ICU. To address the issues of irregular sampling of structured physiological time-series data and the intermittent generation of clinical diagnosis and treatment text data in ICU settings, this paper preserves the structured physiological observation time and the clinical diagnosis and treatment text recording time, and constructs a cross-modal time difference matrix. In this way, the structured physiological time-series data and clinical diagnosis and treatment text data are no longer simply spliced ​​or statically fused, but are fused based on their temporal correspondence, which improves the rationality of multi-source asynchronous clinical data fusion in ICU as a whole. 2) Improve the time sensitivity of auxiliary early warning of the risk of acute renal function deterioration. By mapping cross-modal time differences to time bias terms and introducing a bidirectional cross-fusion process, the model considers both content relevance and recording time relationship when judging the correlation between structured physiological time-series data and clinical diagnosis and treatment text data. As a result, the model can more sensitively capture key time changes related to acute renal function deterioration within the observation window, reduce information mismatch caused by text recording lag or irregular physiological data sampling, capture early signals of renal function deterioration earlier, extend the early warning window, and buy time for clinical intervention. 3) Improve the physiological rationality of auxiliary early warning results While outputting the risk of acute renal function deterioration, the model incorporates serum creatinine and urine output trajectory predictions, and uses risk-trajectory consistency constraints to ensure that the risk output is consistent with the trend of renal function evolution. This approach avoids the model outputting only isolated risk probabilities, and enables the auxiliary early warning results to corroborate the changes in core physiological indicators related to AKI, thereby improving the physiological rationality and clinical credibility of the early warning results. 4) Improve the stability and applicability of the model in real clinical settings. Introducing a three-dimensional representation of "observation-missing mask-time delay" into the structured time series branch naturally adapts to the characteristics of missing ICU data, uneven sampling frequency, and irregular variable updates. The optional text missing consistency training strategy further reduces the model's dependence on the completeness of clinical text, and can still stably output warning results in scenarios with delayed text generation and partial missing text. Overall, this application has a high tolerance for data quality, is adaptable to the information system environments of different medical institutions, and has stronger engineering deployment value and clinical generalization ability.

[0133] In summary, the embodiments of this application construct a full-link time optimization system that includes intramodal time validity representation, intermodal time difference alignment, and output physiological trajectory constraints. It systematically improves the early warning method for acute renal function deterioration in ICU from three levels: data expression, fusion mechanism, and result verification, and has significant improvements in early warning timeliness, result rationality, and scenario adaptability.

[0134] Based on the same inventive concept, this application also provides a device for predicting the risk of physiological function impairment, such as... Figure 4 As shown, the device includes: The data input module 401 is used to acquire multi-source clinical data and input it into the risk prediction model. The multi-source clinical data includes structured physiological time series data and observation time, clinical diagnosis and treatment text data and the recording time of its internal text units. The first encoding module 402 is used to encode structured physiological time-series data and observation time based on the risk prediction model to obtain a structured time-series representation; The second encoding module 403 is used to encode the text data based on clinical diagnosis and treatment text data and recording time through a risk prediction model to obtain a text time-aware representation. The time bias module 404 is used to construct a cross-modal time difference matrix based on the difference between the observation time and the recording time through the risk prediction model, and to map the cross-modal time difference matrix into cross-modal time bias terms corresponding to the observation time and the text unit. The bidirectional fusion module 405 is used to perform bidirectional fusion based on structured temporal representation and text time-aware representation through the risk prediction model, using a bidirectional cross-attention mechanism. During the fusion process, the cross-modal time bias term is used to correct the fusion attention weights. The risk prediction module 406 is used to obtain a cross-modal joint representation based on the bidirectional fusion joint representation through the risk prediction model, and output the probability of functional impairment risk based on the cross-modal joint representation.

[0135] Having described the physiological function impairment risk prediction method and apparatus according to exemplary embodiments of this application, the following describes a physiological function impairment risk prediction device according to another exemplary embodiment of this application. This device may include at least one processor and at least one memory. The memory stores program code that, when executed by the processor, causes the processor to perform the steps in the physiological function impairment risk prediction method according to the various exemplary embodiments of this application described above.

[0136] The following reference Figure 5 This application describes a physiological function impairment risk prediction device 150 according to this embodiment. Figure 5 The physiological function impairment risk prediction device 150 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0137] like Figure 5 As shown, the physiological function impairment risk prediction device 150 is presented in the form of a general electronic device. The components of the physiological function impairment risk prediction device 150 may include, but are not limited to: at least one processor 151, at least one memory 152, and a bus 153 connecting different system components (including memory 152 and processor 151).

[0138] Bus 153 represents one or more of several bus structures, including a memory bus or memory controller, peripheral bus, processor, or local bus using any of the various bus structures.

[0139] The memory 152 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 1521 and / or cache memory 1522, and may further include read-only memory (ROM) 1523.

[0140] The memory 152 may also include a program / utility 1525 having a set (at least one) of program modules 1524, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0141] The physiological function impairment risk prediction device 150 can also communicate with one or more external devices 154 (e.g., keyboard, pointing device, etc.), one or more devices that enable a user to interact with the physiological function impairment risk prediction device 150, and / or any device that enables the physiological function impairment risk prediction device 150 to communicate with one or more other electronic devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 155. Furthermore, the physiological function impairment risk prediction device 150 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 156. As shown, network adapter 156 communicates with other modules used in the physiological function impairment risk prediction device 150 via bus 153. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the physiological function impairment risk prediction device 150, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0142] In some possible implementations, various aspects of the physiological function impairment risk prediction method provided in this application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps in the physiological function impairment risk prediction method according to the various exemplary embodiments of this application described above.

[0143] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0144] The program product for predicting the risk of physiological function impairment according to the embodiments of this application can be a portable compact disc read-only memory (CD-ROM) and include program code, and can run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0145] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0146] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0147] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0148] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for predicting the risk of physiological function impairment, characterized in that, The method includes: Acquire multi-source clinical data and input it into the risk prediction model. The multi-source clinical data includes structured physiological time-series data and observation time, clinical diagnosis and treatment text data and the recording time of its internal text units; The structured time series representation is obtained by encoding the structured physiological time series data and observation time using a risk prediction model. The risk prediction model encodes the clinical diagnosis and treatment text data and the recording time to obtain a text time-aware representation. Based on the difference between the observation time and the recording time, a cross-modal time difference matrix is ​​constructed using a risk prediction model, and the cross-modal time difference matrix is ​​mapped to the cross-modal time bias term corresponding to the observation time and the text unit. Based on the structured temporal representation and textual time-aware representation, the risk prediction model employs a bidirectional cross-attention mechanism for bidirectional fusion. During the fusion process, the cross-modal temporal bias term is used to correct the fusion attention weights. A cross-modal joint representation is obtained based on the bidirectional fusion features through a risk prediction model, and the probability of functional impairment risk is output based on the cross-modal joint representation.

2. The method according to claim 1, characterized in that, Also includes: Based on the cross-modal joint representation, the risk prediction model predicts and outputs the trajectory of key indicators in the future. Determine the probability of trajectory deterioration risk in the trajectory mapping of the key indicators; Evaluate the consistency between the probability of functional impairment risk and the probability of trajectory deterioration risk.

3. The method according to claim 2, characterized in that, The physiological functional impairment is kidney damage, and the key indicators include serum creatinine and urine output.

4. The method according to claim 1, characterized in that, The encoding based on the structured physiological time-series data and observation time to obtain a structured time-series representation includes: The physiological variables in the structured physiological time series data are determined, and the structured time validity characterization is performed at any observation time. The structured time validity characterization includes the observed value, the missing mask, and the time delay. The structured time validity representations for different observation times are encoded separately and arranged in chronological order to obtain the structured time series representation; Wherein, the observation value is a true or filled observation value, the missing mask is used to indicate whether the observation value is true, and the time delay is used to indicate the time since the last true observation value.

5. The method according to claim 1, characterized in that, The encoding based on the clinical diagnosis and treatment text data and the recording time to obtain a text time-aware representation includes: For any given text unit, determine the time interval between that text unit and other text units; A multidimensional polynomial time base is constructed based on the time interval, and a mapping transformation is performed on the multidimensional polynomial time base to generate a time bias weight matrix. Using the time bias weight matrix as a time bias term, all text units are encoded using a time-aware self-attention encoding method with a time bias term to obtain a time-aware representation of the text.

6. The method according to claim 1, characterized in that, The construction of a cross-modal time difference matrix based on the difference between the observation time and the recording time includes: The difference between any observation time and any recording time is used as the cross-modal difference to obtain the cross-modal time difference matrix; or The absolute value of the difference between any observation time and any recording time is used as the cross-modal difference to obtain the cross-modal time difference matrix.

7. The method according to claim 1, characterized in that, The step of mapping the cross-modal time difference matrix to the cross-modal time bias term corresponding to the observation time and text unit includes: After normalizing the cross-modal differences in the cross-modal time difference matrix, a time difference feature vector corresponding to the cross-modal differences is constructed. The time difference feature vector is mapped using a mapping function to obtain the cross-modal time bias term corresponding to any observation time and any text unit; The time difference feature vector includes a constant term, a directional time difference, a time distance, and a time decay index. The time decay index is used to represent the degree of time proximity as the time distance increases.

8. The method according to claim 1, characterized in that, The bidirectional cross-attention mechanism is used for bidirectional fusion, and the cross-modal temporal bias term is used to correct the fusion attention weights during the fusion process, including: In the direction of structured temporal query text, the structured temporal representation is used as the query end, the text time-aware representation is used as the key value end, the first attention score is determined by the attention mechanism, and the first attention score is corrected by the cross-modal temporal bias term. The modified first attention score is normalized to obtain the first fusion attention weight. The first fusion attention weight is used to weight the text time-aware representation and then fused with the structured temporal representation. In the direction of structured temporal text query, the text time-aware representation is used as the query end, the structured temporal representation is used as the key value end, the second attention score is determined by the attention mechanism, and the second attention score is corrected by the cross-modal temporal bias term. The corrected second attention score is normalized to obtain the second fusion attention weight. The structured temporal representation is then weighted using the second fusion attention weight and fused with the text time-aware representation.

9. The method according to claim 1, characterized in that, Also includes: Based on the cross-modal time difference between any observation time and any text unit and the fusion attention weight, the evidence association score between the observation time and the text unit is determined. Based on the evidence association score, select and output evidence pairs formed by observation time and text unit that meet the pairing requirements.

10. The method according to claim 2 or 3, characterized in that, Also includes: Acquire training samples, which include multi-source clinical data samples, labeled risk tags, and key indicator trajectory tags; The training samples are input into the risk prediction model to be trained, and the model outputs the probability of functional impairment risk and the prediction trajectory of key indicators. The loss function is determined based on the difference between the predicted probability of functional impairment risk and the labeled risk tag, and the difference between the predicted trajectory of the key indicator and the label of the key indicator trajectory. The model parameters of the risk prediction model are adjusted using the loss function.

11. The method according to claim 10, characterized in that, Also includes: Using the trajectory deterioration risk mapping function, the predicted trajectory of the key indicator is mapped to the predicted probability of trajectory deterioration risk; The degree of consistency between the trajectory deterioration risk and the functional impairment risk is determined by a distance metric function, and the model parameters are adjusted with the goal of meeting the consistency requirements.

12. The method according to claim 10, characterized in that, The training samples also include unimodal samples, which consist only of structured physiological time-series data, observation time, labeled risk tags, and key indicator trajectory labels; the method further includes: Consistency loss is determined based on the differences in the predicted probabilities of functional impairment risks obtained using the multimodal samples and the predicted trajectories of key indicators obtained using the single-modal samples. The model parameters of the risk prediction model are adjusted using the consistency loss.

13. A device for predicting the risk of physiological function impairment, characterized in that, The device includes: The data input module is used to acquire multi-source clinical data and input it into the risk prediction model. The multi-source clinical data includes structured physiological time-series data and observation time, clinical diagnosis and treatment text data and the recording time of its internal text units. The first encoding module is used to encode the structured physiological time-series data and observation time using a risk prediction model to obtain a structured time-series representation. The second encoding module is used to encode the clinical diagnosis and treatment text data and the recording time based on the risk prediction model to obtain a text time-aware representation. The time bias module is used to construct a cross-modal time difference matrix based on the difference between the observation time and the recording time through a risk prediction model, and to map the cross-modal time difference matrix into cross-modal time bias terms corresponding to the observation time and the text unit. The bidirectional fusion module is used to perform bidirectional fusion based on the structured temporal representation and the text time-aware representation through the risk prediction model, using a bidirectional cross-attention mechanism. During the fusion process, the cross-modal time bias term is used to correct the fusion attention weights. The risk prediction module is used to obtain a cross-modal joint representation based on the bidirectional fusion joint representation through the risk prediction model, and output the probability of functional impairment risk based on the cross-modal joint representation.

14. A device for predicting the risk of physiological function impairment, characterized in that, The method includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-12.

15. A computer storage medium, characterized in that, The computer storage medium stores a computer program that enables the computer to perform the method as described in any one of claims 1-12.

Images