A method and device for injury condition early warning based on multi-modal data fusion

Abstract

The application discloses a kind of based on multi-modal data fusion's injury condition early warning method and device, method includes: obtaining continuous physiological waveform data and relevant structured data;After waveform data is preprocessed, it is converted to two-dimensional image representation by time series topology enhancement algorithm;Structured data is standardized to extract structured features;Through heterogeneous feature correlation weight mechanism, two kinds of features are semantically aligned and complementary fusion;Based on multi-window risk aggregation algorithm, output injury condition early warning result.The application solves the problems of difficulty in waveform and structured data fusion, insufficient use of time series features, and poor early warning stability in the prior art by enhanced image representation of waveform data, deep multi-modal fusion, and multi-scale risk aggregation, improving the accuracy, generalization ability, and engineering practicality of injury condition early warning. It can be widely applied in clinical first aid, intensive care and trauma treatment scenarios.

Description

Technical Field

[0001] This application belongs to the field of multimodal medical data processing and injury risk prediction technology, and in particular relates to an injury early warning method and device based on multimodal data fusion. Background Technology

[0002] In clinical emergency care, intensive care, and trauma treatment scenarios, patients' conditions are characterized by rapid changes, significant fluctuations, and large individual differences. Timely identification of the risk of worsening condition and issuing early warnings are crucial to reducing the probability of adverse events. Injury early warning systems must simultaneously meet the requirements of real-time performance, accuracy, and interpretability; however, current technologies still have significant limitations in data utilization and model building.

[0003] Currently, information sources for injury assessment and early warning mainly fall into two categories: one is the patient's structured information, including age, gender, underlying diseases, vital signs, laboratory test results, rating scales, and medication information; the other is continuous waveform data collected by monitoring equipment, such as electrocardiogram, blood pressure, and respiration. Traditional clinical procedures rely on the comprehensive judgment of medical staff, which is highly dependent on experience. This method struggles to maintain stable identification capabilities when faced with massive amounts of continuous monitoring data, and different personnel have varying understandings of abnormal patterns, leading to insufficient consistency in early warnings. Furthermore, it has limited sensitivity to slowly evolving or early, subtle changes, easily resulting in delayed warnings.

[0004] With the development of medical informatization, some methods attempt to use structured data to build predictive models, and use statistical learning or machine learning models to output risk probabilities or levels. Although structured data is easy to clean, standardize and model, its collection is mostly discrete point-in-time records or low-frequency updates, which makes it difficult to reflect the continuous dynamic changes in the patient's condition. Moreover, affected by sampling frequency, measurement errors and missing values, the model's ability to respond to sudden fluctuations is limited, and it ignores key information such as rhythm, morphological changes and short-term abnormalities contained in waveform signals, affecting the lead time and stability of early warning.

[0005] Another type of analysis method based on waveform data can record changes in physiological processes with high temporal resolution, which is theoretically beneficial for capturing early signs of injury deterioration. However, waveform data has characteristics such as high dimensionality, high noise, and non-stationarity, which poses many difficulties for direct modeling: the original waveform data is large in volume, with a lot of noise and artifacts, requiring complex preprocessing; the waveform morphology and rhythm features have nonlinear and multi-scale characteristics, which traditional manual feature extraction methods cannot fully cover, and their stability is insufficient; waveform data and structured data differ significantly in morphology, dimension, and time alignment, which increases the complexity of joint modeling and engineering deployment.

[0006] To improve the early warning effect, some studies have attempted to fuse two types of data in multiple modes. However, existing fusion methods are mostly simple splicing or high-level coarse-grained fusion, which makes it difficult to fully model the correlation between different modes. This can easily lead to one mode dominating the prediction result while the information of the other mode is diluted, affecting the robustness of the model. At the same time, if the waveform participates in the fusion in the form of a one-dimensional sequence, the model needs to handle temporal feature learning and cross-modal alignment at the same time, which makes training difficult and the generalization ability unstable.

[0007] Therefore, there is an urgent need for a disease early warning technology that can efficiently characterize the temporal features of waveform data and achieve deep fusion of waveform data and structured data, in order to solve the problems of insufficient utilization of temporal features, low fusion efficiency, and poor accuracy and stability of early warning in existing technologies. Summary of the Invention

[0008] This invention addresses the problems of insufficient utilization of temporal features of waveform data, difficulty in effectively integrating waveform data with structured data, and insufficient accuracy and stability in existing injury warning technologies. It provides an injury warning method and device based on multimodal data fusion. By converting one-dimensional continuous physiological waveform data into a two-dimensional image or matrix representation containing temporal correlation and topological structure, implicit temporal dependencies are made explicit, facilitating processing using deep feature extraction algorithms in the field of computer vision. At the same time, through a multimodal deep interactive network, the image-based dynamic waveform features are semantically aligned and fused with the patient's structured features, achieving accurate assessment of physiological risk.

[0009] To address the aforementioned technical problems, a first aspect of this invention discloses a method for injury early warning based on multimodal data fusion, the method comprising: S1, acquire physiological waveform data and structured data; S2, preprocess the physiological waveform data and the structured data to obtain preprocessed physiological waveform data and preprocessed structured data; S3, process the preprocessed physiological waveform data and the preprocessed structured data to obtain injury warning result data.

[0010] As an optional implementation, in a first aspect of the present invention, the preprocessing of the physiological waveform data and the structured data to obtain preprocessed physiological waveform data and preprocessed structured data includes: S21, Perform a first preprocessing on the physiological waveform data to obtain first preprocessed physiological waveform data; S22, the first preprocessed physiological waveform data is processed into an image to obtain preprocessed physiological waveform data; S23, perform a second preprocessing on the structured data to obtain preprocessed structured data.

[0011] As an optional implementation, in a first aspect of the present invention, the first preprocessing of the physiological waveform data to obtain first preprocessed physiological waveform data includes: S211, Denoise the physiological waveform data to obtain denoised physiological waveform data; S212, normalize the denoised physiological waveform data to obtain normalized denoised physiological waveform data; The normalization expression is: , in, This represents the normalized and denoised physiological waveform data; This represents the denoised physiological waveform data; This represents the smallest denoised physiological waveform data in the current waveform segment; This represents the largest denoised physiological waveform data in the current waveform segment; Represents the coefficients of a small constant; S213, the normalized and denoised physiological waveform data is divided into time windows to obtain the first preprocessed physiological waveform data.

[0012] As an optional implementation, in a first aspect of the present invention, the step of performing image processing on the first preprocessed physiological waveform data to obtain preprocessed physiological waveform data includes: S221, the first preprocessed physiological waveform data is converted to obtain physiological waveform graphical data; S222, Perform temporal topology enhancement processing on the physiological waveform graphical data to obtain enhanced physiological waveform graphical data; The expression for the time-series topology enhancement processing is: , ， ， ， ， ， in, This represents the element in the i-th row and j-th column of the enhanced physiological waveform graphical data; Represents the weighting coefficients of physiological waveform graphical data; This represents the element in the i-th row and j-th column of the physiological waveform graphical data; The weight coefficients of the first-order difference matrix are represented. This represents the element in the i-th row and j-th column of the first-order difference matrix of the physiological waveform graphical data; Represents the largest first-order difference matrix; The weight coefficients of the second-order difference matrix are represented. This represents the element in the i-th row and j-th column of the second-order difference matrix of the physiological waveform graphical data; Represents the largest second-order difference matrix; This represents the i-th preprocessed physiological waveform data point; This represents the polar angle corresponding to the i-th first preprocessed physiological waveform data point; This represents the polar angle corresponding to the j-th first preprocessed physiological waveform data point; S223, The enhanced physiological waveform graphical data is mapped to obtain preprocessed physiological waveform data.

[0013] As an optional implementation, in a first aspect of the present invention, the second preprocessing of the structured data to obtain preprocessed structured data includes: S231, perform missing value processing on the structured data to obtain complete structured data; S232, Standardize the complete structured data to obtain standard structured data; S233, the standard structured data is encoded to obtain preprocessed structured data.

[0014] As an optional implementation, in the first aspect of the present invention, the processing of the preprocessed physiological waveform data and the preprocessed structured data to obtain injury warning result data includes: S31, feature extraction is performed on the preprocessed physiological waveform data and the preprocessed structured data to obtain physiological waveform image feature vector and structured data feature vector; S32, Process the feature vector of the physiological waveform image and the feature vector of the structured data to obtain multi-source data fusion feature data; S33, Based on the injury warning model, the multi-source data fusion feature data is processed to obtain injury warning result data.

[0015] As an optional implementation, in the first aspect of the present invention, the processing of the physiological waveform image feature vector and the structured data feature vector to obtain multi-source data fusion feature data includes: S321, Based on the heterogeneous feature association weight acquisition model, the feature vector of the physiological waveform image and the feature vector of the structured data are processed to obtain heterogeneous feature association weight data. The expression for the heterogeneous feature association weight acquisition model is: ， ， ， , , , in, This represents the weight data associated with the heterogeneous features; This represents the Softmax activation function; This represents the dimension of the feature vector of the physiological waveform image; This represents the query vector matrix composed of the structured data feature vectors; The key vector matrix represents the feature vectors of the physiological waveform image. Represents the feature vector of a physiological waveform image; Represents the feature vector of structured data; This represents the transpose of the structured feature vector X; Represents the trace operation of a matrix; This represents the transpose of the feature vector of the physiological waveform image; The covariance matrix represents the feature vector of the physiological waveform image and the feature vector of the structured data; This represents the value of the p-th dimension of the structured feature; This represents the mean of the p-th dimension of the structured feature vector; This represents the value of the q-th dimension of the physiological waveform image feature; This represents the mean of the q-th dimension of the feature vector of the physiological waveform image; This represents the expectation operation; The variance matrix represents the feature vector of the structured data; The variance matrix represents the feature vector of the physiological waveform image; Indicates the dynamic adjustment coefficient; This represents a diagonal matrix composed of the importance weights of the feature vectors of the structured data; S322, Based on the feature enhancement processing model, the feature vector of the physiological waveform image is weighted to obtain image feature enhancement data; The expression for the feature enhancement processing model is: , in, This represents the image feature enhancement data; Represents the feature vector of a physiological waveform image; S323, The image feature enhancement data and the structured data feature vector are concatenated to obtain multi-source data fusion feature data; The concatenation processing expression is: , in, This refers to the multi-source data fusion feature data.

[0016] The second aspect of the present invention discloses an injury early warning device based on multimodal data fusion, the device comprising: a data acquisition module, a data preprocessing module, and a data processing module; The data acquisition module is used to acquire physiological waveform data and structured data; The data preprocessing module is used to preprocess the physiological waveform data and the structured data to obtain preprocessed physiological waveform data and preprocessed structured data. The data processing module is used to process the preprocessed physiological waveform data and the preprocessed structured data to obtain injury warning result data; The data acquisition module, the data preprocessing module, and the data processing module are connected in sequence.

[0017] A third aspect of this invention discloses another injury early warning device based on multimodal data fusion, the device comprising: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute some or all of the steps in the injury early warning method based on multimodal data fusion disclosed in the first aspect of the present invention.

[0018] The fourth aspect of the present invention discloses a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, which, when invoked, execute some or all of the steps in the injury early warning method based on multimodal data fusion disclosed in the first aspect of the present invention.

[0019] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: This invention discloses a method and device for injury early warning based on multimodal data fusion. Addressing the problems of insufficient utilization of temporal features in waveform data, difficulty in effectively fusing waveform data with structured data, and insufficient accuracy and stability in existing injury early warning technologies, this invention provides a method and device for injury early warning based on multimodal data fusion. By converting one-dimensional continuous physiological waveform data into a two-dimensional image or matrix representation containing temporal correlations and topological structures, implicit temporal dependencies are made explicit, facilitating processing using deep feature extraction algorithms from the field of computer vision. Simultaneously, through a multimodal deep interactive network, the image-based dynamic waveform features are semantically aligned and fused with the patient's structured features, achieving accurate assessment of physiological risk. This invention offers the following beneficial effects: 1. By using time-series topology enhancement processing, continuously acquired waveform data is converted into two-dimensional images, enabling the temporal correlation and morphological change characteristics of waveform data to be expressed in a two-dimensional structure. While preserving the temporal characteristics of waveforms, this method improves the stability and learnability of waveform feature expression, overcoming the problem of insufficient feature utilization caused by directly modeling one-dimensional waveform time series in existing technologies.

[0020] 2. By fusing waveform image features and patient structured data features at the feature level, the fusion model effectively solves the fusion difficulties caused by the inconsistency between waveform data and structured data in terms of data form, dimension and time scale. It fully leverages the complementary advantages of the two types of data in terms of dynamic change information and individual difference information, thereby improving the overall prediction and early warning effect.

[0021] 3. By simultaneously introducing waveform data and structured data for joint analysis, it is possible to capture the dynamic changes in the patient's physiological state in real time and reflect the impact of individual patient differences on the development of the injury, thereby improving the generalization ability and stability of the injury early warning model under different patients and different application scenarios.

[0022] 4. A complete technical process has been constructed around the application scenario of injury early warning, from data acquisition, preprocessing, feature extraction, multimodal fusion to early warning output. It can realize the automated assessment and early warning of patient injury risk, reduce the reliance on human experience judgment, improve the timeliness and consistency of early warning, and has strong engineering feasibility and practical application value. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1This is a schematic diagram illustrating an application scenario of the injury early warning device based on multimodal data fusion disclosed in an embodiment of the present invention. Figure 2 This is a schematic diagram of the injury early warning method based on multimodal data fusion disclosed in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an injury early warning device based on multimodal data fusion disclosed in an embodiment of the present invention; Figure 4 This is a schematic diagram of another injury early warning device based on multimodal data fusion disclosed in an embodiment of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0027] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0028] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0029] It should be noted that since the method in this application embodiment is executed in a computer device, the processing objects of each computer device exist in the form of data or information, such as time, which is essentially time information. It is understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data that exist so that the computer device can process them. Specific details will not be elaborated here.

[0030] It should be noted that the artificial intelligence-related technologies that may be involved in this application will be briefly described. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. Artificial intelligence is the study of the design principles and implementation methods of various intelligent machines, enabling machines to have the functions of perception, reasoning, and decision-making.

[0031] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0032] This application provides a method, apparatus, computer device, and computer-readable storage medium for injury early warning based on multimodal data fusion, which will be described in detail below.

[0033] Please see Figure 1 , Figure 1This is a schematic diagram illustrating the application scenario of the injury early warning device based on multimodal data fusion provided in this application within an intensive care unit (ICU). The ICU may include a computer device 100, which integrates the injury early warning device based on multimodal data fusion. Figure 1 Computer equipment in the country.

[0034] In this embodiment, the computer device 100 can be a standalone server, a server network, or a server cluster. For example, the computer device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0035] It is understood that the computer device 100 used in the embodiments of this application can be a device including receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a device may include: cellular or other communication devices having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. Specifically, the computer device 100 may be a desktop terminal or a mobile terminal, and may also be one of a mobile phone, tablet computer, laptop computer, etc.

[0036] Those skilled in the art will understand that Figure 1 The application environment given is merely one application scenario for the solution in this application and does not constitute a limitation on the application scenario of this application. Other application environments may include those that are more complex than those described above. Figure 1 The number of computer devices shown is more or less, for example Figure 1 Only one computer device is shown in the diagram. It is understood that the system may also include one or more other services, which are not limited here.

[0037] In addition, such as Figure 1 As shown, the intensive care system may also include a memory 200 for storing waveform data, structured data, and early warning result data.

[0038] It should be noted that, Figure 1The schematic diagram of the application scenario of the injury early warning device based on multimodal data fusion shown is merely an example. The intensive care system and scenario described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of intensive care systems and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0039] This invention addresses the problems of insufficient utilization of temporal features of waveform data, difficulty in effectively integrating waveform data with structured data, and insufficient accuracy and stability in existing injury warning technologies. It provides an injury warning method and device based on multimodal data fusion. By converting one-dimensional continuous physiological waveform data into a two-dimensional image or matrix representation containing temporal correlation and topological structure, implicit temporal dependencies are made explicit, facilitating processing using deep feature extraction algorithms in the field of computer vision. At the same time, through a multimodal deep interactive network, the image-based dynamic waveform features are semantically aligned and fused with the patient's structured features, achieving accurate assessment of physiological risk. This invention utilizes temporal topology enhancement processing to integrate the fundamental temporal relationships, rates of change, and acceleration features of the original waveform. This overcomes the shortcomings of traditional image conversion, which only retains static temporal relationships, allowing the two-dimensional image to more fully reflect the dynamic evolution of the waveform and enhancing the richness and stability of feature representation. Through a heterogeneous feature association weight matrix formula, the correlation strength between structured features and image features is dynamically calculated. Combining the importance weight of structured features with cross-modal correlation, semantic-level alignment of heterogeneous data is achieved, avoiding the dilution of modal information caused by simple splicing and fully leveraging the complementary advantages of the two types of data. Furthermore, it employs multi-window processing. The risk aggregation formula integrates the warning results of multiple consecutive windows, and combines window length weighting and time decay mechanisms to reduce the impact of single window noise on the warning results, thereby improving the model's ability to identify slowly evolving injuries and the stability of the warning. It constructs a complete technical process from data acquisition, preprocessing, enhanced imaging, deep fusion to multi-window aggregation warning. All formulas have clear physical meaning and adjustable parameters, and can be adaptively adjusted according to different clinical scenarios (emergency care, intensive care, postoperative monitoring), reducing reliance on human experience and improving the timeliness and consistency of warnings, thus having broad clinical application value.

[0040] Example 1 Please refer to 3. Figure 2 This is a schematic diagram of the architecture of a multimodal data fusion-based injury early warning method disclosed in an embodiment of the present invention. Figure 2 The described injury early warning method based on multimodal data fusion is applied to intensive care systems, such as local servers or cloud servers used in intensive care systems; however, this embodiment of the invention is not limited to such applications.Figure 2 As shown, this injury early warning method based on multimodal data fusion includes: S1, acquire physiological waveform data and structured data; It should be noted that the physiological waveform data, including electrocardiogram waveform data, blood pressure waveform data, pulse waveform data, and respiratory waveform data, are continuously collected by monitoring equipment to ensure high temporal resolution. The structured data, including patient basic information (age, gender), underlying diseases, vital sign values, laboratory test indicators, rating scales, and medication information, are obtained from hospital medical information systems (HIS) and laboratory information systems (LIS), covering core information on individual patient differences and providing individual-specific reference for the model. It should be noted that, in this embodiment, the physiological waveform data refers to the electrocardiogram (ECG), arterial blood pressure (ABP), pulse oxygen saturation (SpO2), and respiratory waveform data of 50 critically injured trauma patients continuously collected by a multi-parameter monitor in the intensive care unit (ICU). The structured data refers to the basic information (age, gender), underlying diseases (hypertension, diabetes, no underlying diseases), medication information (sedatives, vasopressors, anti-infectives), vital signs upon admission (heart rate, respiratory rate, body temperature), blood routine indicators (white blood cell count, hemoglobin), and Acute Physiology and Chronic Health Evaluation II (APACHE II) scores of the patients obtained from the hospital's HIS and LIS systems. S2, preprocess the physiological waveform data and the structured data to obtain preprocessed physiological waveform data and preprocessed structured data; S3, process the preprocessed physiological waveform data and the preprocessed structured data to obtain injury warning result data.

[0041] Optionally, the preprocessing of the physiological waveform data and the structured data to obtain preprocessed physiological waveform data and preprocessed structured data includes: S21, Perform a first preprocessing on the physiological waveform data to obtain first preprocessed physiological waveform data; S22, the first preprocessed physiological waveform data is processed into an image to obtain preprocessed physiological waveform data; S23, perform a second preprocessing on the structured data to obtain preprocessed structured data.

[0042] Optionally, the first preprocessing of the physiological waveform data to obtain first preprocessed physiological waveform data includes: S211, Denoise the physiological waveform data to obtain denoised physiological waveform data; It should be noted that in this embodiment, the denoising process uses a high-pass filter to suppress baseline drift and a wavelet thresholding algorithm to suppress random noise while preserving pathological features. S212, normalize the denoised physiological waveform data to obtain normalized denoised physiological waveform data; The normalization expression is: , in, This represents the normalized and denoised physiological waveform data; This represents the denoised physiological waveform data; This represents the smallest denoised physiological waveform data in the current waveform segment; This represents the largest denoised physiological waveform data in the current waveform segment; Represents the coefficients of a small constant; It should be noted that the normalized and denoised physiological waveform data has a value range of [-1, 1]. It should be noted that, in this embodiment, the value of the infinitesimal constant coefficient is... To avoid calculation errors caused by the lack of fluctuation in waveform segment data; It should be noted that in this embodiment, the normalization expression is used to map all denoised waveform data to the range [-1, 1], where... ; S213, the normalized and denoised physiological waveform data is divided into time windows to obtain the first preprocessed physiological waveform data; It should be noted that the time window segmentation refers to the use of a multi-scale dynamic sliding window mechanism, with a step size... The scroll cut-off length is The waveform sequence is used as the learning window, and a prediction window is associated with it. The dynamic evolution trend within provides a standardized segment with complete temporal sequence for subsequent image processing and early warning judgment; among which... , , It can be adaptively adjusted according to clinical scenarios, and the value ranges are as follows: , , The range of values ​​is ; It should be noted that the physiological waveform data undergoes a first preprocessing step to obtain preprocessed physiological waveform data, which eliminates differences in measurement ranges between different monitoring devices and individual physiological baseline deviations, providing numerical stability assurance for angle mapping in the subsequent image conversion process. Finally, a hierarchical dynamic time window segmentation mechanism is introduced, based on a preset learning length. Waveform segments are extracted from the continuous physiological flow and processed in steps. (e.g., a 10-second sliding window) is used to perform high-frequency scrolling to ensure that the model can capture the complete temporal context information of physiological state from micro-fluctuations to macro-evolution.

[0043] Optionally, the step of performing image processing on the first preprocessed physiological waveform data to obtain preprocessed physiological waveform data includes: S221, the first preprocessed physiological waveform data is converted to obtain physiological waveform graphical data; It should be noted that the physiological waveform graphical data mentioned above is Gram matrix data, or Gram angle difference field (GADF), or recursive graph data. Optionally, the conversion process employs the Gram angle field (GAF) algorithm to convert the first preprocessed physiological waveform data into Gram matrix data; It should be noted that in this embodiment, the GAF algorithm is used to perform Gram matrix calculation on the first preprocessed physiological waveform data to obtain Gram matrix data, that is, the physiological waveform graphical data. The expression for calculating the Gram matrix is: , , , in, The first preprocessed physiological waveform data point (i) and the first preprocessed physiological waveform data point (j) are graphically represented to obtain the graphical physiological waveform data. This represents the i-th preprocessed physiological waveform data point; This represents the j-th preprocessed physiological waveform data point. This represents the angle encoding of the i-th first preprocessed physiological waveform data point; Represents timestamp The mapping radius in polar coordinates; N is the total number of waveform data points within a single window; Optionally, the conversion process employs the Gram Angular Difference Field (GADF) algorithm to convert the first preprocessed physiological waveform data into Gram matrix data; , , in, The graph represents the physiological waveform data obtained by graphically representing the i-th first preprocessed physiological waveform data point and the j-th first preprocessed physiological waveform data point; N is the total number of waveform data points in a single window. Optionally, the conversion process employs a recursive graph (RP) algorithm to convert the first preprocessed physiological waveform data into recursive graph data; S222, Perform temporal topology enhancement processing on the physiological waveform graphical data to obtain enhanced physiological waveform graphical data; The expression for the time-series topology enhancement processing is: , , , ， , , in, This represents the element in the i-th row and j-th column of the enhanced physiological waveform graphical data; Represents the weighting coefficients of physiological waveform graphical data; This represents the element in the i-th row and j-th column of the physiological waveform graphical data; The weight coefficients of the first-order difference matrix are represented. This represents the element in the i-th row and j-th column of the first-order difference matrix of the physiological waveform graphical data; Represents the largest first-order difference matrix; The weight coefficients of the second-order difference matrix are represented. This represents the element in the i-th row and j-th column of the second-order difference matrix of the physiological waveform graphical data; Represents the largest second-order difference matrix; This represents the i-th preprocessed physiological waveform data point; This represents the polar angle corresponding to the i-th first preprocessed physiological waveform data point; This represents the polar angle corresponding to the j-th first preprocessed physiological waveform data point; It should be noted that the value range of the element in the i-th row and j-th column of the enhanced physiological waveform recursive graph data is [-1, 1], and the enhanced physiological waveform graphical data is finally mapped to a two-dimensional image; It should be noted that the first-order difference matrix of the physiological waveform graphical data is a row-direction difference, used to characterize the temporal rate of change. When i=1, ; It should be noted that the second-order difference matrix of the physiological waveform graphical data is a row-direction second-order difference, used to characterize the temporal change acceleration; when i=1 or i=2 ; It should be noted that the weighting coefficients of the physiological waveform graphical data range from [0.6, 0.8] to ensure that the basic temporal relationship is not weakened; the weighting coefficients of the first-order difference matrix range from [0.1, 0.2] to highlight the waveform rate of change characteristics; the weighting coefficients of the second-order difference matrix range from [0.1, 0.2] to highlight the waveform acceleration characteristics; the maximum first-order difference matrix, i.e., the maximum value of the first-order difference matrix, is used to normalize the difference features to the range of [0, 1]; the maximum second-order difference matrix, i.e., the maximum value of the second-order difference matrix, is used to normalize the difference features to the range of [0, 1]; constraints... This ensures that feature weights are normalized and avoids any one feature from becoming overly dominant. S223, The enhanced physiological waveform graphical data is mapped to obtain preprocessed physiological waveform data; It should be noted that the enhanced physiological waveform graphical data refers to the Gram matrix data after time-series topology enhancement being mapped to a two-dimensional image, i.e., the preprocessed physiological waveform data. It should be noted that, in this embodiment, The matrix is ​​normalized to the range [0, 255] and converted into a 224×224 grayscale image to obtain the preprocessed physiological waveform data; As can be seen, the physiological waveform data image processing method described in this invention first converts the one-dimensional waveform into a basic Gram matrix / recursive graph using the GAF / RP algorithm, and then fuses the first-order difference (rate of change) and second-order difference (acceleration) features of the waveform through a temporal topology enhancement mechanism, explicitly expressing the dynamic evolution law of the waveform. Based on the algorithm of polar coordinate system mapping or phase space reconstruction, combined with the temporal topology enhancement mechanism, the standardized waveform segments are converted into two-dimensional image representations, fully preserving the temporal correlation, dynamic changes and topological structure of the waveform. This effectively overcomes the problem of insufficient utilization of one-dimensional sequence modeling features in traditional technologies, and provides a standard input modality for subsequent cross-attention fusion with structured data.

[0044] Optionally, the second preprocessing of the structured data to obtain preprocessed structured data includes: S231, perform missing value processing on the structured data to obtain complete structured data; It should be noted that the missing value handling mentioned above means imputing with the mean, or imputing with the median (for continuous data), or imputing with the mode (for discrete data), or removing severely missing data samples to obtain the complete structured data and ensure data integrity; It should be noted that in this embodiment, the median of each indicator was used to fill in the missing values ​​in the blood routine indicators (a total of 3 cases of missing white blood cell count and 2 cases of missing hemoglobin). S232, Standardize the complete structured data to obtain standard structured data; It should be noted that the standardization process refers to the standardization of continuous, complete structured data, such as blood pressure values, test indicators, and rating scale scores, using Z-score. The standardized processing expression is: , in, This represents the mean of the complete structured data; This represents the standard deviation of the complete structured data; It should be noted that in this embodiment, heart rate, respiratory rate, body temperature, white blood cell count, hemoglobin, and APACHE II score were standardized using Z-score. S233, The standard structured data is encoded to obtain preprocessed structured data; It should be noted that the encoding process refers to encoding the discrete standard structured data using a one-hot encoding method, converting it into a binary vector that the model can recognize; It should be noted that in this embodiment, gender (male / female), underlying diseases (hypertension / diabetes / none), and medication information (sedative drugs / vasopressors / anti-infective drugs / none) are encoded using one-hot encoding and converted into binary vectors, ultimately obtaining 15-dimensional structured raw data, which is the preprocessed structured data.

[0045] Optionally, the processing of the preprocessed physiological waveform data and the preprocessed structured data to obtain injury warning result data includes: S31, feature extraction is performed on the preprocessed physiological waveform data and the preprocessed structured data to obtain physiological waveform image feature vector and structured data feature vector; It should be noted that the feature extraction refers to extracting the physiological waveform image feature vector Y from the two-dimensional waveform image through an image feature extraction network (such as CNN, ResNet, ViT); and extracting the structured data feature vector X from the structured data through a feature mapping network (such as a 3-layer fully connected network). It should be noted that in this embodiment, a pre-trained ResNet50 network is used as the image feature extraction network for the preprocessed physiological waveform data. The fully connected layers are removed, and the 224×224 waveform image is input into the network to output a 2048-dimensional image feature vector Y. It should be noted that in this embodiment, a 3-layer fully connected network (15-dimensional input, 64-dimensional and 32-dimensional hidden layers, and 32-dimensional output) is constructed for the preprocessed structured data, and feature mapping is performed on the processed structured data to obtain a 32-dimensional structured feature vector X. S32, Process the feature vector of the physiological waveform image and the feature vector of the structured data to obtain multi-source data fusion feature data; S33, Based on the injury warning model, the multi-source data fusion feature data is processed to obtain injury warning result data.

[0046] Optionally, the processing of the physiological waveform image feature vector and the structured data feature vector to obtain multi-source data fusion feature data includes: S321, Based on the heterogeneous feature association weight acquisition model, the feature vector of the physiological waveform image and the feature vector of the structured data are processed to obtain heterogeneous feature association weight data. The expression for the heterogeneous feature association weight acquisition model is: ， ， ， , , , in, This represents the weight data associated with the heterogeneous features; This represents the Softmax activation function; This represents the dimension of the feature vector of the physiological waveform image; This represents the query vector matrix composed of the structured data feature vectors; The key vector matrix represents the feature vectors of the physiological waveform image. Represents the feature vector of a physiological waveform image; Represents the feature vector of structured data; This represents the transpose of the structured feature vector X; Represents the trace operation of a matrix; This represents the transpose of the feature vector of the physiological waveform image; The covariance matrix represents the feature vector of the physiological waveform image and the feature vector of the structured data; This represents the value of the p-th dimension of the structured feature; This represents the mean of the p-th dimension of the structured feature vector; This represents the value of the q-th dimension of the physiological waveform image feature; This represents the mean of the q-th dimension of the feature vector of the physiological waveform image; This represents the expectation operation; The variance matrix represents the feature vector of the structured data; The variance matrix represents the feature vector of the physiological waveform image; Indicates the dynamic adjustment coefficient; This represents a diagonal matrix composed of the importance weights of the feature vectors of the structured data; It should be noted that the heterogeneous feature association weight data, i.e., the heterogeneous feature association weight matrix, has a dimension of [missing information]. , used to characterize the dynamic correlation strength between structured features and image features; where, Let X be the dimension of the structured data feature vector. Let Y be the dimension of the feature vector Y of the physiological waveform image; It should be noted that the Softmax activation function normalizes each row of the heterogeneous feature association weight matrix so that the sum of the elements in each row of the weight matrix is ​​1, thus ensuring the probabilistic meaning of the weights. It should be noted that the trace operation of the matrix, i.e. the sum of the elements on the main diagonal of the matrix, is used to quantify the overall energy of the matrix. It should be noted that the covariance matrix of the physiological waveform image feature vector and the structured data feature vector is used to characterize the cross-modal correlation of the two types of features. It should be noted that the variance matrix of the structured data feature vectors , dimension This is used to characterize the degree of dispersion of each dimension of the structured features; the variance matrix of the physiological waveform image feature vector , dimension , used to characterize the degree of dispersion of each dimension of image features; the dynamic adjustment coefficient The value range is [0.2, 0.5], used to balance cross-modal correlation and the importance of the structured features themselves; the diagonal matrix formed by the importance weights of the structured data feature vectors. , dimension ,in The importance weights for the p-th dimension of the structured features are obtained through offline training using models such as Random Forest and XGBoost, with the weight of the underlying disease dimension being higher than that of the gender dimension. All non-main diagonal elements are 0. S322, Based on the feature enhancement processing model, the feature vector of the physiological waveform image is weighted to obtain image feature enhancement data; The expression for the feature enhancement processing model is: , in, This represents the image feature enhancement data; This represents the feature vector of the physiological waveform image; S323, The image feature enhancement data and the structured data feature vector are concatenated to obtain multi-source data fusion feature data; The concatenation processing expression is: , in, This represents the multi-source data fusion feature data; It should be noted that in this embodiment, splicing X and We obtain a fusion feature F with dimensions of 32+32=64.

[0047] Optionally, the processing of the physiological waveform image feature vector and the structured data feature vector to obtain multi-source data fusion feature data includes: S3211, High-dimensional feature map data of the feature vector of the physiological waveform image is extracted using a deep residual network; The high-dimensional feature map data expression is: , in, This represents the number of high-dimensional feature maps; This represents the high-dimensional semantic information of the feature vector of the physiological waveform image; The high-dimensional semantic information is represented by the real number field; H and W represent the height and width of the physiological waveform image features in the spatial dimension, respectively; C represents the number of feature channels, corresponding to the dimension of the deep physiological rhythm features extracted by the deep residual network. It should be noted that in this embodiment, the deep residual network is a ResNet network, and the output high-dimensional feature map has a dimension of 7×7×2048; S3212, The structured data feature vector is mapped using the linear transformation layer of the deep residual network to obtain structured representation vector data; The structured representation vector data expression is as follows: , in, This represents the structured representation vector data; This represents the set of results obtained from mapping the structured data feature vectors; The real number field represents the result set, and 1×C indicates that the result set is a C-dimensional dense vector; S3213, Process the high-dimensional feature map data and the structured representation vector data to obtain the feature query matrix Q, the feature key matrix K and the eigenvalue matrix V; It should be noted that the feature query matrix Q is generated from the structured data, and the structured representation vector... With query matrix weight matrix Multiplication; Q represents "what we are looking for"; for example, since the patient is elderly and has a history of myocardial infarction (structured information), the Q vector contains the search instruction "focus on ST segment evolution in ECG waveform"; The feature key matrix K is generated from the physiological waveform data, and the high-dimensional feature map Fimg is combined with the key matrix weight matrix. Multiplication; K represents "what features each region in the waveform image possesses"; the feature key matrix K is used to match with the feature query matrix Q to calculate which time points or morphological features in the waveform are most relevant to the patient's individual background; The eigenvalue matrix V is generated from the physiological waveform data, and the high-dimensional feature map Fimg is combined with the eigenvalue matrix weight matrix. Multiplication; V represents "the actual physiological signal information contained in the waveform image"; when the feature query matrix Q and the feature key matrix K are successfully matched, the deep residual network extracts key early warning information from the feature value matrix V according to the calculated weights; It should be noted that the query matrix weight matrix The key matrix and weight matrix and the value matrix and weight matrix The deep residual network is trained and learned to obtain the result. S3214, Using a feature dynamic weighting model, the feature query matrix, the feature key matrix, and the feature value matrix are processed to obtain multi-source data fusion feature data; The expression for the feature dynamic weighting model is: , in, Indicates the scaling factor; This represents the fused feature vector; It should be noted that the feature vectors of the physiological waveform image and the structured data feature vector are processed to obtain multi-source data fusion feature data. The fusion feature vector obtained by the feature dynamic weighting model is used to adaptively reweight the different receptive field regions in the waveform image according to the patient's individual background. Based on the patient's individual background (such as advanced age, history of myocardial infarction), key abnormal signs are targeted from the real-time monitoring waveform. The deep residual network is used to achieve a deep representation of the waveform rhythm morphology while avoiding gradient vanishing in deep networks.

[0048] Optionally, the process of processing the multi-source data fusion feature data based on the injury warning model to obtain injury warning result data includes: S331, Constructing an injury early warning model based on fused features; It should be noted that in this embodiment, the injury early warning model is constructed using the random forest algorithm, with 100 decision trees, a maximum depth of 10, and a minimum number of sample splits of 5. The model is trained using the multi-source data fusion feature data as input. S332, Based on the injury warning model, perform multi-window risk aggregation processing on the multi-source data fusion feature data to obtain aggregated injury warning result data; The expression for multi-window risk aggregation processing is: , , , in, The aggregated injury warning result data represents the final aggregated injury risk probability, with a value range of [0,1], and is the core indicator of the warning result. This represents the number of learning windows participating in the aggregation, with a value range of [3,8]. Too many windows can easily introduce redundant historical information, while too few windows cannot reflect the trend. Indicates the first The weighting coefficients of each learning window; Indicates the first The length of each learning window; Indicates the first The initial risk probability for each learning window is directly output by the early warning model; This represents the time decay factor, with a value range of [0.3, 0.7], used to reduce the weight of the historical window. The larger the value, the faster the historical window weight decays; Indicates the first The end time of each learning window; Indicates the current warning time; Indicates the first The time difference between the end time of each window and the current warning time, in seconds; This represents the baseline length of a single learning window, used to normalize the time difference and avoid the influence of the time unit on the decay factor. S333, Based on the injury risk level, the aggregated injury warning result data is processed to obtain injury warning result data; It should be noted that the aforementioned risk levels include low risk: Medium risk; High risk: ; It should be noted that a high-risk condition triggers an early warning signal immediately, while a medium-risk condition triggers an early warning signal after three consecutive windows, thus obtaining injury warning data. As can be seen, the injury warning method based on multimodal data fusion described in the embodiments of the present invention solves the problems of insufficient utilization of waveform data temporal features, difficulty in effectively fusing waveform data and structured data, and insufficient accuracy and stability of warnings in existing injury warning technologies. It provides an injury warning method and device based on multimodal data fusion, which converts one-dimensional continuous physiological waveform data into a two-dimensional image or matrix representation containing temporal correlation and topological structure, thereby making implicit temporal dependencies explicit, which is convenient for processing using deep feature extraction algorithms in the field of computer vision. At the same time, through a multimodal deep interactive network, the image-based dynamic waveform features are semantically aligned and fused with the patient's structured features to achieve accurate assessment of physiological risk. This invention utilizes time-series topology enhancement processing to integrate the fundamental temporal relationships, rates of change, and acceleration features of the original waveform. This overcomes the shortcomings of traditional image transformations that only retain static temporal relationships, enabling two-dimensional images to more fully reflect the dynamic evolution of waveforms and enhancing the richness and stability of feature representation. Through a heterogeneous feature association weight matrix formula, the correlation strength between structured features and image features is dynamically calculated. Combining the importance weights of structured features with cross-modal correlation, semantic-level alignment of heterogeneous data is achieved, avoiding the dilution of modal information caused by simple splicing and fully leveraging the complementary advantages of the two types of data. The window risk aggregation formula integrates the warning results of multiple consecutive windows, and combines window length weighting and time decay mechanisms to reduce the impact of single window noise on the warning results, thereby improving the model's ability to identify slowly evolving injuries and the stability of the warning. It constructs a complete technical process from data acquisition, preprocessing, enhanced imaging, deep fusion to multi-window aggregation warning. All formulas have clear physical meaning and adjustable parameters, and can be adaptively adjusted according to different clinical scenarios (emergency care, intensive care, postoperative monitoring), reducing reliance on human experience and improving the timeliness and consistency of warnings, thus having broad clinical application value.

[0049] Example 2 Please see Figure 3 , Figure 3 This is a diagram illustrating the composition of a multimodal data fusion-based injury early warning device disclosed in an embodiment of the present invention. Figure 3 The described injury early warning device based on multimodal data fusion is applied to intensive care systems, such as local servers or cloud servers used in intensive care systems; however, this invention does not limit its application. Figure 3 As shown, the injury early warning device based on multimodal data fusion includes: Data acquisition module 101, data preprocessing module 102, and data processing module 103; The data acquisition module 101 is used to acquire physiological waveform data and structured data; The data preprocessing module 102 is used to preprocess the physiological waveform data and the structured data to obtain preprocessed physiological waveform data and preprocessed structured data. The data processing module 103 is used to process the preprocessed physiological waveform data and the preprocessed structured data to obtain injury warning result data; The data acquisition module 101, the data preprocessing module 102, and the data processing module 103 are connected in sequence.

[0050] Example 3 Please see Figure 4 , Figure 4 This is a schematic diagram of another injury early warning device based on multimodal data fusion disclosed in an embodiment of the present invention. Figure 4 The described device can be applied to intensive care systems, such as local servers or cloud servers used in intensive care systems, and the embodiments of the present invention are not limited thereto. Figure 4 As shown, the device may include: Memory 201 storing executable program code; Processor 202 coupled to memory 201; The processor 202 calls the executable program code stored in the memory 201 to execute the steps in the injury early warning method based on multimodal data fusion described in Embodiment 1.

[0051] Example 4 This invention discloses a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform the steps in the injury early warning method based on multimodal data fusion described in Embodiment 1.

[0052] Example 5 This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the injury early warning method based on multimodal data fusion described in Embodiment 1.

[0053] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0054] It should be noted that all calculation expressions or mathematical functions in the embodiments of the present invention have undergone dimensionless processing of the variables involved before calculation.

[0055] It should be noted that in all the calculation expressions or mathematical functions in the embodiments of the present invention, the values ​​of the input independent variables all meet the reasonable requirements of the input value range of the calculation expression or mathematical function, and can ensure that the calculation expression or mathematical function can be calculated smoothly without violating physical laws or mathematical rules.

[0056] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), once programmable read-only memory (OTPROM), electronically erasable rewritable read-only memory (EEPROM), read-only optical disc (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0057] Finally, it should be noted that the injury early warning method and device based on multimodal data fusion disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for injury early warning based on multimodal data fusion, characterized in that, The method includes: S1, acquire physiological waveform data and structured data; S2, preprocess the physiological waveform data and the structured data to obtain preprocessed physiological waveform data and preprocessed structured data; S3, process the preprocessed physiological waveform data and the preprocessed structured data to obtain injury warning result data.

2. The injury early warning method based on multimodal data fusion according to claim 1, characterized in that, The preprocessing of the physiological waveform data and the structured data to obtain preprocessed physiological waveform data and preprocessed structured data includes: S21, Perform a first preprocessing on the physiological waveform data to obtain first preprocessed physiological waveform data; S22, the first preprocessed physiological waveform data is processed into an image to obtain preprocessed physiological waveform data; S23, perform a second preprocessing on the structured data to obtain preprocessed structured data.

3. The injury early warning method based on multimodal data fusion according to claim 2, characterized in that, The first preprocessing of the physiological waveform data to obtain first preprocessed physiological waveform data includes: S211, Denoise the physiological waveform data to obtain denoised physiological waveform data; S212, normalize the denoised physiological waveform data to obtain normalized denoised physiological waveform data; The normalization expression is: ， in, This represents the normalized and denoised physiological waveform data; This represents the denoised physiological waveform data; This represents the smallest denoised physiological waveform data in the current waveform segment; This represents the largest denoised physiological waveform data in the current waveform segment; Represents the coefficients of a small constant; S213, the normalized and denoised physiological waveform data is divided into time windows to obtain the first preprocessed physiological waveform data.

4. The injury early warning method based on multimodal data fusion according to claim 2, characterized in that, The step of performing image processing on the first preprocessed physiological waveform data to obtain preprocessed physiological waveform data includes: S221, the first preprocessed physiological waveform data is converted to obtain physiological waveform graphical data; S222, Perform temporal topology enhancement processing on the physiological waveform graphical data to obtain enhanced physiological waveform graphical data; The expression for the time-series topology enhancement processing is: ， ， ， ， ， ， in, This represents the element in the i-th row and j-th column of the enhanced physiological waveform graphical data; Represents the weighting coefficients of physiological waveform graphical data; This represents the element in the i-th row and j-th column of the physiological waveform graphical data; The weight coefficients of the first-order difference matrix are represented. This represents the element in the i-th row and j-th column of the first-order difference matrix of the physiological waveform graphical data; Represents the largest first-order difference matrix; The weight coefficients of the second-order difference matrix are represented. This represents the element in the i-th row and j-th column of the second-order difference matrix of the physiological waveform graphical data; Represents the largest second-order difference matrix; This represents the i-th preprocessed physiological waveform data point; This represents the polar angle corresponding to the i-th first preprocessed physiological waveform data point; This represents the polar angle corresponding to the j-th first preprocessed physiological waveform data point; S223, The enhanced physiological waveform graphical data is mapped to obtain preprocessed physiological waveform data.

5. The injury early warning method based on multimodal data fusion according to claim 2, characterized in that, The second preprocessing of the structured data to obtain preprocessed structured data includes: S231, perform missing value processing on the structured data to obtain complete structured data; S232, Standardize the complete structured data to obtain standard structured data; S233, the standard structured data is encoded to obtain preprocessed structured data.

6. The injury early warning method based on multimodal data fusion according to claim 1, characterized in that, The process of processing the preprocessed physiological waveform data and the preprocessed structured data to obtain injury warning result data includes: S31, feature extraction is performed on the preprocessed physiological waveform data and the preprocessed structured data to obtain physiological waveform image feature vector and structured data feature vector; S32, Process the feature vector of the physiological waveform image and the feature vector of the structured data to obtain multi-source data fusion feature data; S33, Based on the injury warning model, the multi-source data fusion feature data is processed to obtain injury warning result data.

7. The injury early warning method based on multimodal data fusion according to claim 6, characterized in that, The process of processing the feature vector of the physiological waveform image and the feature vector of the structured data to obtain multi-source data fusion feature data includes: S321, Based on the heterogeneous feature association weight acquisition model, the feature vector of the physiological waveform image and the feature vector of the structured data are processed to obtain heterogeneous feature association weight data. The expression for the heterogeneous feature association weight acquisition model is: ， ， ， ， ， , in, This represents the weight data associated with the heterogeneous features; This represents the Softmax activation function; This represents the dimension of the feature vector of the physiological waveform image; This represents the query vector matrix composed of the structured data feature vectors; The key vector matrix represents the feature vectors of the physiological waveform image. Represents the feature vector of a physiological waveform image; Represents the feature vector of structured data; This represents the transpose of the structured feature vector X; Represents the trace operation of a matrix; This represents the transpose of the feature vector of the physiological waveform image; The covariance matrix represents the feature vector of the physiological waveform image and the feature vector of the structured data; This represents the value of the p-th dimension of the structured feature; This represents the mean of the p-th dimension of the structured feature vector; This represents the value of the q-th dimension of the physiological waveform image feature; This represents the mean of the q-th dimension of the feature vector of the physiological waveform image; This represents the expectation operation; The variance matrix represents the feature vector of the structured data; The variance matrix represents the feature vector of the physiological waveform image; Indicates the dynamic adjustment coefficient; This represents a diagonal matrix composed of the importance weights of the feature vectors of the structured data; S322, Based on the feature enhancement processing model, the feature vector of the physiological waveform image is weighted to obtain image feature enhancement data; The expression for the feature enhancement processing model is: ， in, This represents the image feature enhancement data; S323, The image feature enhancement data and the structured data feature vector are concatenated to obtain multi-source data fusion feature data; The concatenation processing expression is: ， in, This refers to the multi-source data fusion feature data.

8. A wound warning device based on multimodal data fusion, characterized in that, The device includes: a data acquisition module, a data preprocessing module, and a data processing module; The data acquisition module is used to acquire physiological waveform data and structured data; The data preprocessing module is used to preprocess the physiological waveform data and the structured data to obtain preprocessed physiological waveform data and preprocessed structured data. The data processing module is used to process the preprocessed physiological waveform data and the preprocessed structured data to obtain injury warning result data; The data acquisition module, the data preprocessing module, and the data processing module are connected in sequence.

9. A wound warning device based on multimodal data fusion, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the injury early warning method based on multimodal data fusion as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, which, when invoked, are used to execute the injury warning method based on multimodal data fusion as described in any one of claims 1-7.

Images