An intelligent warning system and method for accompanying based on multi-modal data fusion

Abstract

The application discloses a kind of based on multi-modal data fusion's intelligent early warning system and method of accompanying, belong to wisdom medical treatment and artificial intelligence technical field, this system includes multi-modal data acquisition and pre-processing module, dynamic weight fusion module, intelligent risk assessment module and multidisciplinary collaborative intervention module, through hardware synchronization and software interpolation, the time-space alignment of multi-source sensor data is realized, the dynamic weight fusion mechanism combining multi-head self-attention and evidence theory is used to process modal conflict, risk dynamic assessment is carried out using sliding window exponential weighting and LSTM time series modeling, and based on medical knowledge graph and improved hungarian algorithm, the intelligent scheduling and closed-loop feedback optimization of nursing resource are realized.

Description

Technical Field

[0001] This invention belongs to the field of smart healthcare and artificial intelligence technology, specifically relating to a smart early warning system and method for caregivers based on multimodal data fusion, which is applicable to health risk monitoring and collaborative intervention in elderly care scenarios. Background Technology

[0002] With the continued acceleration of global population aging, health risk management for elderly patients has become one of the core challenges urgently needing to be addressed in the medical and nursing field. In elderly care settings, high-risk events such as falls, pressure sores, and infections are characterized by their suddenness and severe consequences. Traditional nursing models primarily rely on regular rounds and subjective judgment by caregivers to identify potential risks. This one-on-one monitoring approach is not only limited by insufficient human resources but also suffers from inherent flaws such as delayed risk identification and a single assessment dimension, making it difficult to meet the actual needs of refined and continuous health risk management. Statistics show that the overall incidence of high-risk events among elderly patients can reach over 30%. Once a risk event occurs, it not only directly threatens the patient's life and quality of life but also significantly increases medical costs and the burden on families. Therefore, building an intelligent health risk early warning system has significant practical and social value.

[0003] In existing technologies, single-modal monitoring systems are the most common approach. These systems typically collect indicators such as blood pressure and heart rate using physiological sensors or monitor behavior using cameras, then trigger alarms based on preset fixed thresholds. However, a single data source cannot comprehensively reflect the complex and ever-changing health conditions of elderly patients, and its threshold settings often lack personalized adaptability. When faced with complex risk scenarios involving multiple intertwined factors, the accuracy and timeliness of warnings are significantly limited. For example, a system based solely on heart rate monitoring may fail to distinguish between an increase in heart rate due to normal activity and a true sign of arrhythmia, while a video-based behavior analysis system struggles to capture information about changes in the patient's internal physiological state. These inherent limitations severely restrict the practical application of single-modal systems. Furthermore, while some IoT-integrated robotic nursing systems can achieve a certain degree of automated data collection and simple nursing operations, their decision-making logic still relies on preset rules, lacking a deep understanding of unstructured data and failing to achieve collaborative analysis and intelligent reasoning across modal data.

[0004] In recent years, multimodal data fusion technology has made significant progress in the field of smart healthcare, but its application in elderly care early warning scenarios still faces multiple technical bottlenecks. First, data from different modalities exhibit inherent heterogeneity in sampling frequency, data dimensionality, and spatiotemporal resolution. Simple data splicing or static weighted fusion methods can lead to the loss of key information or undue amplification of noise signals. Second, when monitoring signals from multiple modalities contradict each other, existing fusion methods lack effective conflict detection and dynamic weight adjustment mechanisms, easily causing the fusion results to deviate from the true state. Third, elderly care scenarios involve various environments such as hospitals, communities, and homes, with significant differences in data distribution characteristics and risk types across different scenarios. Model parameters obtained in a single training environment are difficult to directly transfer to new scenarios while maintaining good predictive performance. Finally, from risk warning to actual intervention execution, existing systems generally lack the ability to integrate medical staff, patients' families, and caregivers into a unified collaborative framework, making it difficult to quickly transform warning information into effective intervention actions, creating a disconnect between identification and treatment.

[0005] Therefore, how to build an intelligent early warning system that can achieve accurate spatiotemporal alignment and deep fusion of multi-source heterogeneous data, has the ability to adaptively handle modal conflicts, supports cross-scenario dynamic transfer learning, and can closely link risk assessment results with multidisciplinary collaborative intervention has become a key issue that urgently needs to be addressed in the field of elderly care technology. Summary of the Invention

[0006] To address the technical problems of existing elderly care early warning systems, such as insufficient data fusion dimensions, weak modal conflict handling capabilities, poor cross-scenario generalization, and disconnect between early warning and intervention, this invention provides a caregiver intelligent early warning system based on multimodal data fusion. The system includes a multimodal data acquisition and preprocessing module, a dynamic weight fusion module, an intelligent risk assessment module, and a multidisciplinary collaborative intervention module. These four modules form a progressive closed-loop collaborative architecture.

[0007] The multimodal data acquisition and preprocessing module synchronously drives multiple sensors to acquire data in parallel using a unified clock source. It performs linear interpolation time alignment processing based on a sliding window for data with different sampling frequencies and maps the spatial coordinates of each sensor to a global coordinate system using an extrinsic parameter matrix. A lightweight convolutional neural network is deployed on edge devices to extract video pose features in real time, generating a spatiotemporally consistent set of multimodal feature vectors. The dynamic weight fusion module models cross-modal semantic associations of multimodal features using a multi-head self-attention mechanism. It detects modal conflicts and dynamically corrects weights based on evidence theory and employs an elastic weight consolidation algorithm to achieve online adaptive updates of the fusion network. The intelligent risk assessment module performs risk level and risk type classification on the fused features. It dynamically updates risk scores using a sliding window exponential weighted average mechanism and captures risk evolution trends using a long short-term memory network. The multidisciplinary collaborative intervention module maps risk assessment results to a set of intervention measures based on a medical knowledge graph. It achieves intelligent scheduling of nursing tasks by introducing an improved Hungarian algorithm with a load balancing adjustment term and collects intervention feedback data to optimize the risk assessment model through a comparative learning mechanism. In the closed-loop collaborative architecture consisting of the above four modules, the output of the previous module directly serves as the input of the next module, while the execution feedback of the multidisciplinary collaborative intervention module can drive the parameter optimization of the intelligent risk assessment module in turn, achieving a nonlinear collaborative gain effect where 1+1 is much greater than 2.

[0008] This invention also provides a smart early warning method for caregivers based on multimodal data fusion, which is applied to the above-mentioned system. The method includes step S1 multimodal data acquisition and preprocessing, step S2 dynamic weight fusion, step S3 intelligent risk assessment, and step S4 multidisciplinary collaborative intervention. Each step corresponds one-to-one with the function of the corresponding module in the system, forming a complete closed-loop processing flow from data acquisition to fusion analysis, risk assessment and collaborative intervention.

[0009] The beneficial effects of this invention are as follows: By combining hardware synchronization and software interpolation in a spatiotemporal alignment method, the temporal deviation of multimodal data is controlled within 5ms and the spatial error within 5cm, solving the problem of spatiotemporal inconsistency in multi-source heterogeneous data; a dual fusion mechanism of attention weights and evidence discounting effectively handles modal conflicts, enabling the system to maintain reliable early warning output even when sensor data is abnormal; elastic weight consolidation ensures continuous optimization of model parameters without forgetting learned knowledge, giving the system rapid adaptability across scenarios; a knowledge graph-driven intelligent scheduling algorithm improves the efficiency of nursing resource allocation, achieving precise matching of skills and task requirements; and a contrastive learning-driven feedback loop continuously improves the system's early warning performance, causing the system's accuracy to continuously increase with the extension of application time. Attached Figure Description

[0010] Figure 1This is a schematic diagram of the architecture of the intelligent early warning system for caregivers based on multimodal data fusion provided in an embodiment of the present invention.

[0011] Figure 2 This is a flowchart illustrating the intelligent early warning method for caregivers based on multimodal data fusion provided in an embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the following embodiments are only for explaining the invention and are not intended to limit the scope of protection of the invention.

[0013] Reference Figure 1 The intelligent early warning system for caregivers based on multimodal data fusion provided in this invention includes a multimodal data acquisition and preprocessing module 1, a dynamic weight fusion module 2, an intelligent risk assessment module 3, and a multidisciplinary collaborative intervention module 4. These four modules form a progressive closed-loop collaborative architecture of data acquisition → feature fusion → risk assessment → collaborative intervention → feedback optimization. The output of each module constitutes the key input of the next, and the intervention feedback data collected by the multidisciplinary collaborative intervention module 4 can in turn drive the parameter optimization of the intelligent risk assessment module 3, thereby achieving continuous improvement in the overall system performance. The following provides a detailed description of each module.

[0014] The multimodal data acquisition and preprocessing module is the data foundation layer of this system. Its core function is to acquire multi-source heterogeneous sensor data from nursing scenarios and transform it into high-quality feature vectors that are consistent in time and space, providing reliable input for downstream fusion analysis. In one embodiment of the present invention, the workflow of this module can be divided into three stages: multi-sensor hardware synchronous acquisition, timestamp alignment and feature extraction, and spatial coordinate system unification and mapping.

[0015] In the multi-sensor hardware synchronous acquisition stage, the system deploys a PPS pulse generator as a unified clock source to synchronously drive the internal clock signals of the physiological sensors, visual sensors, and environmental sensors. Preferably, the PPS pulse generator outputs one high-precision synchronization pulse per second. Each sensor receives this pulse and calibrates its local clock via hardware interrupt, thereby eliminating clock drift errors between different devices at the hardware level. In this embodiment, the specific configuration of the sensor array is as follows: The physiological sensors include a wearable heart rate monitor, a blood pressure monitor, and a three-axis accelerometer and a three-axis gyroscope integrated into the wristband. Their sampling frequency is uniformly set to 1Hz, used to continuously acquire physiological parameters such as heart rate, blood pressure, and body temperature of elderly patients, as well as inertial data such as motion acceleration and angular velocity; the visual sensors use an RGB-D depth camera, deployed in the main activity areas of the ward or home environment, with a sampling frequency set to 30Hz, used to acquire the patient's spatial location information and human posture characteristics in real time; the environmental sensors include a temperature and humidity sensor and a smoke detector, installed at key risk points in the nursing environment, with a sampling frequency set to 0.5Hz, used to continuously monitor parameters such as temperature, humidity, and air quality in the nursing environment. The three types of sensors mentioned above operate in parallel under the unified clock drive of the PPS pulse generator, ensuring that all acquisition channels maintain strict synchronization on the time base.

[0016] In the timestamp alignment and feature extraction stages, due to the significant differences in sampling frequencies among the three types of sensors, software interpolation algorithms are needed to align data with different sampling frequencies to a unified time reference. This invention employs a sliding window-based linear interpolation method to achieve this goal. Preferably, the sliding window size is set to 200ms. Within this window range, linear interpolation calculations are performed on non-uniformly sampled data points, thereby resampling all modal data to a unified time resolution. After time alignment, the standard deviation of the time deviation between different modal data is controlled within 5ms, effectively meeting the stringent time consistency requirements of subsequent cross-modal fusion analysis. Regarding feature extraction, to meet the real-time processing needs of video data, the system deploys a lightweight MobileNetV3 convolutional neural network on an edge computing device. This network employs a depthwise separable convolution and inverse residual structure design, significantly reducing computational complexity while maintaining high feature extraction accuracy. MobileNetV3 extracts a 128-dimensional pose feature vector for each frame of video image, covering information such as the spatial coordinates of key human joints, joint angles, and movement speed. Preferably, the system further combines ROI cropping and background subtraction algorithms to process only the video content of the patient's activity area, thereby reducing the amount of video data transmitted by more than 85%, effectively reducing network bandwidth pressure and cloud computing load. For physiological time-series data, the system uses the Z-score normalization method to eliminate the dimensional differences between different physiological parameters. For missing data caused by sensor failure or communication interruption, a sliding window mean imputation strategy with a window size of 10 seconds is used to ensure the integrity of physiological feature vectors. After the above processing, the physiological feature vector generated in each time window has a dimension of 64, the behavioral feature vector has a dimension of 128, and the environmental feature vector has a dimension of 32.

[0017] In the spatial coordinate system mapping stage, the system constructs a global coordinate system for the nursing environment using multi-camera calibration parameters. Preferably, the Zhang Zhengyou calibration method is used to obtain the intrinsic parameter matrix and distortion coefficients of each RGB-D camera, and then the extrinsic parameter matrix between each camera, including the rotation matrix and translation vector, is determined using a checkerboard calibration board. Utilizing a triangular geometric positioning model, the system can map the local motion data acquired by the accelerometer and gyroscope in the wearable device to the global space defined by the camera coordinate system. In one embodiment of the invention, a Kalman filter is also introduced to smooth the spatial coordinate mapping results to suppress positioning jitter caused by sensor noise. After spatial alignment processing, the spatial error of each sensor data in the global coordinate system is controlled within 5cm. The aforementioned temporal and spatial alignment processes together ensure the consistency of multimodal feature vectors in the spatiotemporal dimensions, providing high-quality input data for the dynamic weight fusion module 2. It should be particularly noted that the design of the multimodal data acquisition and preprocessing module 1 in this invention embodies the layered computing concept of edge-cloud collaboration. Feature extraction from video data is performed on edge devices, transmitting only 128-dimensional feature vectors instead of the original video stream to the cloud. This design not only reduces data transmission bandwidth requirements by more than 85% but also effectively avoids the risk of privacy leaks during the transmission of the original video data. Physiological and environmental data, due to their smaller size, can be directly transmitted to the cloud for processing via an encrypted channel. This differentiated computational allocation strategy enables the system to operate stably in nursing environments with only basic network conditions, significantly lowering the deployment threshold. Furthermore, the multimodal feature vector set output by this module is encapsulated using a unified data structure. The output of each time window includes a 224-dimensional (64+128+32) concatenated vector along with a corresponding global timestamp and spatial coordinate labels. This standardized data interface design ensures seamless integration with the downstream dynamic weight fusion module 2.

[0018] The dynamic weighted fusion module 2 is the core processing layer of this system. Its function is to deeply fuse the heterogeneous feature vectors from the multimodal data acquisition and preprocessing module 1 to generate high-dimensional fusion features that can comprehensively represent the patient's health status. One of the key innovations of this invention is the organic combination of attention-based semantic association modeling and evidence-based uncertainty-perceived confidence weighting. This allows it to capture deep semantic relationships between multimodal data and effectively address complex situations of data conflicts between modalities.

[0019] In the multi-head self-attention weight calculation stage, the system concatenates the preprocessed multimodal feature vectors into a unified input tensor according to modality type. ,in Physiological feature vectors, For behavioral feature vectors, For environmental feature vectors, , , These represent the dimensions of each modality feature. This input tensor is fed into a multi-head self-attention module, where query matrices are generated through linear projection layers. Key matrix Sum matrix The specific calculation process is as follows: , in, , and It is a learnable linear projection weight matrix; The total dimension of the input features; The dimension of the key vector is set to 64 in this embodiment. Subsequently, the system calculates the scaled dot product attention score: , in, This is a scaling factor used to prevent when... Larger times result in excessively large values ​​of the dot product cause the gradient of the softmax function to vanish. Applying the softmax normalization function to the attention score yields the attention weight matrix: , in, Each element in Indicates the first The modal feature for the first Attention weights for each modal feature, with values ​​ranging from [value range missing]. Furthermore, the sum of all weights in the same row is 1. The final attention-weighted fusion feature is: , In one embodiment of the present invention, the multi-head attention module employs four attention heads for parallel computation. Each head independently learns the intermodal association patterns in a different subspace. Finally, the outputs of the four heads are concatenated and subjected to a linear transformation to obtain a fused feature vector with a dimension of 256. This multi-head parallel mechanism enables the system to capture complementary information between modalities simultaneously from multiple semantic levels, and compared to the single-head attention mechanism, it can more comprehensively model cross-modal semantic associations.

[0020] In the uncertainty perception confidence weighting stage, the system quantifies the uncertainty of predictions for each modality based on Dempster-Shafer evidence theory. Preferably, for each modality... ( (Corresponding to the physiological, behavioral, and environmental modalities respectively), the system generates evidence for each modality through a linear evidence network with a three-layer fully connected structure. Each layer contains 32 neurons. Evidence body Representing modes The distribution of support for each possible hypothesis (i.e., each risk category). This invention calculates the degree of conflict between different modal evidence bodies. To measure the degree of inconsistency between modal data: , in, For a non-empty subset of the hypothesis space; For modality The body of evidence against the hypothesis The reliability score is assigned, and the range of values ​​is: ; The range of values ​​is ,when When all modal predictions are completely consistent, A value close to 1 indicates severe intermodal conflict. This invention sets the conflict threshold at 0.3. When the system determines that there is a significant conflict between modalities, it is necessary to adjust the fusion weights of each modality through an evidence discounting mechanism. The formula for calculating the evidence discounting factor is: , in, To be applied to the mode The discount factor, with a value range of 100%. The corrected fusion weights are: , in, The original weights output by the multi-head self-attention mechanism; This represents the final weight after evidence discounting. The threshold of 0.3 is selected based on statistical analysis in elderly care scenarios—under normal monitoring conditions, the conflict degree between the three modalities is usually maintained within the range of 0.1 to 0.2. When the conflict degree exceeds 0.3, it often means that the sensor of a certain modality has experienced data anomalies or transmission errors. In this case, reducing the weight of the abnormal modality can effectively avoid interference from erroneous data on the fusion results. Preferably, the system also introduces a confidence weighting mechanism based on the prediction entropy value as a supplement, the calculation formula of which is: , in, For modality The information entropy of the predicted distribution, measured in bits; The temperature coefficient controls the sensitivity of the confidence weight; in this embodiment, it is set to 1.0. The confidence weights are based on prediction entropy, and their values ​​range from [value range missing]. The more concentrated the prediction distribution (i.e., the lower the information entropy), the closer this weight is to 1. This confidence weight is related to the weight after evidence discount. Further multiplication yields the final integrated weights, which are used to generate the dynamically weighted fusion feature vector. .

[0021] In the adaptive model update strategy phase, the system employs two triggering mechanisms to maintain the adaptability of the fusion model in dynamic environments. The first is a periodic update mechanism: every 24 hours, the system freezes the parameters of the underlying feature extraction network (i.e., MobileNetV3 and the physiological data normalization layer), and only fine-tunes the parameters of the multi-head self-attention module and the evidence network, using the Elastic Weight Consolidation (EWC) algorithm, whose loss function is defined as: , in, The standard loss function for the current task; This is the regularization coefficient, which is set to 0.5 in this embodiment; For the Fisher information matrix The diagonal elements reflect the parameters. The importance of previously learned tasks; The current parameter value; The parameter values ​​are those from the previous update. The EWC algorithm constrains the update magnitude of important parameters through the Fisher information matrix, ensuring that the model does not forget previously learned knowledge from other scenarios while learning new scenario data features. This characteristic is particularly crucial for systems that need to be deployed across multiple nursing scenarios such as hospitals, communities, and homes. The second type is an event-driven update mechanism: when the system detects a significant shift in the input data distribution, it automatically triggers the online learning process. The specific judgment condition is: when the KL divergence of the predicted distribution between two adjacent time windows exceeds 0.15, or when modal conflict detection is triggered in three consecutive time windows (i.e., When the system considers that the current environment has undergone substantial changes, it uses the most recently collected incremental samples to quickly fine-tune the fusion network. During the fine-tuning process, the regularization threshold of KL divergence is set to 0.003 to avoid excessive shift of model parameters.

[0022] The intelligent risk assessment module 3 receives the fused feature vector output by the dynamic weight fusion module 2. The system performs risk level determination, risk type identification, and dynamic risk score calculation, providing a quantitative risk assessment basis for downstream collaborative intervention decisions. In one embodiment of the invention, this module includes three functional subunits: multimodal feature classification and risk prediction, dynamic risk score update, and multi-task joint training and evaluation.

[0023] In the multimodal feature classification and risk prediction subunit, feature vectors are fused. First, feature dimensionality reduction is performed using a two-layer fully connected network. The first layer compresses the feature dimension from 256 to 64, and the second layer further compresses it to 32. ReLU activation and Dropout regularization (with a dropout rate of 0.3) are used between the two layers to prevent overfitting. The dimensionality-reduced feature vectors are then input into two parallel classification heads: a risk level classification head outputs a 4-dimensional vector corresponding to four risk levels: low risk, low-medium risk, medium-high risk, and high risk. The values ​​of each dimension are normalized using Softmax to represent the probability distribution of belonging to that level. The risk type classification head outputs a 6-dimensional vector corresponding to six risk events: falls, pressure sores, infections, medication errors, malnutrition, and cognitive decline. Softmax normalization is also used to output the confidence scores for each risk type. This parallel design with two classification heads allows the system to simultaneously answer two key questions: how severe is the risk and what type of risk is most likely? This provides multi-dimensional reference information for subsequent intervention decisions.

[0024] In the dynamic risk score update subunit, the system employs a sliding window mechanism to continuously and dynamically update the risk score. Preferably, the sliding window size is set to 5 minutes. Within each window period, the system combines the current window's risk level classification probability with historical risk scores, and calculates the updated risk score using an exponentially weighted average. , in, For the first The comprehensive risk score for each time window, with a value range of [value range missing]. ; The overall risk score for the previous time window; The high-risk probability value output by the risk level classification header within the current window (i.e., the probability of the corresponding high-risk level in the 4-dimensional output vector); The attenuation coefficient is set to 0.9 in this embodiment. This value means that 90% of the weight in the current score comes from historical cumulative scores and 10% comes from the current observation, thus maintaining score stability while taking into account sensitive response to new data. Based on comprehensive risk scoring... The system determines the risk level according to the following rules: when If the risk exceeds this threshold for three consecutive windows (i.e., within 15 minutes), it is considered high-risk (Level 3); when When, it is judged as medium risk (level 2); when When the risk level is low (0-1), a high-risk assessment requires an additional condition of exceeding the threshold for three consecutive windows. This is to filter out false high-risk signals caused by transient sensor noise or brief changes in patient position, thereby reducing the false alarm rate. In one embodiment of the invention, the design of this multi-level assessment mechanism fully considers the actual evolution characteristics of risk events in elderly care scenarios—true health risks usually exhibit a gradual deterioration trend rather than an instantaneous jump. Therefore, the reliability of the assessment is improved by requiring consistency in scores across multiple consecutive windows. Simultaneously, for extreme emergencies (e.g., a sudden jump in risk score above 0.95 within a single window), the system also has a bypass triggering mechanism to directly activate a high-risk alarm, ensuring that the response to sudden dangerous events is not delayed due to the consecutive window assessment condition.

[0025] To capture the evolution trend of risk states over a longer time span, the system introduces a Long Short-Term Memory (LSTM) network to perform temporal modeling of historical risk state sequences. The LSTM network receives risk score sequences from the most recent 24 time windows (i.e., 2 hours) as input, with its hidden layer dimension set to 64. Through the coordinated operation of forget gates, input gates, and output gates, it selectively remembers or forgets historical risk information. Preferably, the LSTM hidden state is reset every 24 hours to avoid long-term dependency bias causing the model to overemphasize distant historical information while ignoring recent changes. The output of the LSTM network is fused with the risk score of the current window to jointly constitute the final risk assessment result vector. This result includes information in three dimensions: risk level label, confidence distribution of six risk categories, and risk trend prediction (rising, stable, or declining).

[0026] In the multi-task joint training and evaluation sub-unit, the system uses a weighted loss function to jointly optimize the two tasks of risk level classification and risk type classification: , in, The ranking loss function based on the AUC-ROC index measures the model's ability to distinguish between different risk levels. The loss function is maximized based on F1-score, focusing on the harmonic mean of precision and recall of the model on imbalanced data; and These are the weight coefficients for the two loss components. This weight configuration reflects the practical need in elderly care scenarios where accurately distinguishing risk levels takes precedence over risk type segmentation. The hyperparameters during training were set as follows: learning rate 0.001, batch size 32, Adam optimizer used, weight decay coefficient 0.0001, and 100 training epochs.

[0027] The multidisciplinary collaborative intervention module 4 is the decision-making and execution layer of this system. Its core function is to transform the risk assessment results output by the intelligent risk assessment module 3 into specific intervention action plans, and to achieve optimal allocation of nursing resources through intelligent scheduling algorithms, while establishing a feedback loop from intervention execution to model optimization. In one embodiment of the present invention, this module includes four functional sub-units: medical knowledge graph construction and query, intelligent task allocation engine, multi-terminal real-time data synchronization, and closed-loop feedback and model optimization.

[0028] In the medical knowledge graph construction and query subunit, the system constructs a dedicated knowledge graph for the field of geriatric care based on the DARLING framework. Preferably, the knowledge graph defines four core entity types: risk level entities (containing nodes at four levels from low to high risk), intervention entity entities (such as fall prevention protocols, pressure ulcer turning plans, infection isolation protocols, etc.), caregiver qualification entities (such as first aid certification, advanced caregiver certificates, specialist nursing qualifications, etc.), and patient attribute entities (including age, medical history, medication records, allergy information, etc.). The rules governing relationships between entities are automatically generated through medical guideline text mining and are reviewed and confirmed by clinical experts. In one embodiment of this invention, a typical inference rule example is: if the risk level is greater than or equal to level 2 and the patient's medical history includes osteoporosis, the system recommends a set of intervention measures including activating bed rail protection, notifying family members, arranging dedicated personnel for care, and developing a fall prevention care plan. The knowledge graph uses the hyperplane mapping method in the DARLING framework to associate patient demographic data with risk entities, enabling dynamic adjustment of the recommended priority of intervention measures based on the individual characteristics of different patients. Preferably, the knowledge graph also incorporates evidence-based evaluation of the effectiveness of interventions. When multiple interventions are applicable to a specific risk scenario, the system prioritizes recommending the combination of interventions with the highest level of evidence, thereby ensuring the scientific rigor and standardization of the intervention plan. Furthermore, the knowledge graph supports incremental updates. When new medical guidelines or nursing standards are published, administrators can quickly incorporate new knowledge entries into the graph using structured templates without rebuilding the entire knowledge base.

[0029] In the intelligent task allocation engine subunit, the system receives risk event parameters (including risk type, risk level, and patient spatial location) output by the intelligent risk assessment module 3, as well as a list of currently on-duty caregivers' skill profiles. It then calculates the optimal task allocation scheme using an improved Hungarian algorithm. Unlike the traditional Hungarian algorithm, this invention introduces dynamic weight adjustment and load balancing mechanisms in the matching degree calculation. Specifically, the adjusted matching degree calculation formula for each caregiver-task pair is as follows: , in, This is a skills matching score, dynamically calculated based on the degree of matching between the caregiver's qualification tags and the skills required for the task, with a value range of [value missing]. For example, the skill match rate for high-risk tasks related to falls is 1.0 for nurses with first aid certification, while the match rate for similar tasks is 0.7 for nurses with only basic nursing qualifications. Spatial distance weights are used, based on the Euclidean distance between the caregiver's current location and the patient's location. Calculate, the range of values ​​is The closer the distance, the higher the weight. The specific calculation method is as follows: ,in The maximum diagonal distance of the nursing area; and These are the coefficients for skill weighting and distance weighting, respectively. This ratio reflects the principle that skill suitability takes precedence over spatial convenience in the allocation of nursing tasks. As a load balancing adjustment item, its calculation formula is: , in, The number of tasks currently assigned to the caregiver; The average number of tasks for all on-duty caregivers; The load balancing coefficient is set to 0.3 in this embodiment. The mechanism of this adjustment is as follows: caregivers with fewer tasks than the average will receive a positive matching gain, while those with more tasks will receive a negative matching penalty, thus promoting a more balanced distribution of tasks among caregivers. Furthermore, the algorithm includes a hard constraint: a single caregiver cannot be assigned more than 4 tasks within 2 hours; when a caregiver reaches this limit, the system automatically removes them from the candidate assignment list. The improved Hungarian algorithm, through the aforementioned dynamic weight calculation and constraints, reduces the time complexity of task allocation from that of traditional greedy algorithms. Optimized to The level ensures that the match between caregivers' skills and task requirements remains above 90%.

[0030] In the multi-terminal real-time data synchronization subunit, the system uses the WebSocket protocol to achieve bidirectional real-time communication between multiple terminals, including caregiver mobile devices, nurse workstations, and family applications. Preferably, the WebSocket data frame structure includes five fields: operation type, timestamp, risk level, intervention measure ID, and execution status. The client employs a random heartbeat interval strategy (base interval 30s, random offset range ±10s) to avoid instantaneous server load spikes caused by multiple clients simultaneously sending heartbeat packets. The server uses a frame queue management mechanism to ensure message processing latency does not exceed 50ms, and end-to-end data synchronization latency is controlled within 500ms. This real-time synchronization mechanism enables caregivers' mobile devices to receive intervention command pushes within 500ms when the system detects a high-risk event, nurse workstations to synchronously update the risk status dashboard, and family applications to obtain real-time patient care progress information, thereby establishing an information sharing channel between medical staff and patients' families.

[0031] In the closed-loop feedback and model optimization subunit, caregivers record intervention results via mobile terminal using voice input or structured forms. The system automatically extracts risk mitigation effectiveness indicators (such as changes in risk scores after intervention) and generates timestamped feedback records. Based on the collected intervention feedback data, the system uses a contrastive learning mechanism to optimize the model parameters of the intelligent risk assessment module 3. Specifically, using risk assessment feature pairs before and after intervention for the same patient as positive samples, and randomly paired feature pairs from different patients or different time periods as negative samples, the InfoNCE contrastive loss function is constructed: , in, and These are the feature representations of positive sample pairs before and after intervention, respectively. For the first Feature representation of each negative sample; The cosine similarity function; The temperature over-parameter is set to 0.07 in this embodiment; The negative sample ratio is set to 1:16, meaning each pair of positive samples is paired with 16 negative samples. The contrastive learning optimization process uses a learning rate of 0.0001 and a batch size of 32. By minimizing the InfoNCE loss function, the model learns the causal feature representation of the impact of intervention behavior on risk status, thereby continuously improving the accuracy and predictive ability of risk assessment. This closed-loop feedback mechanism transmits the execution results of the multidisciplinary collaborative intervention module 4 back to the intelligent risk assessment module 3, realizing a complete closed loop from assessment → intervention → feedback → optimization. This allows the overall system performance to continuously improve with the accumulation of practical application data.

[0032] Reference Figure 2 This invention also provides a method for intelligent early warning of caregiving based on multimodal data fusion, applied to the aforementioned intelligent early warning system for caregiving. The method includes the following steps: Step S1: Multimodal data acquisition and preprocessing.

[0033] In step S1, the system first synchronously drives the physiological sensors, visual sensors, and environmental sensors deployed in the nursing scenario to perform parallel data acquisition through a unified clock source provided by the PPS pulse generator. As described in the multimodal data acquisition and preprocessing module 1 in the system embodiment, the physiological sensors acquire physiological indicators such as heart rate, blood pressure, and body temperature, as well as inertial data such as acceleration and angular velocity at a frequency of 1Hz; the visual sensors acquire RGB-D depth image sequences at a frequency of 30Hz; and the environmental sensors acquire environmental parameters such as temperature, humidity, and air quality at a frequency of 0.5Hz. During the data acquisition process, the synchronization pulse output by the PPS pulse generator every second adds a high-precision timestamp to all sensor data, eliminating the clock drift problem caused by the difference in crystal oscillator frequencies within each device. Subsequently, linear interpolation time alignment processing based on a sliding window (window size 200ms) is performed on the acquired multimodal raw data to resample data with different sampling frequencies to a unified time reference, with the standard deviation of the time deviation controlled within 5ms. Simultaneously, the spatial coordinates of each sensor are uniformly mapped to a global coordinate system defined by the multi-camera calibration parameters through a pre-calibrated extrinsic parameter matrix, with spatial errors controlled within 5cm. A Kalman filter is introduced to smooth the spatial mapping results to suppress positioning jitter. On the edge computing device, a lightweight MobileNetV3 convolutional neural network is used to extract pose features from video keyframes in real time. This network adopts a depthwise separable convolutional structure to adapt to resource-constrained edge deployment environments, and combines ROI cropping and background subtraction algorithms to reduce data transmission volume. Z-score normalization and sliding window mean imputation are performed on the physiological time-series data. Finally, a spatiotemporally consistent multimodal feature vector set containing a 64-dimensional physiological feature vector, a 128-dimensional behavioral feature vector, and a 32-dimensional environmental feature vector is generated and passed to step S2 for fusion processing through a standardized data interface.

[0034] Step S2, dynamic weight fusion.

[0035] In step S2, the multimodal feature vector set output from step S1 is input into the dynamic weight fusion processing flow. First, a four-head multi-head self-attention mechanism is used to model cross-modal semantic associations of physiological, behavioral, and environmental features, generating initial fused features. As described in the dynamic weight fusion module 2 of the system embodiment, this process generates query, key, and value matrices through linear projection, calculates intermodal association scores using scaled dot product attention, and obtains the attention weight distribution through softmax normalization. Second, based on Dempster-Shafer evidence theory, the prediction uncertainty of each modality is quantitatively evaluated. When the intermodal conflict degree... When the threshold of 0.3 is exceeded, the evidence discount factor is applied. The fusion weights are dynamically adjusted, and a confidence-weighted mechanism based on predicted entropy is introduced as a supplement. Finally, an elastic weight consolidation algorithm (regularization coefficient) is employed. The attention module parameters are updated online every 24 hours, and event-driven rapid fine-tuning is triggered when a significant shift in data distribution (KL divergence exceeding 0.15) or continuous modal conflict is detected. This step outputs a dynamically weighted fusion feature vector with dimension 256. .

[0036] Step S3, Intelligent Risk Assessment.

[0037] In step S3, the fused feature vector output in step S2 is processed. Risk assessment is performed. First, feature dimensionality reduction is achieved through a two-layer fully connected network (256→64→32 dimensions). Then, parallel classification is performed using a risk level classification head (outputting a 4-dimensional probability distribution) and a risk type classification head (outputting a 6-dimensional confidence vector). As described in the intelligent risk assessment module 3 of the system embodiment, the system uses a sliding window exponential weighted average mechanism with a window size of 5 minutes and a decay coefficient of 0.9 to dynamically update the risk score, and follows... Furthermore, exceeding the standard for three consecutive windows indicates a high risk. For medium risk, Risk levels are determined for low-risk rules. Simultaneously, an LSTM network (64 hidden layer dimensions, with hidden states reset every 24 hours) is used to perform time-series modeling of the risk score sequence over the most recent 2 hours, capturing the risk evolution trend. A weighted loss function is used during the training phase. Perform multi-task joint optimization.

[0038] Step S4: Multidisciplinary collaborative intervention.

[0039] In step S4, a collaborative intervention process is executed based on the risk assessment results output in step S3. First, using a medical knowledge graph built on the DARLING framework, risk levels and risk types are mapped to corresponding sets of intervention measures, and the intervention priorities are dynamically adjusted according to the patient's individual attributes. Second, an improved Hungarian algorithm incorporating a load balancing adjustment term is used to comprehensively calculate the adjusted matching degree between caregivers and tasks. Under the hard constraint that a single person can perform no more than four tasks within two hours, the system outputs the optimal task allocation scheme. Then, intervention instructions are pushed in real-time to multiple terminals, including caregiver mobile devices, nurse workstations, and family member apps, via the WebSocket protocol, with end-to-end synchronization latency controlled within 500ms. Finally, intervention execution result data fed back by caregivers is collected. The contrastive learning process is driven by the InfoNCE contrastive loss function (negative sample sampling rate 1:16, temperature hyperparameter 0.07, learning rate 0.0001), which inversely optimizes the feature representation capability of the risk assessment model in step S3, forming a complete closed loop from assessment to intervention to model optimization. Through this closed-loop mechanism, the system can continuously learn the causal relationship between the actual intervention effect and risk assessment prediction, enabling the model's prediction accuracy to continuously improve with the accumulation of clinical application data.

[0040] The steps S1 to S4 described above correspond one-to-one with the multimodal data acquisition and preprocessing module 1, dynamic weight fusion module 2, intelligent risk assessment module 3, and multidisciplinary collaborative intervention module 4 in the system embodiment, forming a complete processing chain of data acquisition → feature fusion → risk assessment → collaborative intervention → feedback optimization. The feedback result of step S4 can drive the continuous optimization of model parameters in step S3, achieving iterative improvement of the system's early warning performance. A key feature of this invention is that the steps are not simply sequential but form a closed-loop control structure with feedback adjustment capabilities. The high-quality spatiotemporal alignment features provided by step S1 lay the foundation for accurate fusion in step S2. The dynamic weight fusion result of step S2 directly determines the reliability of the risk assessment in step S3, and the assessment output of step S3 is transformed into executable nursing actions through knowledge graph mapping in step S4. More importantly, the intervention effect feedback data collected in step S4 is then fed back to step S3 for model parameter optimization. This reverse information flow enables the system to possess a self-evolutionary capability similar to the experience accumulation and skill improvement of human medical teams. This deep coupling and synergistic relationship between the steps produces a significant nonlinear gain effect—that is, the overall early warning performance of the system far exceeds the simple sum of the effects of each step running independently, achieving true synergistic efficiency.

[0041] To verify the practical effect of the intelligent early warning system for caregivers based on multimodal data fusion proposed in this invention, a comparative test was conducted for three months in the geriatric ward environment of a tertiary hospital. The test environment included a 20-bed geriatric nursing ward, with a complete sensor array and edge computing equipment deployed. A total of 85 elderly patients participated in the test, with an average age of 76.4 years.

[0042] In terms of spatiotemporal alignment accuracy, the multimodal data acquisition and preprocessing module 1, through the synergistic effect of the PPS pulse generator and sliding window linear interpolation, controls the standard deviation of the time deviation between the physiological sensor and the visual sensor to 4.2ms (the target value is within 5ms), and the spatial alignment error of triangulation and Kalman filtering is 3.8cm (the target value is within 5cm). Both indicators are better than the set targets.

[0043] Regarding the accuracy of risk warnings, the collaborative work of the dynamic weighted fusion module 2 and the intelligent risk assessment module 3 enabled the system's overall warning accuracy to reach 93.2%, which is 17.5% higher than the baseline scheme using only a single physiological modality and 8.3% higher than the multimodal scheme using static equal-weighted fusion. Among them, the evidence discount mechanism reduced the false alarm rate from 24.6% to 7.2% in modal conflict scenarios, fully verifying the effectiveness of uncertainty perception confidence weighting.

[0044] Regarding early warning response speed, the system's end-to-end early warning latency averages 1.1 seconds, with edge feature extraction taking approximately 320 milliseconds, cloud fusion and evaluation taking approximately 480 milliseconds, and WebSocket message push taking approximately 300 milliseconds. This latency is reduced by approximately 78% compared to the more than 5-second latency of traditional centralized processing solutions, meeting the rapid response requirements for emergency risk events such as falls.

[0045] In terms of nursing resource allocation efficiency, the improved Hungarian algorithm processed 2,347 nursing task allocation requests during the test. The average matching degree between caregiver skills and task requirements was 91.6%. The introduction of the load balancing coefficient reduced the standard deviation of the workload among caregivers from 4.2 to 1.8. The constraint that no single person has more than 4 tasks within 2 hours was achieved 100%.

[0046] Regarding model adaptability, the Elastic Weight Consolidation (EWC) algorithm reduced the system's warning accuracy by only 2.1% (from 93.2% to 91.1%) when migrating from a hospital ward scenario to a community elderly care center scenario, while the control scheme without EWC experienced a 14.7% decrease in accuracy under the same migration conditions. The contrastive learning-driven feedback loop, during three months of continuous operation, gradually reduced the system's generalization error on newly added risk samples from an initial 18.3% to 4.6%, validating the positive effect of the closed-loop feedback mechanism on the continuous improvement of system performance.

[0047] In terms of system deployment and scalability, the MobileNetV3 lightweight convolutional neural network deployed at the edge, combined with ROI cropping and background subtraction algorithms, reduces the amount of video data transmitted by 87%, enabling the system to run stably in ordinary wards equipped with basic network environments without the need for dedicated high-speed network support. The system's strategy of freezing the underlying network every 24 hours and updating only the attention module parameters reduces the computational resources required for online updates by approximately 62%, significantly improving the system's practicality and scalability. Furthermore, the standardized medical knowledge graph built on the DARLING framework has good universality, shortening the system's deployment and adaptation cycle in new medical institutions from one month in traditional solutions to less than seven days.

[0048] In terms of user experience and collaborative efficiency, the multi-terminal real-time data synchronization mechanism based on the WebSocket protocol enables caregivers to receive complete intervention instructions within 500ms after receiving a risk warning. Family members can view the patient's risk status and care progress in real time through a dedicated application. This function received a 92% satisfaction rating from patients' families during the testing period. The improved Hungarian algorithm for intelligent task allocation reduced the average task response time for caregivers from 4.3 minutes in the manual allocation mode to 1.2 minutes, improving the overall efficiency of nursing services by approximately 72%.

[0049] In summary, this invention organically integrates key technologies such as precise spatiotemporal alignment of multimodal data, dynamic weight fusion and conflict handling, sliding window-based dynamic risk assessment, and knowledge graph-driven collaborative intervention to construct an intelligent early warning system covering the entire chain from perception to fusion, assessment, intervention, and optimization. It demonstrates significantly superior performance compared to existing technologies in terms of early warning accuracy, response speed, resource allocation efficiency, and cross-scenario generalization ability. The system architecture proposed in this invention features excellent modular design characteristics. Modules interact with each other through standardized interfaces, enabling flexible configurability and scalability. In practical deployment, modules can be selectively replaced or upgraded according to the specific needs of different nursing scenarios without affecting the overall operational stability of the system. For example, when upgrading sensor hardware, only the configuration parameters of the multimodal data acquisition and preprocessing module 1 need to be adjusted to complete the adaptation, while the downstream fusion analysis and risk assessment logic does not need to be modified. This modular and decoupled design concept not only reduces system maintenance costs but also reserves ample space for future technological evolution, allowing for the introduction of more advanced algorithms or richer data modalities. Furthermore, the elastic weight consolidation algorithm and contrastive learning feedback mechanism introduced in this invention endow the system with the ability to continuously learn and self-optimize. This allows the system to continuously accumulate clinical experience and knowledge during long-term operation, gradually improving the accuracy of early warning and the effectiveness of intervention, forming a data-driven self-reinforcing cycle. The technical solution of this invention is not only applicable to typical nursing scenarios such as hospital geriatric wards, community elderly care centers, and home-based elderly care, but can also be extended to related scenarios such as postoperative rehabilitation monitoring, long-term chronic disease management, and behavioral early warning in the field of mental health by adjusting the output configuration of entity relationship rules and risk type classification heads in the knowledge graph. It has broad technical application prospects and market promotion value.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Those skilled in the art will recognize that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the invention. The scope of protection of the present invention is defined by the appended claims and their equivalents.

[0051] The embodiments of the present invention are not limited to the specific embodiments described above. Those skilled in the art can make various equivalent changes or substitutions based on the technical solutions of the present invention, and all such changes or substitutions should be included within the protection scope of the present invention.

Claims

1. A smart early warning system for caregivers based on multimodal data fusion, characterized in that, include: The multimodal data acquisition and preprocessing module is used to synchronously drive physiological sensors, visual sensors, and environmental sensors to acquire data in parallel through a unified clock source. It performs linear interpolation time alignment based on a sliding window on the raw multimodal data to eliminate the timing deviation caused by the difference in sampling frequency. It maps the spatial coordinates of each sensor to the global coordinate system through an extrinsic parameter matrix. It deploys a lightweight convolutional neural network on the edge device to extract pose features from video keyframes and performs standardization and missing value imputation on the physiological time series data to generate a spatiotemporally consistent set of multimodal feature vectors. The dynamic weight fusion module is used to receive the multimodal feature vector set, perform cross-modal semantic association modeling of physiological features, behavioral features and environmental features through a multi-head self-attention mechanism to generate initial fusion features, quantify the prediction uncertainty of each modality based on evidence theory, dynamically correct the fusion weights through an evidence discount mechanism when the intermodal conflict exceeds a preset threshold, and use an elastic weight consolidation algorithm to adaptively update the fusion network parameters online. The intelligent risk assessment module receives the fused feature vectors, performs risk level classification and risk type classification after dimensionality reduction through a fully connected network, dynamically updates the risk score using a sliding window exponential weighted average mechanism, and combines a long short-term memory network to perform time-series modeling of historical risk states, outputting the risk assessment results. The multidisciplinary collaborative intervention module is used to map the risk assessment results into a set of intervention measures based on a medical knowledge graph. It performs intelligent scheduling of nursing tasks by integrating skill weights and distance weights and introducing a load balancing adjustment term using an improved Hungarian algorithm. It pushes intervention instructions to multiple terminals via a real-time communication protocol and collects execution feedback data to optimize the model parameters of the intelligent risk assessment module through a comparative learning mechanism.

2. The intelligent early warning system for caregivers based on multimodal data fusion according to claim 1, characterized in that, In the multimodal data acquisition and preprocessing module, the unified clock source is a PPS pulse generator, the sampling frequency of the physiological sensor is 1Hz, the sampling frequency of the visual sensor is 30Hz, and the sampling frequency of the environmental sensor is 0.5Hz; the window size of the sliding window is 200ms, the standard deviation of the time deviation after time alignment does not exceed 5ms, and the spatial error of the spatial alignment does not exceed 5cm.

3. The intelligent early warning system for caregivers based on multimodal data fusion according to claim 1, characterized in that, In the dynamic weight fusion module, the preset threshold is 0.

3. When the intermodal conflict degree exceeds 0.3, an evidence discount factor is applied to the fusion weight of each modality for correction. The regularization coefficient of the elastic weight consolidation algorithm is 0.5, the online update cycle is 24h, and the underlying feature extraction network is frozen during the update, while only the attention module parameters are updated.

4. The intelligent early warning system for caregivers based on multimodal data fusion according to claim 1, characterized in that, In the intelligent risk assessment module, the sliding window size is 5 minutes, and the decay coefficient of the index-weighted average is 0.

9. The risk level is divided into 4 levels, where a risk score greater than or equal to 0.7 and exceeding the limit for 3 consecutive windows is considered high risk, a risk score between 0.4 and 0.7 is considered medium risk, and a risk score less than 0.4 is considered low risk.

5. The intelligent early warning system for caregivers based on multimodal data fusion according to claim 1, characterized in that, In the dynamic weight fusion module, the multi-head self-attention mechanism generates query matrix, key matrix and value matrix for multimodal feature vectors through a linear projection layer, calculates the correlation score between modalities using scaled dot product attention, and obtains the attention weight distribution after normalization. The evidence theory uses a three-layer linear evidence network to generate evidence body for each modality, and calculates the intermodal conflict degree based on the intersection metric of the evidence bodies.

6. The intelligent early warning system for caregivers based on multimodal data fusion according to claim 1, characterized in that, In the intelligent risk assessment module, the risk type classification output includes confidence vectors for six types of risks: falls, pressure sores, infections, medication abnormalities, malnutrition, and cognitive decline. The multi-task joint training uses a weighted loss function, where the weight coefficient of the weighted AUC loss is 0.8 and the weight coefficient of the F1-score maximization loss is 0.

2.

7. The intelligent early warning system for caregivers based on multimodal data fusion according to claim 1, characterized in that, In the multidisciplinary collaborative intervention module, the contrastive learning mechanism adopts the InfoNCE loss function, using the risk assessment feature pairs before and after the intervention as positive samples and randomly paired non-associated feature pairs as negative samples. The negative sample sampling rate is 1:16, and the feature representation capability of the risk assessment module is optimized by minimizing the contrastive loss.

8. The intelligent early warning system for caregivers based on multimodal data fusion according to claim 1, characterized in that, In the multimodal data acquisition and preprocessing module, the lightweight convolutional neural network is based on the MobileNetV3 architecture and is deployed on an edge computing device to perform real-time inference; the visual sensor is an RGB-D depth camera, and the pose feature extraction is further combined with ROI cropping and background subtraction algorithms to reduce the amount of video data transmission.

9. The intelligent early warning system for caregivers based on multimodal data fusion according to claim 1, characterized in that, In the multidisciplinary collaborative intervention module, the medical knowledge graph is constructed based on the DARLING framework, and the defined entity types include risk level, intervention measures, caregiver qualifications, and patient attributes; in the improved Hungarian algorithm, the coefficient of skill weight is 0.7, the coefficient of distance weight is 0.3, the load balancing coefficient is 0.3, and the constraint is that the number of tasks assigned to a single caregiver within 2 hours does not exceed 4.

10. A method for intelligent early warning of caregiving based on multimodal data fusion, applied to the system described in any one of claims 1 to 9, characterized in that, Includes the following steps: Step S1, Multimodal data acquisition and preprocessing: Parallel data acquisition is performed by synchronously driving multiple types of sensors through a unified clock source. The acquired multimodal raw data is subjected to linear interpolation time alignment based on sliding window and unified spatial coordinate mapping based on extrinsic parameter matrix. Lightweight feature extraction is performed on video data on edge devices, and physiological data is standardized to generate a spatiotemporally consistent set of multimodal feature vectors. Step S2, Dynamic Weight Fusion: The multimodal feature vector set is modeled for cross-modal semantic association through a multi-head self-attention mechanism to generate initial fusion features. Based on evidence theory, intermodal conflicts are detected and dynamic weights are corrected. An elastic weight consolidation algorithm is used to perform online adaptive updates of the fusion network parameters. Step S3, Intelligent Risk Assessment: The fusion of high-dimensional feature vectors is reduced in dimensionality through a fully connected network. Risk level classification and risk type classification are performed respectively. The risk score is dynamically updated through a sliding window exponential weighted average mechanism. The historical risk status is modeled in time series by combining a long short-term memory network. Step S4, Multidisciplinary Collaborative Intervention: Based on the medical knowledge graph, the risk assessment results are mapped to a set of intervention measures. An improved Hungarian algorithm with a load balancing adjustment term is introduced to perform intelligent scheduling of nursing tasks. Intervention instructions are pushed to multiple terminals via a real-time communication protocol. Nursing execution feedback data is collected and the risk assessment model parameters in step S3 are optimized in reverse through a comparative learning mechanism.

Images