Condition monitoring management system and method for critically ill patients

Abstract

The present application relates to the technical field of disease monitoring and management, and specifically discloses a disease monitoring and management system and method for critically ill patients, which uses deep learning-based image analysis and feature extraction technology to extract facial semantics from each patient's facial image, and then dynamically aggregates the facial semantic features of each patient in multiple structures, while extracting the facial semantic features of the first frame patient and the facial semantic features of the last frame patient, and based on the multi-dimensional representation between the facial semantic features of the first frame patient, the facial semantic features of the last frame patient, and the facial semantic dynamic aggregation features of the patient, intelligently judge whether to stop the calculation, and obtain the target type image quantity value and the facial target image cycle duration ratio. In this way, the temporal continuity of expression changes can be better understood to ensure that no important expression transition points are missed. At the same time, more subtle expression changes can be captured, thereby improving the accuracy of expression recognition.

Description

Technical Field

[0001] The present application relates to the technical field of disease condition monitoring and management, and more specifically, to a disease condition monitoring and management system and method for critically ill patients. Background Art

[0002] Severely ill patients need to be closely monitored and receive intensive treatment in the intensive care unit (ICU) due to organ dysfunction, severe trauma or disease. These patients' vital signs are unstable and require continuous and dynamic monitoring to assess their pathophysiological state and the severity of their illness. Effective monitoring can not only help doctors to provide early warnings, accurately assess the severity of the disease and the effectiveness of treatment, but also adjust treatment plans in a timely manner to improve the success rate of treatment.

[0003] The invention with publication number CN118737361B discloses a patient monitoring and management system for critical care units based on data analysis. It collects and analyzes facial images of patients in different treatment cycles to calculate the number of target type images and obtain the facial target image cycle duration ratio. It also analyzes data such as symptoms, physiological indicators, and medical resource usage to calculate the periodic disease reference index and disease change covariance. Based on these data analyses, it monitors the patient's disease trend in real time and issues early warnings.

[0004] The target type image quantity value in the above patent is obtained by performing individual facial expression recognition on each frame in a video stream and counting the number of images showing a specific expression (such as pain). However, processing each frame of the image individually may ignore the temporal continuity of expression changes and the semantic association between images, resulting in inaccurate judgment of the patient's emotional state. For example, short but important changes in expression (such as a rapid change from smiling to crying) may be ignored in single-frame detection, affecting the accuracy of the overall emotional state assessment. In addition, this method mainly focuses on static expression features and is not sensitive enough to capture dynamic expression changes. The dynamic changes in the emotions of these patients can often provide richer information and reflect the real fluctuations in the patient's expression and emotions.

[0005] Therefore, an optimized condition monitoring and management program for critically ill patients is desired. Summary of the invention

[0006] The present application provides a condition monitoring management system and method for critically ill patients, which can better understand the temporal continuity of expression changes and ensure that no important expression transition points are missed. At the same time, more subtle expression changes can be captured, thereby improving the accuracy of expression recognition.

[0007] In a first aspect, a condition monitoring and management system for critically ill patients is provided, comprising: an indicator calculation module, for obtaining multiple treatment monitoring cycles of critically ill patients, and calculating the patient's typical symptom data, the cumulative abnormal time ratio of periodic physiological indicators, the patient's condition weight and the periodic medical resource utilization coefficient for each treatment monitoring cycle; an image acquisition module, for obtaining the facial video stream of each treatment monitoring cycle and extracting unit video frames therefrom to obtain a time queue of the patient's facial image; an image analysis module, for analyzing the time queue of the patient's facial image to obtain a facial target image cycle duration ratio; a monitoring module, for obtaining condition change monitoring data based on the periodic medical resource utilization coefficient and in combination with the facial target image cycle duration ratio, the patient's condition weight and the cumulative abnormal time ratio of the periodic physiological indicators; an early warning module, for obtaining the condition cycle monitoring data of the critically ill patient and generating a monitoring early warning in combination with the condition change monitoring data;

[0008] Among them, the image analysis module includes: a facial image extraction unit, which is used to extract semantic features from the time queue of the patient's facial images to obtain a time series of patient facial semantic features; a facial semantic aggregation unit, which is used to perform feature dynamic aggregation of multiple gate structures on the time series of the patient's facial semantic features to obtain patient facial semantic dynamic aggregation features; a multi-dimensional representation unit, which is used to extract the first frame patient facial semantic features and the last frame patient facial semantic features from the time series of the patient's facial semantic features, and calculate the fusion response features between the first frame patient facial semantic features, the last frame patient facial semantic features and the patient facial semantic dynamic aggregation features to obtain the current facial semantic search matching multi-dimensional representation, and based on the current facial semantic search matching multi-dimensional representation, obtain the facial target image period duration ratio.

[0009] Furthermore, the facial image extraction unit is used to: pass the time queue of the patient's facial images through a patient facial semantic feature extractor based on a deep separable convolutional neural network model to obtain a time series of patient facial semantic feature vectors as the time series of the patient's facial semantic features.

[0010] Furthermore, the facial semantic aggregation unit includes:

[0011] A patient facial semantic feature mapping subunit is used to input each patient facial semantic feature vector in the time series of the patient facial semantic feature vector into a Poincare space mapper to obtain a time series of patient facial semantic feature heterogeneous modulation coding feature vectors;

[0012] A patient facial semantic aggregation subunit is used to perform propagation aggregation on the time series of the patient facial semantic feature heterogeneous modulation coding feature vector to obtain the patient facial semantic feature heterogeneous propagation representation vector;

[0013] The patient facial semantic value calculation subunit is used to calculate the semantic association and semantic space jump value of each patient facial semantic feature heterogeneous modulation coding feature vector in the time series of the patient facial semantic feature heterogeneous modulation coding feature vector based on the patient facial semantic feature heterogeneous propagation representation vector to obtain the time series of patient facial semantic association and the time series of patient facial semantic space jump value;

[0014] The patient's facial information transfer modulation subunit is used to input the time series of the patient's facial semantic association and the time series of the patient's facial semantic space jump value into the information transfer modulation unit based on the multi-gate structure to obtain the time series of the patient's facial information transfer modulation weight;

[0015] A patient facial foreign feature significant processing subunit, used for calculating the position-weighted sum of the time series of the patient facial semantic feature foreign modulation coding feature vectors based on the time series of the patient facial information transmission modulation weights to obtain the patient facial foreign feature significant propagation representation vector;

[0016] A patient facial exotic feature compensation subunit, used for inputting the patient facial exotic feature significant propagation representation vector and the patient facial semantic feature exotic propagation representation vector into a residual unit to obtain a patient facial exotic feature significant propagation compensation representation vector;

[0017] The patient's facial semantic dynamic aggregation subunit is used to input the patient's facial foreign feature significant propagation compensation representation vector into the Euclidean space mapper to obtain the patient's facial semantic dynamic aggregation feature vector as the patient's facial semantic dynamic aggregation feature.

[0018] Furthermore, the patient facial semantic aggregation subunit is used to: input the time series of the patient facial semantic feature heterogeneous modulation coding feature vector into a feature sequence encoder based on a forward LSTM model to obtain the patient facial semantic feature heterogeneous propagation representation vector.

[0019] Furthermore, the patient's facial semantic value calculation subunit is used to:

[0020] Calculate the semantic association between each patient facial semantic feature heterogeneous modulation coding feature vector in the time series of the patient facial semantic feature heterogeneous modulation coding feature vector and the patient facial semantic feature heterogeneous propagation representation vector to obtain the time series of the patient facial semantic association;

[0021] Calculate the semantic space jump value of each patient facial semantic feature heterodomain modulation coding feature vector in the time series of the patient facial semantic feature heterodomain modulation coding feature vector relative to the patient facial semantic feature heterodomain propagation representation vector to obtain the time series of the patient facial semantic space jump value.

[0022] Furthermore, the multidimensional representation unit includes:

[0023] A first-frame patient facial semantic feature extraction subunit is used to extract a first-frame patient facial semantic feature vector as the first-frame patient facial semantic feature and a last-frame patient facial semantic feature vector as the last-frame patient facial semantic feature from the time series of the patient facial semantic feature vector;

[0024] A response feature calculation subunit is used to use the facial semantic feature vector of the first frame patient as a reference facial semantic template, the facial semantic dynamic aggregation feature vector of the patient as a dynamic facial semantic template, and the facial semantic feature vector of the last frame patient as a current facial semantic search feature, and calculate the response feature vectors between the current facial semantic search feature and the reference facial semantic template and the dynamic facial semantic template respectively to obtain a reference facial semantic response feature vector and a dynamic facial semantic response feature vector;

[0025] A current facial semantic search and matching subunit, used for fusing the reference facial semantic response feature vector and the dynamic facial semantic response feature vector to obtain a current facial semantic search and matching multidimensional representation vector as the current facial semantic search and matching multidimensional representation;

[0026] The facial target image cycle duration ratio calculation subunit is used to search and match the multi-dimensional representation vector based on the current facial semantics to obtain the facial target image cycle duration ratio.

[0027] Furthermore, the facial target image period duration ratio calculation subunit is used to:

[0028] The current facial semantic search matching multidimensional representation vector is passed through a counting end controller to obtain a control result, wherein the control result is used to indicate whether to use the last frame of the patient's facial image in the time queue of the patient's facial image as a target type image counting stop frame;

[0029] In response to the last frame of the patient's facial image being a target type image counting stop frame, counting the number of images of the plurality of patient's facial images to obtain a target type image number value;

[0030] The product of the target type image quantity value and the duration of the unit video frame is calculated, and then the ratio of the product to the duration of the facial video stream is calculated to obtain the facial target image period duration ratio.

[0031] Furthermore, the counting end controller is a classifier-based counting end controller.

[0032] In a second aspect, a method for monitoring and managing the condition of critically ill patients is provided, including: obtaining multiple treatment monitoring cycles of critically ill patients, and calculating the patient's typical symptom data, the cumulative abnormal time ratio of the periodic physiological indicators, the patient's condition weight and the periodic medical resource utilization coefficient for each treatment monitoring cycle; obtaining the facial video stream of each treatment monitoring cycle and extracting unit video frames therefrom to obtain a time queue of the patient's facial image; analyzing the time queue of the patient's facial image to obtain a facial target image cycle duration ratio; obtaining condition change monitoring data based on the periodic medical resource utilization coefficient and in combination with the facial target image cycle duration ratio, the patient's condition weight and the cumulative abnormal time ratio of the periodic physiological indicators; obtaining the periodic condition monitoring data of the critically ill patient and generating a monitoring warning in combination with the condition change monitoring data;

[0033] Among them, the time queue of the patient's facial images is analyzed to obtain the facial target image cycle time ratio, including: performing semantic feature extraction on the time queue of the patient's facial images to obtain a time series of patient facial semantic features; performing feature dynamic aggregation of multiple gate structures on the time series of the patient's facial semantic features to obtain patient facial semantic dynamic aggregation features; extracting the first frame patient facial semantic features and the last frame patient facial semantic features from the time series of the patient's facial semantic features, and calculating the fusion response features between the first frame patient facial semantic features, the last frame patient facial semantic features and the patient facial semantic dynamic aggregation features to obtain a current facial semantic search and matching multidimensional representation, and based on the current facial semantic search and matching multidimensional representation, obtaining the facial target image cycle time ratio.

[0034] Furthermore, semantic feature extraction is performed on the time queue of the patient's facial images to obtain a time series of patient facial semantic features, including: passing the time queue of the patient's facial images through a patient facial semantic feature extractor based on a deep separable convolutional neural network model to obtain a time series of patient facial semantic feature vectors as the time series of the patient's facial semantic features.

[0035] The present application provides a condition monitoring management system and method for critically ill patients, which uses deep learning-based image analysis and feature extraction technology to extract facial semantics from each patient's facial image, and then dynamically aggregates the facial semantic features of each patient in multiple structures, while extracting the facial semantic features of the first frame patient and the facial semantic features of the last frame patient, and based on the multi-dimensional representation between the facial semantic features of the first frame patient, the facial semantic features of the last frame patient, and the facial semantic dynamic aggregation features of the patient, intelligently judge whether to stop the calculation, and obtain the target type image quantity value and the facial target image cycle duration ratio. In this way, the temporal continuity of expression changes can be better understood to ensure that no important expression transition points are missed. At the same time, more subtle expression changes can be captured, thereby improving the accuracy of expression recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application are briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.

[0037] Figure 1 This is a schematic block diagram of a condition monitoring and management system for critically ill patients according to an embodiment of the present application.

[0038] Figure 2 This is a schematic diagram of data flow of an image analysis module in a condition monitoring and management system for critically ill patients according to an embodiment of the present application.

[0039] Figure 3 This is a schematic block diagram of an image analysis module in a condition monitoring and management system for critically ill patients according to an embodiment of the present application.

[0040] Figure 4 This is a schematic block diagram of a facial semantic aggregation unit in a condition monitoring and management system for critically ill patients according to an embodiment of the present application.

[0041] Figure 5 This is a schematic block diagram of a multi-dimensional characterization unit in a condition monitoring and management system for critically ill patients according to an embodiment of the present application.

[0042] Figure 6 This is a schematic flowchart of a method for monitoring and managing the condition of critically ill patients according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0044] Based on this, this application proposes a condition monitoring and management system for critically ill patients, such as Figure 1 As shown, the condition monitoring and management system for critically ill patients includes: an indicator calculation module 110, which is used to obtain multiple treatment monitoring cycles of critically ill patients, and calculate the patient's typical symptom data, the cumulative abnormal time ratio of periodic physiological indicators, the patient's condition weight and the periodic medical resource utilization coefficient for each treatment monitoring cycle; an image acquisition module 120, which is used to obtain the facial video stream of each treatment monitoring cycle and extract unit video frames therefrom to obtain the time queue of the patient's facial image; an image analysis module 130, which is used to analyze the time queue of the patient's facial image to obtain the facial target image cycle duration ratio; a monitoring module 140, which is used to obtain the condition change monitoring data based on the periodic medical resource utilization coefficient and in combination with the facial target image cycle duration ratio, the patient's condition weight and the cumulative abnormal time ratio of the periodic physiological indicators; an early warning module 150, which is used to obtain the condition cycle monitoring data of the critically ill patient and generate a monitoring early warning in combination with the condition change monitoring data.

[0045] Exemplarily, in the index calculation module 110, multiple treatment monitoring cycles of critically ill patients are obtained, and the typical symptom data of patients in each treatment monitoring cycle, the cumulative abnormal time ratio of the physiological index of the cycle, the weight of the patient's condition and the utilization coefficient of the medical resources of the cycle are calculated. It should be understood that in order to fully understand and analyze the changes in the condition of critically ill patients, first, by collecting the types of symptoms that appear in patients in different treatment monitoring cycles, and recording the time when each type of symptom first appears, the symptom types are numbered according to the time sequence. At the same time, sample patients with the same disease as the current patient are selected, and their clinical symptom texts are analyzed to identify the common typical clinical symptoms. Then, the order of symptom appearance of the current patient and the sample patient is compared, the number of symptoms with the same order is counted, and the ratio of the number of symptoms with the same order is calculated; and the typical clinical symptoms and the symptoms of the current patient are compared, the number of similar symptoms is counted, and the similarity ratio of the typical symptoms of the patient is calculated. The ratio of the number of symptoms with the same order and the similarity ratio of the typical symptoms of the patient are defined as the typical symptom data of the patient. For the cumulative abnormal time ratio of the physiological index of the cycle, the system will use the monitoring equipment to monitor the patient's physiological indicators (such as heart rate, blood pressure, etc.) in real time during each treatment monitoring cycle. When the value of the monitored physiological index exceeds the normal range, it is marked as an abnormal period and the duration of the abnormal period is recorded. Then calculate the ratio of the total duration of the abnormal period of each physiological indicator to the cumulative monitoring duration of the entire monitoring cycle, that is, obtain the abnormal time ratio of the physiological indicator. Finally, sum the abnormal time ratios of all physiological indicators to obtain the cumulative abnormal time ratio of the periodic physiological indicators. For the patient's condition weight, the system will give symptom severity scores for various symptoms that appear in each treatment monitoring cycle, and count the number of occurrences of each symptom in the cycle. Combined with the symptom severity score and the number of occurrences, a comprehensive value is calculated by a specific formula to represent the patient's condition weight in the cycle. The cycle medical resource utilization coefficient is obtained by obtaining the drug dosage and the number of emergency drug types of critically ill patients in each treatment monitoring cycle, as well as the accumulated medical expenditure value in the cycle through the online medical record system. According to the above-mentioned drug dosage, the number of emergency drug types and the medical expenditure value, the cycle medical resource utilization coefficient is calculated using a specific formula. These indicators together constitute the basic data for disease monitoring, which is crucial to understanding the changing trend of the patient's health status. Further, the above data have been explained and illustrated in the invention patent with publication number CN118737361B.

[0046] Exemplarily, in the image acquisition module 120, the facial video stream of each treatment monitoring cycle is acquired and the unit video frame is extracted from it to obtain the time queue of the patient's facial image. It should be understood that the patient's health status can be better understood by analyzing the changes in the patient's facial expression. Facial expression is an important indicator of emotion and physical state. For example, emotions such as pain, sadness or anxiety are usually reflected on the face. Continuous video streams can capture more subtle and short-term changes in expression, which is particularly important for evaluating changes in the patient's condition because it can provide information about the patient's emotional state and possible physical discomfort. Compared with static images, dynamic video analysis can establish a temporal continuity model for changes in patient expression, which helps to more accurately judge the patient's emotional state and avoid errors caused by processing each frame of the image separately. In order to achieve this process, the patient's facial condition is first continuously recorded by a camera or other video acquisition device installed in the intensive care unit (ICU) to form a facial video stream. Then, unit video frames are extracted from these video streams at a certain time interval (such as one frame per second), so that a patient's facial image sequence consisting of multiple discrete moments is formed.

[0047] Exemplarily, in the image analysis module 130, the time queue of the patient's facial images is analyzed to obtain the facial target image cycle duration ratio. Accordingly, in the image analysis module, the technical concept of the present application is to use deep learning-based image analysis and feature extraction technology to extract facial semantics of each patient's facial image, and then perform multi-structure feature dynamic aggregation of each patient's facial semantic features, and extract the facial semantic features of the first frame patient and the facial semantic features of the last frame patient at the same time, and intelligently determine whether to stop the calculation based on the multi-dimensional representation between the facial semantic features of the first frame patient, the facial semantic features of the last frame patient, and the facial semantic dynamic aggregation features of the patient to obtain the target type image quantity value and the facial target image cycle duration ratio. In this way, the temporal continuity of expression changes can be better understood to ensure that no important expression transition points are missed. At the same time, more subtle expression changes can be captured, thereby improving the accuracy of expression recognition.

[0048] In one embodiment, Figure 2 and Figure 3As shown, the image analysis module 130 includes: a facial image extraction unit 131, which is used to extract semantic features from the time queue of the patient's facial images to obtain a time series of patient facial semantic features; a facial semantic aggregation unit 132, which is used to perform feature dynamic aggregation of multiple gate structures on the time series of the patient's facial semantic features to obtain patient facial semantic dynamic aggregation features; a multi-dimensional representation unit 133, which is used to extract the first frame patient facial semantic features and the last frame patient facial semantic features from the time series of the patient's facial semantic features, and calculate the fusion response features between the first frame patient facial semantic features, the last frame patient facial semantic features and the patient facial semantic dynamic aggregation features to obtain the current facial semantic search matching multi-dimensional representation, and based on the current facial semantic search matching multi-dimensional representation, obtain the facial target image period duration ratio.

[0049] Exemplarily, in the facial image extraction unit 131, it is used to extract semantic features from the time queue of the patient's facial images to obtain a time series of the patient's facial semantic features. It should be understood that considering that facial expressions are important indicators of emotions, such as sadness, pain, tears, wailing, etc., and pain is usually reflected in facial expressions, such as frowning, closing eyes or clenching teeth. And the facial images of patients at various time points contain different degrees of expression states and implicit feature information. Therefore, in order to capture and mine the subtle expression changes at each time point to enhance the understanding of the patient's comprehensive health status, in one embodiment, the facial image extraction unit is used to: pass the time queue of the patient's facial images through a patient facial semantic feature extractor based on a deep separable convolutional neural network model to obtain a time series of patient facial semantic feature vectors as the time series of the patient's facial semantic features. In this way, useful information related to health status can be efficiently extracted from a continuous facial video stream, so as to better understand and quantify the changes in health status of critically ill patients at different time points. In particular, the depthwise separable convolution decomposes the standard convolution into two operations: depthwise convolution and pointwise convolution. That is, through depthwise convolution, the model can learn the spatial information on each input channel; while through pointwise convolution, information can be fused across channels. Such a design helps capture richer and more complex facial features, thereby better understanding the changes in the patient's health status.

[0050] Exemplarily, in the facial semantic aggregation unit 132, it is used to perform dynamic feature aggregation of multiple gate structures on the time series of the patient's facial semantic features to obtain the patient's facial semantic dynamic aggregation features. It should be understood that, considering that the time series of the patient's facial semantic feature vector represents the patient's facial feature information over time, such as showing more frowns, closed eyes or other painful expressions, this may indicate that the degree of pain is increasing. That is, when the patient's facial semantics and emotions change, the temporal dynamic information can perceive these subtle changes more keenly, providing a continuous and detailed record of emotional fluctuations. Therefore, in the technical solution of the present application, in order to more carefully analyze the dependencies and changing trends between different time frames, so as to better understand the changing patterns of the patient's facial expressions over time, so as to more comprehensively evaluate the patient's condition, in the technical solution of the present application, the time series of the patient's facial semantic features are subjected to dynamic feature aggregation of multiple gate structures to obtain the patient's facial semantic dynamic aggregation features.

[0051] In one embodiment, Figure 4As shown, the facial semantic aggregation unit 132 includes: a patient facial semantic feature mapping subunit 1321, which is used to input each patient facial semantic feature vector in the time series of the patient facial semantic feature vector into the Poincare space mapper to obtain a time series of the patient facial semantic feature heterogeneous modulation coding feature vector; a patient facial semantic aggregation subunit 1322, which is used to propagate and aggregate the time series of the patient facial semantic feature heterogeneous modulation coding feature vector to obtain the patient facial semantic feature heterogeneous propagation representation vector; a patient facial semantic value calculation subunit 1323, which is used to calculate the semantic association and semantic space jump value of each patient facial semantic feature heterogeneous modulation coding feature vector in the time series of the patient facial semantic feature heterogeneous modulation coding feature vector based on the patient facial semantic feature heterogeneous propagation representation vector to obtain the time series of patient facial semantic association and the time series of patient facial semantic space jump value; a patient facial information transmission modulation subunit 1324, which is used to The time series of the patient's facial semantic association and the time series of the patient's facial semantic space jump value are input into the information transfer modulation unit based on the multiple gate structure to obtain the time series of the patient's facial information transfer modulation weights; the patient's facial foreign feature significant processing subunit 1325 is used to calculate the position-weighted sum of the time series of the patient's facial semantic feature foreign modulation coding feature vector based on the time series of the patient's facial information transfer modulation weights to obtain the patient's facial foreign feature significant propagation representation vector; the patient's facial foreign feature compensation subunit 1326 is used to input the patient's facial foreign feature significant propagation representation vector and the patient's facial semantic feature foreign propagation representation vector into the residual unit to obtain the patient's facial foreign feature significant propagation compensation representation vector; the patient's facial semantic dynamic aggregation subunit 1327 is used to input the patient's facial foreign feature significant propagation compensation representation vector into the Euclidean space mapper to obtain the patient's facial semantic dynamic aggregation feature vector as the patient's facial semantic dynamic aggregation feature.

[0052] Exemplarily, in the patient facial semantic feature mapping subunit 1321, each patient facial semantic feature vector in the time series of the patient facial semantic feature vector is respectively input into the Poincare space mapper to obtain the time series of the patient facial semantic feature heterogeneous modulation coding feature vector. Specifically, the process can be expressed as follows:

[0053] ;

[0054] ;

[0055] ;

[0056] in, is the time series of the patient's facial semantic feature vector, , , and are the first, second, and third facial semantic feature vectors in the time series of the patient, respectively. and The facial semantic feature vectors of patients, and are the first weight matrix and the second weight matrix respectively, , , and They are the first, second, and third in the time series of the patient's facial semantic feature heterogeneous modulation encoding feature vector. and The facial semantic features of the patient are modulated by heterogeneous modulation encoding feature vectors It is the time series of the feature vector of the heterogeneous modulation encoding of the facial semantic features of the patient.

[0057] That is, the time queue of the initial feature vector is input into the Poincare space mapper, which uses the unique geometric properties of the Poincare space to capture the nonlinear relationship and hierarchy between features. It should be known to those skilled in the art that the Poincare space is a hyperbolic space that can effectively reduce the curse of dimensionality while maintaining the distance relationship between nodes (i.e., the semantic association between feature vectors at different time sequence nodes).

[0058] In one embodiment, the patient facial semantic aggregation subunit 1322 is used to: input the time series of the patient facial semantic feature heterogeneous modulation coding feature vector into a feature sequence encoder based on a forward LSTM model to obtain the patient facial semantic feature heterogeneous propagation representation vector. Specifically, the process can be expressed as:

[0059] ;

[0060] in is the time series of the feature vector of the heterogeneous modulation coding of the facial semantic features of the patient, Encode the forward LSTM. It is the heterogeneous propagation representation vector of the patient’s facial semantic features.

[0061] That is, the time queue of the patient's facial semantic feature heterogeneous modulation coding feature vector is input into the feature sequence encoder based on the forward LSTM model to use the feature sequence encoder based on the forward LSTM model to simulate the transmission of the time queue of the initial feature heterogeneous modulation coding feature vector based on the Poincare semantic space to obtain the patient's facial semantic feature heterogeneous propagation representation vector. The forward long short-term memory (LSTM) network can not only capture the long-term dependencies in the time series, but also realize the selective memory and update of information through the synergy of the forget gate, input gate and output gate, so as to avoid overfitting when processing long time series while maintaining the memory of key information.

[0062] In one embodiment, the patient facial semantic value calculation subunit 1323 is used to: calculate the semantic association between each patient facial semantic feature heterogeneous modulation coding feature vector in the time series of the patient facial semantic feature heterogeneous modulation coding feature vector and the patient facial semantic feature heterogeneous propagation representation vector to obtain the time series of the patient facial semantic association; calculate the semantic space jump value of each patient facial semantic feature heterogeneous modulation coding feature vector in the time series of the patient facial semantic feature heterogeneous modulation coding feature vector relative to the patient facial semantic feature heterogeneous propagation representation vector to obtain the time series of the patient facial semantic space jump value. Specifically, the process can be expressed as follows:

[0063] ;

[0064] ;

[0065] ;

[0066] ;

[0067] in, is the heterogeneous propagation representation vector of the patient’s facial semantic features, To calculate the square of the length of a vector, is the inverse hyperbolic cosine function, , , and They are the first, second, and third in the time series of the semantic association of the patient's face. and The semantic relevance of each patient's face, is the time series of the semantic association of the patient's face, To calculate the length of a vector, , , and They are the first, second, and third time series of the facial semantic space jump values ​​of the patient. and The facial semantic space jump value of each patient, is the time series of the patient's facial semantic space jump values.

[0068] That is, the semantic association between each initial feature foreign domain modulation coding feature vector in the time queue of the initial feature foreign domain modulation coding feature vector and the initial feature foreign domain propagation representation vector is calculated to obtain the time queue of semantic association. Here, the semantic association is used to represent the semantic intrinsic association between the message propagation feature in the Poincare space and each initial feature foreign domain modulation coding feature vector, so as to facilitate the subsequent introduction of the attention mechanism to enhance the model's attention to specific features. Next, the semantic space jump value of each initial feature foreign domain modulation coding feature vector in the time queue of the initial feature foreign domain modulation coding feature vector relative to the initial feature foreign domain propagation representation vector is calculated to obtain the time queue of the semantic space jump value. This indicator measures the trend and rate of change of feature expression over time by the degree of difference in the spatial span between the message propagation feature in the Poincare space and each initial feature foreign domain modulation coding feature vector. The calculation of the semantic space jump value can not only reveal the dynamic characteristics of the feature, but also discover periodic patterns or abnormal events in the data.

[0069] Exemplarily, in the patient facial information transfer modulation subunit 1324, the time series of the patient facial semantic association and the time series of the patient facial semantic space jump value are input into the information transfer modulation unit based on the multi-gate structure to obtain the time series of the patient facial information transfer modulation weight. Specifically, the process can be expressed as follows:

[0070] ;

[0071] in, is the time series of the semantic association of the patient's face, is the time series of the patient's facial semantic space jump value, and are the semantic association weight matrix and the semantic space weight matrix respectively. is the bias vector, represents matrix multiplication, yes function, is the time series of the modulation weights of the patient’s facial information transfer.

[0072] That is, the information transfer modulation unit dynamically adjusts the direction and strength of the information flow by combining the semantic relevance and the semantic space jump value through a multi-gating mechanism. This mechanism enables the network to selectively strengthen or weaken the transmission of specific information adaptively according to the relationship between features, thereby improving the accuracy and robustness of feature representation. The working principle of the information transfer modulation unit is similar to the attention allocation process in the human brain, which can make the model more efficient in processing complex tasks.

[0073] Exemplarily, in the patient's facial foreign feature significant processing subunit 1325 and the patient's facial foreign feature compensation subunit 1326, based on the time series of the patient's facial information transmission modulation weights, the position-weighted sum of the time series of the patient's facial semantic feature foreign modulation coding feature vector is calculated to obtain the patient's facial foreign feature significant propagation representation vector, and the patient's facial foreign feature significant propagation representation vector and the patient's facial semantic feature foreign propagation representation vector are input into the residual unit to obtain the patient's facial foreign feature significant propagation compensation representation vector. Specifically, the process can be expressed as follows:

[0074] ;

[0075] in yes The corresponding patient facial information transfer modulation weights, is the number of vectors in the time series of the patient's facial semantic feature heterogeneous modulation coding feature vector, It is the compensation representation vector of the significant propagation of the patient's facial foreign features.

[0076] That is, the importance of the feature and the temporal changes are comprehensively considered by weighted summation, and a foreign feature significant propagation representation vector that comprehensively reflects the characteristics of the feature evolution over time is obtained. The calculation method of weighted sum by position not only considers the importance of the feature, but also combines the trend of change in the time dimension, so that the final feature vector can fully reflect the spatiotemporal characteristics of the original data. Next, the foreign feature significant propagation representation vector and the initial feature foreign propagation representation vector are input into the residual unit. Residual connection technology is used to solve the gradient vanishing problem that may occur during the training of deep networks and promote the direct transfer of features. By introducing the residual learning framework, good performance can be maintained even when the network is very deep, while enhancing the richness and diversity of feature representation. The residual unit solves the gradient vanishing problem in deep networks by introducing jump connections, and also provides an additional learning path for the model.

[0077] Exemplarily, in the patient face semantic dynamic aggregation subunit 1327, the patient face foreign feature significant propagation compensation representation vector is input into the Euclidean space mapper to obtain the patient face semantic dynamic aggregation feature vector as the patient face semantic dynamic aggregation feature. Specifically, the process can be expressed as:

[0078] ;

[0079] and are the third weight matrix and the fourth weight matrix respectively, is the Euclidean space mapping operation, is the patient's facial semantic dynamic aggregated feature vector.

[0080] That is, the foreign feature significant propagation compensation representation vector is input into the Euclidean space mapper. This process converts the feature vector after complex processing back to the traditional Euclidean space, which is convenient for subsequent standard machine learning algorithms to use. Through appropriate mapping functions and parameter adjustments, effective projection from high-dimensional feature space to low-dimensional space can be achieved, which facilitates feature dimensionality reduction and visualization.

[0081] Exemplarily, in the multidimensional representation unit 133, it is used to extract the facial semantic features of the first frame patient and the facial semantic features of the last frame patient from the time series of the facial semantic features of the patient, and calculate the fusion response features between the facial semantic features of the first frame patient, the facial semantic features of the last frame patient and the dynamic aggregation features of the facial semantics of the patient to obtain the current facial semantic search matching multidimensional representation, and based on the current facial semantic search matching multidimensional representation, obtain the facial target image cycle duration ratio. It should be understood that by analyzing the state at the beginning (first frame) and the state at the end (last frame) of the monitoring cycle, the changes of the patient during the monitoring cycle can be intuitively evaluated, such as changes in emotions, pain levels or fatigue levels. This comparison helps to identify the trend of the patient's disease progression and emotional fluctuations. At the same time, using the dynamic aggregation of facial semantic features throughout the monitoring cycle (i.e., the dynamic aggregation features of the facial semantics of the patient) and comparing them with the first frame and the last frame can capture the overall change pattern of the patient throughout the monitoring process. This not only focuses on the state of the starting point and the end point, but also takes into account the changes in the entire process, providing a more comprehensive analysis perspective. The dynamic aggregation features of the first frame, the last frame, and the entire cycle are combined to form a comprehensive current facial semantic search matching multi-dimensional representation, which can more accurately identify changes in the patient's health status, especially those important but short-term expression changes that may affect the patient's health status. Based on the current facial semantic search matching multi-dimensional representation, the system can intelligently determine whether to stop calculating the number of target type images and ultimately obtain the facial target image cycle duration ratio. This ratio reflects the frequency and duration of specific facial expressions or states during the monitoring period, providing data support for disease change monitoring data and early warning information.

[0082] In one embodiment, Figure 5As shown, the multidimensional representation unit 133 includes: a first-frame patient facial semantic feature extraction subunit 1331, which is used to extract the first-frame patient facial semantic feature vector as the first-frame patient facial semantic feature and the tail-frame patient facial semantic feature vector as the tail-frame patient facial semantic feature from the time series of the patient facial semantic feature vector; a response feature calculation subunit 1332, which is used to use the first-frame patient facial semantic feature vector as a reference facial semantic template, the patient facial semantic dynamic aggregation feature vector as a dynamic facial semantic template, and the tail-frame patient facial semantic feature vector as a current facial semantic search feature to calculate the current facial semantic feature. The front facial semantic search feature is respectively matched with the response feature vectors between the baseline facial semantic template and the dynamic facial semantic template to obtain a baseline facial semantic response feature vector and a dynamic facial semantic response feature vector; a current facial semantic search matching subunit 1333 is used to fuse the baseline facial semantic response feature vector and the dynamic facial semantic response feature vector to obtain a current facial semantic search matching multidimensional representation vector as the current facial semantic search matching multidimensional representation; a facial target image period duration ratio calculation subunit 1334 is used to obtain the facial target image period duration ratio based on the current facial semantic search matching multidimensional representation vector.

[0083] Exemplarily, in the first-frame patient facial semantic feature extraction subunit 1331, the first-frame patient facial semantic feature vector is extracted from the time series of the patient facial semantic feature vector as the first-frame patient facial semantic feature and the last-frame patient facial semantic feature vector is extracted as the last-frame patient facial semantic feature. It should be understood that the first frame and the last frame represent the states at the beginning and end of the monitoring period, respectively, and can be used as important reference points. Therefore, in order to more accurately and clearly analyze the specific changes in the patient's facial state and to more accurately obtain the number of target type images, in the technical solution of the present application, the first-frame patient facial semantic feature vector and the last-frame patient facial semantic feature vector are extracted from the time series of the patient facial semantic feature vector. Therefore, by extracting the data of these two frames, it is possible to intuitively evaluate the changes in the patient during the monitoring period, such as changes in emotions, pain levels, or fatigue levels.

[0084] Exemplarily, in the response feature calculation subunit 1332, the facial semantic feature vector of the first frame patient is used as the reference facial semantic template, the facial semantic dynamic aggregation feature vector of the patient is used as the dynamic facial semantic template, and the facial semantic feature vector of the last frame patient is used as the current facial semantic search feature, and the response feature vectors between the current facial semantic search feature and the reference facial semantic template and the dynamic facial semantic template are calculated to obtain the reference facial semantic response feature vector and the dynamic facial semantic response feature vector, which can be expressed by the formula:

[0085] ;

[0086] ;

[0087] in, is the facial semantic feature vector of the patient in the tail frame, It is the point multiplication by position. is the facial semantic feature vector of the patient in the first frame, is the reference bias vector, is the baseline facial semantic response feature vector, is the patient's facial semantic dynamic aggregate feature vector, is the dynamic bias vector, is the dynamic facial semantic response feature vector.

[0088] That is, in order to more accurately evaluate the facial semantic features at the current time point, so as to intelligently evaluate whether the current frame is the last frame of the last target type, in the technical solution of the present application, the facial semantic feature vector of the patient in the first frame is used as the reference facial semantic template, the dynamic aggregation feature vector of the patient's facial semantics is used as the dynamic facial semantic template, and the facial semantic feature vector of the patient in the last frame is used as the current facial semantic search feature, and the response feature vectors between the current facial semantic search feature and the reference facial semantic template and the dynamic facial semantic template are calculated to obtain the reference facial semantic response feature vector and the dynamic facial semantic response feature vector. It can be understood that by comparing the last frame (current facial semantic search feature) with the first frame (reference facial semantic template), the changes in the facial semantic features of the patient during the monitoring period can be intuitively understood, and this comparison helps to identify the temporal change trend of the patient's disease progression and emotional fluctuations. The use of dynamic facial semantic templates (i.e., the dynamic aggregation of facial semantic features throughout the monitoring cycle) and comparison with the last frame can capture the overall change pattern of the patient during the monitoring cycle, which not only focuses on the status of the starting and end points, but also considers the changes in the entire process, thereby providing a more comprehensive analysis perspective. This helps to improve the system's ability to detect subtle changes, especially those important changes that may affect the patient's health status.

[0089] Exemplarily, in the current facial semantic search matching subunit 1333, the reference facial semantic response feature vector and the dynamic facial semantic response feature vector are fused to obtain the current facial semantic search matching multidimensional representation vector as the current facial semantic search matching multidimensional representation. It should be understood that considering that the reference facial semantic response feature vector and the dynamic facial semantic response feature vector respectively represent the response information of the current frame semantic feature relative to the first frame and the entire time frame. Therefore, in order to combine the reference facial semantic response feature vector (reflecting the changes from the beginning to the end of monitoring) with the dynamic facial semantic response feature vector (reflecting the change pattern within the entire monitoring cycle) to depict a more comprehensive patient state change map, in the technical solution of the present application, the reference facial semantic response feature vector and the dynamic facial semantic response feature vector are fused to obtain the current facial semantic search matching multidimensional representation vector, that is, the obtained vector not only contains the overall change trend, but also contains a detailed dynamic change pattern, which helps to more accurately identify the changes in the patient's health status.

[0090] Exemplarily, in the facial target image cycle duration ratio calculation subunit 1334, the facial target image cycle duration ratio is obtained based on the current facial semantic search matching multidimensional representation vector. It should be understood that the current facial semantic search matching multidimensional representation vector is passed through a classifier-based counting end controller to obtain a control result, and the control result is used to indicate whether the last frame of the patient's facial image in the time queue of the patient's facial image is used as a target type image counting stop frame, and in response to the last frame of the patient's facial image being a target type image counting stop frame, the number of images of multiple patient facial images is counted to obtain the target type image number value. That is, the reference facial semantic response feature vector and the dynamic facial semantic response feature vector obtained by fusing the reference facial semantic response feature vector and the dynamic facial semantic response feature vector are used for classification processing to intelligently determine whether to stop the calculation. In this way, the temporal continuity of expression changes can be better understood to ensure that no important expression transition points are missed. At the same time, more subtle expression changes can be captured, thereby improving the accuracy of expression recognition. Finally, the product of the number of target type images and the duration of the unit video frame is calculated, and then the ratio of the product to the duration of the facial video stream is calculated to obtain the facial target image cycle duration ratio. This can quantify the frequency and duration of specific facial expressions or states within the monitoring period, providing data support for disease change monitoring data and early warning information.

[0091] In one embodiment, the facial target image cycle duration ratio calculation subunit 1334 is used to: pass the current facial semantic search matching multidimensional representation vector through a counting end controller to obtain a control result, and the control result is used to indicate whether to use the last frame of the patient's facial image in the time queue of the patient's facial image as a target type image counting stop frame; in response to the last frame of the patient's facial image being a target type image counting stop frame, count the number of images of multiple patient facial images to obtain a target type image quantity value; calculate the product of the target type image quantity value and the duration of the unit video frame, and then calculate the ratio of the product to the duration of the facial video stream to obtain the facial target image cycle duration ratio.

[0092] In one embodiment, the counting end controller is a classifier-based counting end controller. Specifically, the current facial semantic search matching multidimensional representation vector is input into a pre-trained classifier. The classifier uses a softmax function to convert the current facial semantic search matching multidimensional representation vector into a probability distribution. There are two types of classification results: one is the target type image counting stop frame, and the other is the non-target type image counting stop frame. The softmax function calculates the probability of each category so that the sum of all category probabilities is 1. According to the probability distribution calculated by the softmax function, the category with the highest probability is selected as the classification result. If the probability of the target type image counting stop frame is the highest, the current frame is determined to be the target type image counting stop frame; otherwise, continue to process the next frame. In one embodiment, if the classifier determines that the current frame is not the target type image counting stop frame, the system will take the following steps: update the current facial semantic search feature, and use the next frame image as the new current facial semantic search feature. Re-execute the steps of facial semantic feature extraction, calculation of fusion response features, etc. to generate a new current facial semantic search matching multidimensional representation. The newly generated current facial semantic search matching multidimensional representation vector is input into the classifier again for judgment. The above process will continue until a suitable stop frame is found, that is, the classifier determines that a frame is a counting stop frame.

[0093] In the case where the patient's facial semantic dynamic aggregate feature vector represents the time-domain dynamic propagation aggregate image semantic features of the patient's facial image, the baseline facial semantic response feature vector and the dynamic facial semantic response feature vector respectively represent the image semantic response features relative to the time domain start and the entire time domain. Therefore, the current facial semantic search matching multidimensional representation vector will also have query response interaction richness due to the misalignment of the image semantic response distribution across the time domain, resulting in repeated unstructured probability mapping, affecting the accuracy of the classification results.

[0094] Preferably, the current facial semantic search matching multidimensional representation vector is passed through a classifier-based counting end controller to obtain a control result, including:

[0095] Determine the number of non-zero eigenvalues ​​in the current facial semantic search matching multidimensional representation vector, and subtract the number of non-zero eigenvalues ​​from the length of the current facial semantic search matching multidimensional representation vector to obtain the zero-dimensional value of the current facial semantic search matching multidimensional representation. ;

[0096] Calculate the zero-dimensional value of the current facial semantic search matching multidimensional representation minus one of the zero-dimensional value of the current facial semantic search matching multidimensional representation The first current facial semantic search matching multidimensional representation field fitting value obtained by multiplying and dividing by the zero-dimensional value of the current facial semantic search matching multidimensional representation and the second current face semantic search matching multidimensional representation field fitting value ;

[0097] Calculate the square root of the sum of squares of all eigenvalues ​​of the current facial semantic search matching multidimensional representation vector to obtain the current facial semantic search matching multidimensional representation mode representation value ,in, The first multidimensional representation vector representing the current facial semantic search match eigenvalues, Indicates the representation value of the current facial semantic search matching multi-dimensional representation mode;

[0098] Search the current facial semantics to match the multidimensional representation zero differentiable value Calculate each eigenvalue of the multidimensional representation vector of the current face semantic search match as an index The power function , and multiply it by the first current facial semantic search matching multidimensional representation field fitting value To obtain the local representation value of the current facial semantic search matching multidimensional representation ;

[0099] Calculate each eigenvalue of the current facial semantic search matching multidimensional representation vector Matching multidimensional representation field fitting value with the second current face semantic search and the current facial semantic search matching multidimensional representation mode representation value The product of gets the current facial semantic search matching multidimensional representation center pointing value ;

[0100] The current facial semantic search matches the multi-dimensional representation local representation value Search for the center point value of the multidimensional representation matching the current facial semantics The weighted difference The current face semantic search matching multi-dimensional representation vector composed of optimizations, where and represents the weighted hyperparameter;

[0101] The optimized current facial semantic search matching multi-dimensional representation vector is passed through a classifier-based counting end controller to obtain a control result.

[0102] Based on the vector field fitting law of high-dimensional manifold, the isolated zeros in the current facial semantic search matching multidimensional representation vector are used as the differentiable dimension of the high-dimensional manifold, so as to fix the normal direction pointing to the distribution center near the local coordinates represented by the eigenvalues ​​of the current facial semantic search matching multidimensional representation vector. In this way, the invalid repetition in the probability mapping process caused by the unstructured characteristics of the semantically rich distribution of the current facial semantic search matching multidimensional representation vector is avoided by effectively aligning the characteristic distribution pattern of the current facial semantic search matching multidimensional representation vector to the probability density differentiable field, thereby improving the accuracy of the control result obtained by the classifier-based counting end controller.

[0103] Exemplarily, in the monitoring module 140, the monitoring module 140 is used to obtain the condition change monitoring data based on the periodic medical resource utilization coefficient and in combination with the facial target image period duration ratio, the patient's condition weight and the periodic physiological index cumulative abnormal time ratio. It should be understood that by comprehensively considering the periodic medical resource utilization coefficient, the facial target image period duration ratio, the patient's condition weight and the periodic physiological index cumulative abnormal time ratio, the system can more comprehensively and accurately monitor and evaluate the condition changes of critically ill patients, and provide strong support for clinical decision-making.

[0104] Exemplarily, in the early warning module 150, it is used to obtain the periodic monitoring data of the critically ill patient and generate a monitoring early warning in combination with the condition change monitoring data. It should be understood that the condition change monitoring data is calculated based on the above-mentioned periodic monitoring data, which can reflect the trend of the patient's condition change in different treatment monitoring cycles. By dynamically analyzing these data, early signs of worsening of the condition can be more accurately identified. Combining the condition cycle monitoring data and the condition change monitoring data, the system can evaluate the patient's health status in real time. If it is detected that certain key indicators are out of the normal range or change significantly, the system will immediately generate a monitoring early warning to remind medical staff to pay attention to the patient's condition changes. Monitoring early warning can help the medical team take quick action, such as adjusting the dosage of the drug, increasing the level of care, or conducting further examinations. This helps prevent a sharp deterioration of the condition and improve the treatment effect and patient safety.

[0105] In summary, the condition monitoring and management method for critically ill patients according to the embodiment of the present application is explained, which uses image analysis and feature extraction technology based on deep learning to perform facial semantic extraction of each patient's facial image, and then performs dynamic feature aggregation of multiple structures on the facial semantic features of each patient, and extracts the facial semantic features of the first frame patient and the facial semantic features of the last frame patient, and based on the multi-dimensional representation between the facial semantic features of the first frame patient, the facial semantic features of the last frame patient and the facial semantic dynamic aggregation features of the patient, it intelligently determines whether to stop the calculation, and obtains the target type image quantity value and the facial target image cycle duration ratio. In this way, the temporal continuity of expression changes can be better understood to ensure that no important expression transition points are missed. At the same time, more subtle expression changes can be captured, thereby improving the accuracy of expression recognition.

[0106] Figure 6 Schematic flow chart of the method for monitoring and managing the condition of critically ill patients according to an embodiment of the present application. Figure 6 As shown, the condition monitoring and management method for critically ill patients includes: S1, obtaining multiple treatment monitoring cycles of critically ill patients, and calculating the patient's typical symptom data, the cumulative abnormal time ratio of periodic physiological indicators, the patient's condition weight and the periodic medical resource utilization coefficient for each treatment monitoring cycle; S2, obtaining the facial video stream of each treatment monitoring cycle and extracting unit video frames therefrom to obtain the time queue of the patient's facial image; S3, analyzing the time queue of the patient's facial image to obtain the facial target image cycle duration ratio; S4, based on the periodic medical resource utilization coefficient, and in combination with the facial target image cycle duration ratio, the patient's condition weight and the cumulative abnormal time ratio of the periodic physiological indicators, obtaining the condition change monitoring data; S5, obtaining the critically ill patient's condition cycle monitoring data and combining it with the condition change monitoring data to generate a monitoring warning.

[0107] In one embodiment, the time queue of the patient's facial images is analyzed to obtain the facial target image cycle duration ratio, including: performing semantic feature extraction on the time queue of the patient's facial images to obtain a time series of patient facial semantic features; performing feature dynamic aggregation of multiple gate structures on the time series of the patient's facial semantic features to obtain patient facial semantic dynamic aggregation features; extracting the first frame patient facial semantic features and the last frame patient facial semantic features from the time series of the patient's facial semantic features, and calculating the fusion response features between the first frame patient facial semantic features, the last frame patient facial semantic features and the patient facial semantic dynamic aggregation features to obtain a current facial semantic search and match multidimensional representation, and based on the current facial semantic search and match multidimensional representation, obtaining the facial target image cycle duration ratio.

[0108] In one embodiment, semantic feature extraction is performed on the time queue of the patient's facial images to obtain a time series of patient facial semantic features, including: passing the time queue of the patient's facial images through a patient facial semantic feature extractor based on a deep separable convolutional neural network model to obtain a time series of patient facial semantic feature vectors as the time series of the patient's facial semantic features.

[0109] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned method for monitoring and managing the condition of critically ill patients have been described in detail above. Figures 1 to 5 The present invention has been described in detail in the description of the condition monitoring management system for critically ill patients, and therefore, its repeated description will be omitted.

[0110] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0111] In the various embodiments of the present application, unless otherwise specified or logically conflicting, the terms and / or descriptions between the various embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0112] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0114] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A condition monitoring and management system for critically ill patients, characterized in that: include: An indicator calculation module is used to obtain multiple treatment monitoring cycles of critically ill patients, and calculate the patient's typical symptom data, the cumulative abnormal time ratio of the periodic physiological indicators, the patient's condition weight and the periodic medical resource utilization coefficient for each treatment monitoring cycle; an image acquisition module is used to obtain the facial video stream of each treatment monitoring cycle and extract unit video frames therefrom to obtain the time queue of the patient's facial image; an image analysis module is used to analyze the time queue of the patient's facial image to obtain the facial target image cycle duration ratio; a monitoring module is used to obtain the condition change monitoring data based on the periodic medical resource utilization coefficient and in combination with the facial target image cycle duration ratio, the patient's condition weight and the cumulative abnormal time ratio of the periodic physiological indicators; An early warning module is used to obtain the periodic monitoring data of the critically ill patient's condition and generate a monitoring early warning in combination with the condition change monitoring data; Among them, the image analysis module includes: a facial image extraction unit, which is used to extract semantic features from the time queue of the patient's facial images to obtain a time series of patient facial semantic features; a facial semantic aggregation unit, which is used to perform feature dynamic aggregation of multiple gate structures on the time series of the patient's facial semantic features to obtain patient facial semantic dynamic aggregation features; a multi-dimensional representation unit, which is used to extract the first frame patient facial semantic features and the last frame patient facial semantic features from the time series of the patient's facial semantic features, and calculate the fusion response features between the first frame patient facial semantic features, the last frame patient facial semantic features and the patient facial semantic dynamic aggregation features to obtain the current facial semantic search matching multi-dimensional representation, and based on the current facial semantic search matching multi-dimensional representation, obtain the facial target image period duration ratio.

2. The condition monitoring and management system for critically ill patients according to claim 1, characterized in that: The facial image extraction unit is used to: pass the time queue of the patient's facial images through a patient facial semantic feature extractor based on a deep separable convolutional neural network model to obtain a time series of patient facial semantic feature vectors as the time series of the patient's facial semantic features.

3. The condition monitoring and management system for critically ill patients according to claim 2, characterized in that: The facial semantic aggregation unit comprises: A patient facial semantic feature mapping subunit is used to input each patient facial semantic feature vector in the time series of the patient facial semantic feature vector into a Poincare space mapper to obtain a time series of patient facial semantic feature heterogeneous modulation coding feature vectors; A patient facial semantic aggregation subunit is used to perform propagation aggregation on the time series of the patient facial semantic feature heterogeneous modulation coding feature vector to obtain the patient facial semantic feature heterogeneous propagation representation vector; The patient facial semantic value calculation subunit is used to calculate the semantic association and semantic space jump value of each patient facial semantic feature heterogeneous modulation coding feature vector in the time series of the patient facial semantic feature heterogeneous modulation coding feature vector based on the patient facial semantic feature heterogeneous propagation representation vector to obtain the time series of patient facial semantic association and the time series of patient facial semantic space jump value; The patient's facial information transfer modulation subunit is used to input the time series of the patient's facial semantic association and the time series of the patient's facial semantic space jump value into the information transfer modulation unit based on the multi-gate structure to obtain the time series of the patient's facial information transfer modulation weight; A patient facial foreign feature significant processing subunit, used for calculating the position-weighted sum of the time series of the patient facial semantic feature foreign modulation coding feature vectors based on the time series of the patient facial information transmission modulation weights to obtain the patient facial foreign feature significant propagation representation vector; A patient facial foreign feature compensation subunit, used for inputting the patient facial foreign feature significant propagation representation vector and the patient facial semantic feature foreign feature propagation representation vector into a residual unit to obtain a patient facial foreign feature significant propagation compensation representation vector; The patient's facial semantic dynamic aggregation subunit is used to input the patient's facial foreign feature significant propagation compensation representation vector into the Euclidean space mapper to obtain the patient's facial semantic dynamic aggregation feature vector as the patient's facial semantic dynamic aggregation feature.

4. The condition monitoring and management system for critically ill patients according to claim 3, characterized in that: The patient facial semantic aggregation subunit is used to: input the time series of the patient facial semantic feature heterogeneous modulation coding feature vector into a feature sequence encoder based on a forward LSTM model to obtain the patient facial semantic feature heterogeneous propagation representation vector.

5. The condition monitoring and management system for critically ill patients according to claim 4, characterized in that: The patient's facial semantic value calculation subunit is used to: Calculate the semantic association between each patient facial semantic feature heterogeneous modulation coding feature vector in the time series of the patient facial semantic feature heterogeneous modulation coding feature vector and the patient facial semantic feature heterogeneous propagation representation vector to obtain the time series of the patient facial semantic association; Calculate the semantic space jump value of each patient facial semantic feature heterodomain modulation coding feature vector in the time series of the patient facial semantic feature heterodomain modulation coding feature vector relative to the patient facial semantic feature heterodomain propagation representation vector to obtain the time series of the patient facial semantic space jump value.

6. The condition monitoring and management system for critically ill patients according to claim 5, characterized in that: The multidimensional representation unit comprises: A first-frame patient facial semantic feature extraction subunit is used to extract a first-frame patient facial semantic feature vector as the first-frame patient facial semantic feature and a last-frame patient facial semantic feature vector as the last-frame patient facial semantic feature from the time series of the patient facial semantic feature vector; A response feature calculation subunit is used to use the facial semantic feature vector of the first frame patient as a reference facial semantic template, the facial semantic dynamic aggregation feature vector of the patient as a dynamic facial semantic template, and the facial semantic feature vector of the last frame patient as a current facial semantic search feature, and calculate the response feature vectors between the current facial semantic search feature and the reference facial semantic template and the dynamic facial semantic template respectively to obtain a reference facial semantic response feature vector and a dynamic facial semantic response feature vector; A current facial semantic search and matching subunit, used for fusing the reference facial semantic response feature vector and the dynamic facial semantic response feature vector to obtain a current facial semantic search and matching multidimensional representation vector as the current facial semantic search and matching multidimensional representation; The facial target image cycle duration ratio calculation subunit is used to search and match the multi-dimensional representation vector based on the current facial semantics to obtain the facial target image cycle duration ratio.

7. The condition monitoring and management system for critically ill patients according to claim 6, characterized in that: The facial target image period duration ratio calculation subunit is used to: The current facial semantic search matching multidimensional representation vector is passed through a counting end controller to obtain a control result, wherein the control result is used to indicate whether to use the last frame of the patient's facial image in the time queue of the patient's facial image as a target type image counting stop frame; In response to the last frame of the patient's facial image being a target type image counting stop frame, counting the number of images of the plurality of patient's facial images to obtain a target type image number value; The product of the target type image quantity value and the duration of the unit video frame is calculated, and then the ratio of the product to the duration of the facial video stream is calculated to obtain the facial target image period duration ratio.

8. The condition monitoring and management system for critically ill patients according to claim 7, characterized in that: The counting end controller is a classifier-based counting end controller.

9. A method for monitoring and managing the condition of critically ill patients, characterized in that: include: Acquire multiple treatment monitoring cycles of critically ill patients, and calculate the patient's typical symptom data, the cumulative abnormal time ratio of the periodic physiological indicators, the patient's condition weight and the periodic medical resource utilization coefficient for each treatment monitoring cycle; acquire the facial video stream of each treatment monitoring cycle and extract unit video frames therefrom to obtain the time queue of the patient's facial image; analyze the time queue of the patient's facial image to obtain the facial target image cycle duration ratio; based on the periodic medical resource utilization coefficient, and in combination with the facial target image cycle duration ratio, the patient's condition weight and the cumulative abnormal time ratio of the periodic physiological indicators, obtain the condition change monitoring data; Obtaining the periodic monitoring data of the critically ill patient's condition and combining it with the monitoring data of the condition change to generate a monitoring warning; Among them, the time queue of the patient's facial images is analyzed to obtain the facial target image cycle time ratio, including: performing semantic feature extraction on the time queue of the patient's facial images to obtain a time series of patient facial semantic features; performing feature dynamic aggregation of multiple gate structures on the time series of the patient's facial semantic features to obtain patient facial semantic dynamic aggregation features; extracting the first frame patient facial semantic features and the last frame patient facial semantic features from the time series of the patient's facial semantic features, and calculating the fusion response features between the first frame patient facial semantic features, the last frame patient facial semantic features and the patient facial semantic dynamic aggregation features to obtain a current facial semantic search and matching multidimensional representation, and based on the current facial semantic search and matching multidimensional representation, obtaining the facial target image cycle time ratio.

10. The method for monitoring and managing the condition of critically ill patients according to claim 9, characterized in that: The method further comprises: performing semantic feature extraction on the time queue of the patient's facial images to obtain a time series of the patient's facial semantic features, comprising: passing the time queue of the patient's facial images through a patient's facial semantic feature extractor based on a deep separable convolutional neural network model to obtain a time series of patient's facial semantic feature vectors as the time series of the patient's facial semantic features.

Images