A method for anomaly identification in ICU patients based on video surveillance

Abstract

This application discloses a method for anomaly identification in ICU patients based on video surveillance, relating to the field of video surveillance technology. The method includes: collecting historical data; establishing a standard set and a habitual feature set based on the collected historical data; generating a prediction set based on the standard set and habitual feature set; collecting real-time data; comparing the real-time data with the standard set, habitual feature set, and prediction set to obtain anomaly coefficients; calculating a comprehensive risk coefficient based on the anomaly coefficients; and triggering an early warning based on the comprehensive risk coefficient. The method also includes real-time collection of several patient condition types; selecting one condition type through a two-level identification method; quantifying the patient's risk level through the calculation of the comprehensive risk coefficient; providing intuitive decision support for medical staff; and improving the accuracy of judgment through comparative analysis of the standard set, habitual feature set, and prediction set.

Description

Technical Field

[0001] This invention relates to the field of video surveillance technology, and in particular to a method for identifying abnormalities in ICU patients based on video surveillance. Background Technology

[0002] In modern healthcare systems, the management and monitoring of chronic diseases has always been a key focus and challenge in clinical nursing. This is especially true for patients with multiple chronic diseases, such as heart failure, diabetes, and coronary heart disease, whose conditions are complex and variable, with diverse symptoms and potential interactions between different diseases, making disease monitoring and anomaly identification exceptionally difficult. With the rapid development of video surveillance technology and artificial intelligence algorithms, video surveillance-based disease anomaly identification methods have gradually become a research hotspot. This method collects patients' facial expressions, movements, and physiological data in real time using video surveillance equipment, and analyzes this data using advanced image processing and machine learning algorithms to achieve real-time monitoring and early warning of anomalies.

[0003] Traditional disease monitoring methods rely primarily on manual observation and regular checkups by healthcare professionals. This approach is not only time-consuming and labor-intensive but also susceptible to subjective factors, leading to a low rate of timely detection of changes in the condition and delays in treatment. Existing video-based methods for identifying disease anomalies still have some shortcomings. First, normal facial expressions, movements, and physiological data vary among different patients, and data for the same patient may also differ at different times and under different conditions. Second, patients develop habitual facial expressions and movements in their daily lives. These habitual behaviors may be unrelated to the disease state but are easily misinterpreted as abnormal. Summary of the Invention

[0004] This application provides a method for identifying abnormal ICU patients based on video surveillance. By calculating a comprehensive risk coefficient, the method quantifies the degree of risk of patients, providing intuitive decision support for medical staff. Through comparative analysis of standard sets, habitual feature sets, and prediction sets, the method improves the accuracy of judgment.

[0005] This application provides a method for anomaly identification in ICU patients based on video surveillance, including:

[0006] S101, Collect historical data, establish a standard set and a habit feature set based on the collected historical data, and generate a prediction set based on the standard set and the habit feature set; the standard set consists of the patient's facial expressions, actions and corresponding physiological data in a normal state, the habit feature set includes normal habits and disease habits, normal habits refer to the most frequent facial expressions and actions unrelated to the disease since the patient was admitted to the hospital, and disease habits refer to the facial expressions and actions generated after the patient became ill, and the prediction set includes physiological indicator thresholds and behavioral feature attention areas;

[0007] S102, collect real-time data, compare the real-time data with the standard set, habitual feature set, and prediction set to obtain the data anomaly coefficient, calculate the comprehensive risk coefficient based on the data anomaly coefficient, and trigger an early warning based on the comprehensive risk coefficient; compare the real-time collected physiological data with the normal physiological data range in the standard set, set a physiological data anomaly index according to the degree of deviation; compare the real-time collected facial expression and movement data with the normal facial expressions and movements in the standard set, and match them with habitual facial expressions and movements in the habitual feature set, set an facial expression and movement anomaly index according to the degree of anomaly; count the frequency of occurrence of habitual facial expressions and movements collected in real-time, compare it with the normal frequency in the habitual feature set, set a frequency anomaly index according to the degree of frequency anomaly, and calculate the comprehensive risk coefficient using the following formula: ,in, This represents the overall risk factor. Indicates an index of abnormal physiological data. Indicates an abnormality index of facial expressions and movements. Indicates the frequency anomaly index. , , Let be the weights of each index, and + + =1;

[0008] S103 collects several types of patient conditions in real time and selects one type of condition through a two-level identification method;

[0009] The two-level identification method consists of physiological indicator identification and behavioral feature identification. The risk coefficient of each disease type is calculated based on the patient's real-time physiological data. The disease type with the highest risk coefficient is selected for behavioral feature identification. Behavioral feature identification is performed based on the selected high-risk diseases. The behavioral feature identification is performed by mapping the trajectory of conflicting actions in a three-dimensional spatial coordinate system and calculating the percentage of the overlap area between the trajectory and the preset attention area of ​​each disease type as the matching degree. When the matching degree of a disease type exceeds a threshold, the disease type is identified. The threshold refers to the preset comprehensive matching degree threshold for that disease type.

[0010] Preferably,

[0011] S201, Obtain data on patients with missing facial expressions from the standard set and habit feature set, construct a database based on the obtained data on patients with missing facial expressions, input the data in the database into the machine learning model based on the constructed database, the model outputs whether each action is a compensatory behavior, the determined compensatory behavior records are used as a compensatory behavior library, combined with the verified mapping relationship in the compensatory behavior library, actions with compensatory characteristics are selected, and finally a structured record is generated, the structured record being a compensatory mapping table;

[0012] S202 matches the real-time detected action data with the compensation mapping table, calculates the action matching degree and confidence level, and generates an early warning decision based on the confidence level.

[0013] Preferably, the formula for calculating the dynamic time warp distance is: ,in, For dynamic time curvature, For the detected action time series, This is the action feature vector at the i-th time point; The reference action time series preset in the compensation mapping table Let j be the action feature vector at time point j. Let P represent the Euclidean distance, and let P be the set of all regularized paths that satisfy the boundary conditions, monotonicity, and continuity. The formula for converting the dynamic time regularized distance into action matching degree is: ,in, This is the dynamic time-normalized distance between the current real-time action and the reference action. This is the maximum dynamic time-normalized distance obtained by comparing all actions in historical data.

[0014] Preferably, the formula for calculating the confidence level is: ,in, The weights for action matching degree, The weight of physiological offset.

[0015] Preferably, the generated early warning decision is as follows: based on the calculated confidence level, a threshold range is set, which includes a maximum threshold and a minimum threshold. When the confidence level is less than the minimum threshold, it is considered low confidence and no early warning is triggered. When the confidence level is greater than or equal to the minimum threshold and less than or equal to the maximum threshold, it is considered medium confidence and multi-disease type identification is initiated. When the confidence level is greater than the maximum threshold, a compensatory early warning is triggered and pushed to the doctor's terminal.

[0016] Preferably,

[0017] S301, construct a disease type matrix based on the type of disease;

[0018] S302, based on the disease type matrix, update the standard set, habitual feature set, and prediction set.

[0019] Preferably, the formula for identifying associated disease types based on disease type and calculating the influence coefficient based on disease type and associated disease type is as follows: ,in, The influence coefficient represents the weighted impact of changes in the main disease type indicator on the risk of related disease types. To correlate the risk increment of different disease types, The changes in indicators related to the main disease type. The baseline risk represents the basic risk of the associated disease type under the influence of the unaffected disease type.

[0020] One or more technical solutions provided in this application have at least the following technical effects or advantages: by calculating the comprehensive risk coefficient, the degree of patient risk is quantified, providing intuitive decision support for medical staff; by comparing and analyzing the standard set, habitual feature set, and prediction set, the accuracy of judgment is improved; the calculation of the comprehensive risk coefficient enables medical staff to understand the degree of patient risk more quickly and take corresponding nursing measures; the prediction set is generated according to the patient's specific situation and disease characteristics, making the early warning more personalized and improving the effectiveness of the early warning; for multiple concurrent diseases, the final predicted disease type is determined through two-level progressive identification, accurately judging the patient's current risk status, issuing timely early warnings to medical staff, and continuously optimizing the system to improve its accuracy and reliability;

[0021] A personalized compensatory mapping table is generated for each patient, which improves the accuracy of compensatory behavior recognition. By combining facial expressions, actions and physiological data, a three-level correlation decision is made through a collaborative decision-making layer, which improves the accuracy and reliability of early warning. Based on the confidence level calculation results, dynamic early warning classification is realized, providing medical staff with more precise treatment suggestions and enabling precise monitoring of special populations. Through multimodal compensatory behavior collaborative early warning, the accuracy and timeliness of disease warning are improved.

[0022] By dynamically updating the standard set, habitual feature set, and prediction set using a disease type matrix, the limitations of a single disease type model are avoided, the sensitivity of early warning is improved, and a dynamic decision-making framework for collaborative management of multiple disease types is constructed, reducing the false alarm rate. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating an abnormal identification method for ICU patients based on video surveillance according to the present invention.

[0024] Figure 2 This is a schematic diagram of the process for generating the compensation mapping table in this invention;

[0025] Figure 3 This is a flowchart illustrating the process of constructing a disease type matrix for this invention. Detailed Implementation

[0026] To facilitate understanding of the present invention, a more complete description of this application will be given below with reference to the accompanying drawings, which illustrate preferred embodiments of the invention. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to enable a more thorough and complete understanding of the disclosure of the present invention.

[0027] It should be noted that the terms "vertical," "horizontal," "up," "down," "left," "right," and similar expressions used in this article are for illustrative purposes only and do not represent the only possible implementation.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention; the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0029] Example 1: Figure 1 This is a flowchart illustrating an abnormal identification method for ICU patients based on video surveillance according to an embodiment of the present invention, including:

[0030] S101, Collect historical data, establish a standard set and a habitual feature set based on the collected historical data, and generate a prediction set based on the standard set and the habitual feature set;

[0031] Specifically, the procedure involves collecting facial expressions, movements, and physiological data from patients since admission. Facial expression data is collected using video surveillance equipment, with parameters adjusted to clearly capture facial expressions. A Facial Action Coding System (FACS) is used, employing the patient's facial features as reference points to identify expressions by recognizing their position and changes. The timing and frequency of various facial expressions are recorded to create an facial expression dataset. Similarly, FACS is used to capture movements of the patient's fingers, arms, and other body parts. Video analysis algorithms are used to identify these movements, recording their timing and frequency to create a movement dataset. Finally, physiological data is recorded using a physiological measuring instrument. The collected facial expression, movement, and physiological data are cleaned, aligned according to timestamps, and standardized. Data such as heart rate and blood pressure are converted into uniform numerical ranges. Facial expression and movement data are encoded, categorizing expressions as happy, sad, and angry, and movements as waving, raising hands, and nodding. Based on medical knowledge and clinical experience, standard ranges for physiological data are determined. For data from the same time point, if the physiological data falls within the standard range, the facial expression and movement data for that time point are retained. The selected facial expression, movement, and physiological data are integrated to form a standard set, which includes various facial expressions and movements of the patient in a normal state and their corresponding physiological data.

[0032] The habit feature set includes normal habits and disease habits. Normal habits refer to the most frequent facial expressions and movements unrelated to the disease since the patient's admission. Disease habits refer to facial expressions and movements that occur after the patient becomes ill, provided that the corresponding physiological data is normal and no abnormalities are found upon questioning. The frequency of each facial expression and movement in the dataset is statistically analyzed. Based on the size and diversity of the dataset, a threshold is set to distinguish between frequently occurring facial expressions and movements and occasional abnormal behaviors. Facial expressions and movements with a frequency greater than the threshold are filtered out as normal habits, while those known to be directly related to the disease state are excluded. Facial expressions and movements that occur after the patient becomes ill, but whose corresponding physiological data is normal and no abnormalities are found upon questioning, are identified as disease habits. These facial expressions and movements are recorded in detail, including their time of occurrence, frequency, and corresponding physiological data. Through communication with the patient, it is confirmed that these facial expressions and movements are indeed normal. Abnormal reactions related to the patient's disease state but not directly caused by the disease, such as: Patient A has lung cancer, takes 25 deep breaths every afternoon, maintains a heart rate of 70-80 beats / minute, and reports difficulty breathing but no chest pain. Computed tomography (CT) scan shows decreased lung function, which is determined to be a deep breathing habit caused by dyspnea, and is included in the disease habit database; The data of normal habits and disease habits are integrated to form a dataset containing all the patient's habitual actions and expressions, recording the frequency of each habitual action or expression since the patient's admission. The frequency can be expressed as a frequency index. For disease habits, in addition to recording their frequency, their correlation score or level with the disease state is also recorded. Based on the integrated data and the recorded frequency information, a habit feature set of the patient is formed. The habit feature set contains a detailed description of all the patient's habitual actions and expressions, their frequency of occurrence, and their correlation information with the disease state.

[0033] Based on the patient's diagnosis, disease characteristics are analyzed to identify the disease type, the patient's past medical history is compiled, and the patterns and symptoms of disease progression are identified. For each disease type, pathological data is extracted. A mapping relationship between symptoms and behaviors is established based on the patient's past history, for example: chest pain → clutching the chest, shortness of breath → mouth breathing, etc. The range of changes in patient indicators is extracted from the analysis of disease characteristics (from physiological changes to pathological changes). For example, for heart failure patients, physiological indicators such as normal heart rate and blood pressure ranges are recorded, as well as pathological indicators such as the thresholds for changes in heart rate and blood pressure when the condition worsens (e.g., heart rate exceeding 120 beats / minute, systolic blood pressure below 90 mmHg). To assess the severity of the condition, a time-series analysis model (ARIMA model, or Autoregressive Integrated Moving Average model) was used to predict abnormal thresholds for pathological indicators. Specifically, based on time-series data of patients' historical physiological indicators (such as heart rate and blood pressure), the stationarity of the series was verified using the Augmented Dickey-Fuller test (ADF) and differencing was performed (if non-stationary) to determine the parameters (p, d, q) of the ARIMA model. Next, the AIC criterion was used to optimize the model and fit the data to predict future trends in the indicators. Then, combining clinically defined pathological thresholds (such as heart rate >120 beats / minute for heart failure patients) and the confidence intervals of the predicted values, the degree of abnormality was classified: mild abnormality was defined as a predicted value exceeding the threshold but within the lower limit of the 95% confidence interval; moderate abnormality was defined as a value exceeding the lower limit but not reaching twice the standard deviation; and severe abnormality was defined as a value exceeding twice the standard deviation. Finally, the prediction results were compared with real-time patient data, and the model parameters were dynamically adjusted to improve the accuracy of the early warning. For example, modeling heart rate data from continuous monitoring of heart failure patients can predict the probability and severity of heart rate exceeding warning levels in the next 6 hours. Pathological abnormalities can be categorized into mild, moderate, and severe levels based on the abnormality threshold. Normal facial expressions and movements in a standard set can be compared with disease characteristics. Physiological data within the range of patient indicator changes can be used to locate physiological data within the standard set. Then, facial expressions and movements within the standard set can be located based on the physiological data. This allows for the determination of whether facial expressions and movements will change when the disease worsens. For instance, when heart failure patients experience symptoms such as shortness of breath and chest pain, they may find it difficult to maintain a smile, instead frowning. Comparing the smile with the expressions that appear during disease exacerbation and recording the matching results allows for the determination of whether the movement will change during disease exacerbation.When a heart failure patient's condition worsens, they may experience difficulty breathing, weakness, and other symptoms that prevent them from walking normally and naturally. This can manifest as slow walking, unsteady gait, and the need to hold onto walls for support. The system compares the patient's natural walking with the movements observed during disease exacerbation and records the matching results. Behaviors unrelated to disease characteristics are eliminated from the habitual feature set. A Bayesian network algorithm is used to calculate the probability of specific behaviors occurring during disease exacerbation. The matching results of facial expressions and movements are used as input to the prediction model Extreme Gradient Boosting (XGBoost). After processing, XGBoost outputs a data structure containing a range of facial expressions, movements, and physiological data, known as the prediction set. This prediction set is used to provide early warnings of abnormal situations in patients, improving response speed.

[0034] S102, collect real-time data, compare the real-time data with the standard set, habitual feature set and prediction set to obtain the data anomaly coefficient, calculate the comprehensive risk coefficient based on the data anomaly coefficient, and trigger an early warning based on the comprehensive risk coefficient;

[0035] Furthermore, cameras and physiological monitoring instruments are used to collect patients' physiological expressions and movements in real time. The collected data is compared with a standard set. Image recognition and motion capture technologies are used to analyze the real-time collected patient expressions and movements, and the analysis results are compared with normal expressions and movements in the standard set to determine if any abnormalities have occurred. For example, facial recognition algorithms are used to detect patients' facial expressions to determine if there are abnormal expressions such as pain or anxiety; motion capture algorithms are used to analyze patients' body movements to determine if there are abnormal movements such as trembling or convulsions. The real-time collected physiological data is compared with the normal physiological data range in the standard set. If a certain physiological indicator exceeds the normal range... If the data is not properly matched, it is considered abnormal. For example, if a patient's heart rate exceeds the upper limit of the standard set or their blood pressure is below the lower limit of the standard set, it is considered abnormal physiological data. The real-time collected data is compared with the habit feature set. Features of the real-time collected patient expressions and movements are extracted and matched with habitual expressions and movements in the habit feature set. If the match is successful, it is judged as a habitual movement; if the match fails, it is a movement caused by disease. For example, if a patient habitually scratches their head, when the scratching movement is captured in real time, it is judged as a habitual movement by comparing it with the habit feature set. Finally, the real-time collected expressions and movements are compared with disease-related expressions and movements in the prediction set. If a match is successful, real-time physiological data is used to determine whether the patient is experiencing an attack. For example, the prediction set specifies that patients with coronary heart disease will clutch their chest when an attack occurs. When the patient's chest-clutching action is captured in real time, and physiological data such as heart rate and blood pressure are also abnormal, the patient is judged to be experiencing an attack. During the monitoring process, the frequency of habitual actions or expressions is counted in real time. By setting time windows, the number of times habitual actions or expressions occur within a specific time window is counted. The real-time frequency is compared with the normal frequency. The normal frequency can be set based on the patient's past habit feature set data. If the real-time frequency exceeds the normal frequency range, it is judged as an abnormal frequency. For example, the normal frequency of a patient scratching their head is 2-3 times per hour. If the number of times the patient scratches their head reaches 5 times within a 10-minute window, it is judged as an abnormal frequency.

[0036] Based on the deviation of real-time physiological data from the normal range, an abnormality coefficient for physiological data is set; the greater the deviation, the higher the abnormality coefficient. Based on the comparison of real-time facial expressions and movements with the standard set and habitual feature set, an abnormality coefficient for facial expressions and movements is set; the greater the abnormality, the higher the abnormality coefficient. Based on the comparison of the real-time frequency of habitual movements or expressions with the normal frequency, a frequency abnormality coefficient is set; the greater the frequency abnormality, the higher the abnormality coefficient. A comprehensive risk factor is calculated based on the abnormality coefficients of physiological data, facial expressions and movements, and frequency abnormalities, using the following formula: ,in, This represents the overall risk factor. Indicates an index of abnormal physiological data. Indicates an abnormality index of facial expressions and movements. Indicates the frequency anomaly index. , , Let be the weights of each index, and + + =1. When the calculated comprehensive risk coefficient exceeds the preset threshold, an early warning is issued. The warning information includes the patient's real-time facial expressions, movements, physiological data, and comprehensive risk coefficient.

[0037] S103 collects several types of patient conditions in real time and selects one type of condition through a two-level identification method;

[0038] Specifically, the system obtains the patient's condition type in real time from the hospital's information system. Based on the clinical characteristics of common condition types, it calls the corresponding prediction set, which includes physiological indicator thresholds and behavioral feature attention areas. Conflict signal detection rules for each condition type are extracted from the prediction set. For example, for cardiovascular diseases, the prediction set defines a conflict signal as the simultaneous occurrence of a blood glucose level below 4 mmol / L, ST segment depression, and a sudden increase in hand movement amplitude to 3-5 cm. The real-time data is compared with the conflict signal detection rules one by one, and the data at the current moment is checked at set time intervals to see if there are signal combinations that conform to the rules. If a signal combination in the real-time data is detected at a certain moment that completely conforms to the conflict signal rules defined in the prediction set, it is determined that a conflict signal has been detected. Once a conflict signal is detected, the system immediately marks a conflict flag in the current patient data record. The conflict flag indicates that a monitoring conflict exists, thereby initiating the condition type identification process.

[0039] The two-level identification method consists of physiological indicator identification and behavioral feature identification. For physiological indicator identification, the risk coefficient for each disease type is calculated based on the patient's real-time physiological data. For example, the risk coefficient for diabetes is calculated as (standard blood glucose value - real-time blood glucose value) × weighting coefficient, and the risk coefficient for coronary heart disease is calculated as ST segment depression amplitude × weighting coefficient. The calculated risk coefficients for each disease type are compared pairwise, and the disease type with the highest risk coefficient is selected for behavioral feature identification. Image processing and sensor data analysis techniques are used to extract conflicting action features from the real-time collected action data. According to pre-set conflicting action rules, it is determined whether the extracted action is a conflicting action. For example, for diabetic patients, abnormal hand tremors are identified as conflicting actions; for coronary heart disease patients, abnormal chest compressions may be considered conflicting actions. A three-dimensional spatial coordinate system is established, and the trajectory of the conflicting actions is mapped onto this coordinate system. To understand the spatial distribution and movement patterns of movements, and based on the characteristics and clinical manifestations of different disease types, corresponding attention zones are defined. For diabetes, the attention zone is the area of ​​horizontal hand tremors; for coronary heart disease, the attention zone is the area of ​​circular chest compressions. The three-dimensional spatial analysis results of conflicting movements are matched with the attention zones of each disease type to calculate the matching degree. For example, for hand tremors, the proportion of their trajectory in the diabetes attention zone (horizontal hand tremors area) is calculated; for chest compressions, the proportion in the coronary heart disease attention zone (circular chest compressions area) is calculated. The matching degree is calculated using the overlap area calculation method, which is an existing technology and will not be elaborated here. Based on clinical experience and statistical analysis, a comprehensive matching degree threshold is set for each disease type. When the comprehensive matching degree of a disease type exceeds this threshold, the probability of that disease type is considered to be the highest, i.e., the disease type is identified, and an early warning is issued based on the identified disease type.

[0040] A specific example is as follows: Patient Li, a 65-year-old male, has a history of diabetes and coronary heart disease and is currently hospitalized in the Department of Cardiology. The hospital uses a multi-source data fusion and dynamic early warning scheme to monitor and provide early warnings for Li's condition in real time. The hospital's information department connects the Hospital Information System (HIS), Electronic Medical Record System (EMR), and monitoring system, and also connects to the data interfaces of monitoring devices such as electrocardiogram monitors and blood glucose meters. Through the HIS system, the monitoring system can obtain Li's basic information and admission diagnosis; from the EMR system, it can obtain Li's past medical history and treatment records; the electrocardiogram monitor transmits Li's electrocardiogram data in real time, including ST segment changes; the blood glucose meter measures and uploads Li's blood glucose values ​​at regular intervals. Medical experts have configured corresponding prediction sets for Li based on the clinical characteristics of diabetes and coronary heart disease. For diabetes, the standard blood glucose value is set at 7 mmol / L, and a real-time blood glucose value below 4 mmol / L is considered abnormal; for coronary heart disease, the ST segment depression amplitude exceeding 0.1 mV is set. This was considered an abnormal signal. Simultaneously, behavioral characteristics of interest were identified: horizontal hand tremors for diabetes and circular chest compressions for coronary heart disease. The monitoring system received a real-time list of Li's diagnosed conditions, confirming he had diabetes and coronary heart disease, and retrieved the prediction sets corresponding to these two conditions. At 10:00 AM on a certain day, the monitoring system simultaneously acquired the following data: the blood glucose meter showed Li's real-time blood glucose level was 3.8 mmol / L, lower than the 4 mmol / L threshold set in the prediction set; the ECG monitor showed Li's ST segment depression amplitude was 0.12 mV, exceeding the 0.1 mV threshold; and simultaneously, the camera installed in the ward captured a sudden increase in Li's hand movement amplitude to 4 cm. When the system detects this specific signal combination simultaneously, it immediately marks a conflict, triggers an early warning mechanism, identifies physiological indicators, and the monitoring system obtains Li's real-time physiological data from the electrocardiogram monitor and blood glucose meter. At this time, the blood glucose level is 3.8 mmol / L, and the ST segment depression amplitude is 0.12 mV. According to the formula in the prediction set, for diabetes, the risk coefficient = (standard blood glucose value - real-time blood glucose value) × 0.2 = (7 - 3.8) × 0.2 = 0.64; for coronary heart disease, the risk coefficient = ST segment depression amplitude × 1.5 = 0.12 × 1.5 = 0.18. Comparing the risk coefficients of the two conditions, 0.64 > 0.18, diabetes is selected to proceed to the next level of behavioral feature recognition. Using the camera installed in the ward to capture Li's motion data, it was found that Li's hand was involuntary horizontal shaking. The hand shaking motion of Li was deconstructed in three dimensions, and analyzed from the dimensions of motion trajectory, force analysis, and spatial coordinates.Analysis revealed that 80% of the hand tremor trajectories occurred in the horizontal hand tremor area (diabetes focus area), while only 20% occurred in the chest circular compression area (coronary artery disease focus area). Based on the matching results, the high degree of matching between hand tremor movements and the diabetes focus area helped confirm the final diagnosis of diabetes. The monitoring system issued a diabetes warning, notifying the on-duty doctor and nurse. Based on the warning information and treatment guidelines from the system's evidence-based medicine knowledge base, the doctor immediately conducted further examinations and assessments for Mr. Li, adjusted the treatment plan, and administered appropriate medication and blood glucose monitoring to prevent the condition from worsening. Through multi-source data fusion and a dynamic warning system, changes in the patient's condition were detected promptly, providing strong support for clinical treatment.

[0041] The technical solutions in the above-described embodiments of this application have at least the following technical effects or advantages: by calculating the comprehensive risk coefficient, the degree of patient risk is quantified, providing intuitive decision support for medical staff; by comparing and analyzing the standard set, habitual feature set, and prediction set, the accuracy of judgment is improved; the calculation of the comprehensive risk coefficient enables medical staff to understand the degree of patient risk more quickly and take corresponding nursing measures; the prediction set is generated according to the patient's specific situation and disease characteristics, making the early warning more personalized and improving the effectiveness of the early warning; for multiple concurrent diseases, the final predicted disease type is determined through two-level progressive identification, accurately judging the patient's current risk status, issuing timely early warnings to medical staff, and continuously optimizing the system to improve its accuracy and reliability.

[0042] Example 2: Based on the facial warning and disease complication identification in Example 1, this example generates a set of compensatory actions according to the patient's individual circumstances when facial expressions are unclear or unable to provide clear expressions. This avoids confusion with habitual feature sets and establishes a distinguishing standard, such as... Figure 2 As shown.

[0043] S201: Obtain data on patients with expression loss from the standard set and habitual feature set, construct a database, and generate a compensation mapping table based on the database;

[0044] Furthermore, patients with facial nerve palsy, post-laryngeal cancer surgery, and other facial expression deficiencies were extracted from the standard set and habitual feature set. The extracted data were cleaned, and a database was constructed based on the patients' pathological types, missing facial expressions, and changes in physiological indicators. A time window (e.g., 5 minutes before and after the occurrence of abnormal physiological indicators) was set to capture high-frequency accompanying actions within this time period. Within the time window, the frequency of each action was statistically analyzed, and the Pearson correlation coefficient was used to identify the correlation between action frequency and abnormal pathophysiological indicators, determining that actions occur frequently when indicators are abnormal and serve as compensatory behaviors. The specific method is as follows: Within a 5-minute time window before and after the occurrence of abnormal pathophysiological indicators, the frequency of various actions (such as pressing the temples, blinking, etc.) of the patient is counted, and the degree of abnormality of key physiological indicators (such as heart rate variability and blood oxygen saturation) within the corresponding time window is recorded. Then, the Pearson correlation coefficient r between the frequency of each action and the abnormal value of the physiological indicator is calculated, and high-frequency actions with significant correlation (|r|>0.5 and p<0.05) are screened out. Then, the compensatory characteristics of these actions are verified in combination with clinical knowledge—for example, a patient who has undergone laryngeal cancer surgery frequently uses gestures to point to the painful area because he is unable to frown (r=0.05). 62), or facial nerve palsy patients who have difficulty controlling the corner of their mouth and nod their heads (r=0.58), ultimately these actions that occur frequently during periods of abnormal indicators and are significantly related to the pathological state are identified as compensatory behaviors. A machine learning model, Support Vector Machine (SVM), is used to identify these high-frequency accompanying actions. The specific steps are: based on the Pearson correlation coefficient, the frequency data of actions significantly related to pathophysiological indicators are selected, features are extracted (such as the frequency of action occurrence, duration, and temporal relationship with abnormal indicators), and they are labeled as compensatory behaviors (positive samples are confirmed compensatory actions, negative samples are irrelevant actions); then, Historical data is divided into training and testing sets. The training set is used to optimize the support vector machine (SVM) model using a kernel function (such as RBF) to maximize the classification margin to distinguish between compensatory behavior and normal actions. Then, the test set is used to evaluate the model performance (such as accuracy and recall), and hyperparameters are adjusted through cross-validation. Finally, newly collected patient action frequency data is input into the trained SVM model, and the output probability score (e.g., >0.7 is considered compensatory behavior) is used. For example, frequent throat touching (frequency 25 times / 5 minutes) in post-laryngeal cancer patients is classified as compensatory behavior for dyspnea (probability 0).83), thus supplementing the compensation mapping table, using historical data to train the machine learning model, dividing the historical data into training set and test set, the training set is used for model training, and the test set is used to evaluate the model performance, inputting the data in the database into the trained machine learning model support vector machine, the model outputs whether each action is a compensatory behavior, the compensatory behavior refers to the behavior model used to replace or compensate for the lack of expression for patients with unclear facial expressions or unable to provide clear expressions, the compensatory behavior includes action type, frequency of occurrence and duration, according to patient ID, compensatory behavior and corresponding missing expression records to form a compensatory behavior library, generate a compensation mapping table according to the compensatory behavior library, group the original data in the compensatory behavior library by patient ID, divide it into multiple subsets, each subset corresponds to a patient's historical behavior record, for example: patient A's data includes records of pressing the temple with fingers when frowning is missing, and raising the corners of the mouth when smiling is missing; patient B's data includes increased blinking frequency when closing the eyes is missing, through the facial coding system (FACS) to identify the expressions that the patient cannot complete, according to the patient's grouped subsets. The alternative actions for patients were selected through a screening process. Specifically, based on the anatomical coding rules of the Facial Coding System (FACS) (e.g., AU4 represents corrugator supercilii activity), action units (AUs) were detected for the target facial expressions (e.g., a smile corresponds to AU12+AU6). If the activation intensity of a key AU remained below a threshold (e.g., AU12 < 0.5 for 3 seconds), it was determined to be a missing expression. Next, action data occurring during the same period as the missing expression were extracted from a subset of the patient's historical behaviors (e.g., the frequency of the "corner of the mouth lifting" action recorded when patient A's smile was missing increased to 15). (Times / minute) Combined with verified mapping relationships in the compensatory behavior database (such as the functional similarity between "lifting the corner of the mouth" and AU12), actions with compensatory characteristics are screened out; finally, a structured record is generated. For example, a patient with facial nerve palsy, due to AU4 failure (inability to frown), frequently performs "pressing the temple with fingers" (frequency 20 times / 5 minutes). This action is included in the compensatory mapping table as a substitute behavior for pain expression. A compensatory mapping table is generated for the facial expressions and substitute actions that the patient cannot perform. The compensatory mapping table includes fields for missing facial expressions, compensatory actions, action descriptions, and functional descriptions.

[0045] S202, match the real-time detected action data with the compensation mapping table, calculate the confidence level, and generate early warning decisions based on the confidence level;

[0046] Specifically, the camera is used to detect motion data in real time, and the detected data is matched with a compensation mapping table. The formula for calculating the dynamic time-warped distance is as follows: ,,in, For dynamic time curvature, The detected action time series is denoted as A= ,in The motion feature vector (such as joint angle and acceleration) at the i-th time point; The preset reference action time series in the compensation mapping table is represented as B= ,in Let j be the action feature vector at time point j. Representing Euclidean distance, calculating real-time action features Compared with reference action characteristics The difference at the same point in time, P, is the set of all regularized paths that satisfy the boundary conditions, monotonicity, and continuity. The formula for converting the dynamic time regularized distance into action matching degree is: ,in, This is the dynamic time-normalized distance between the current real-time action and the reference action. The maximum dynamic time-normalized distance obtained by comparing all actions in historical data is used to calculate the patient's physiological deviation. The formula is as follows: The comprehensive risk factor is calculated in step S102, and the confidence level is calculated based on the degree of action matching and the degree of physiological deviation, using the following formula: .in, The weights for action matching degree, As a weight for physiological offset, in this embodiment =0.6. The core goal of compensatory behavior is to replace the function of missing facial expressions; therefore, the accuracy of the action execution is the primary consideration. =0.4, physiological abnormalities directly affect the effectiveness of compensatory movements (such as pain causing movement deformation), and should be considered as an important correction factor.

[0047] Based on the calculated confidence level, a threshold range is set, including a maximum threshold and a minimum threshold. When the confidence level is less than the minimum threshold, it is considered low confidence, and no warning is triggered. Low confidence indicates insufficient matching between real-time actions and the compensation mapping table, and no significant abnormality in physiological deviation, suggesting unconscious habitual behavior. When the confidence level is greater than or equal to the minimum threshold and less than or equal to the maximum threshold, it is considered medium confidence, initiating multi-condition identification to avoid interference from other diseases. When the confidence level is greater than the maximum threshold, a compensation warning is triggered and pushed to the doctor's terminal. If the physiological deviation is > 80% (e.g., heart rate > 120 bpm, blood oxygen < 90%), it is upgraded to an emergency warning. High confidence indicates that the actions highly match the compensation mapping table, and the physiological state is significantly abnormal, indicating failure of compensation behavior or deterioration of the patient's health. Dynamic warning levels are assigned based on the confidence level: high confidence is a Level 1 warning (confidence > 0.85, red alert); medium confidence is a Level 2 warning (0.7 < 0.85, red alert); and low confidence is a Level 2 warning (confidence > 0.85, red alert). A confidence level ≤ 0.85 indicates an orange alert; a low confidence level indicates a level 3 warning; and a confidence level ≤ 0.4 indicates a green alert. Warning information will be sent to medical staff via SMS, email, APP push notifications, etc. The notification content includes the warning level, patient information, description of compensatory behavior, and changes in physiological indicators.

[0048] The technical solutions in the above-described embodiments of this application have at least the following technical effects or advantages: generating a personalized compensatory mapping table for each patient improves the accuracy of compensatory behavior recognition; combining facial expressions, actions, and physiological data, and conducting three-level correlation decision-making through a collaborative decision-making layer improves the accuracy and reliability of early warning; realizing dynamic early warning grading based on confidence calculation results provides medical staff with more precise treatment suggestions; achieving precise monitoring of special populations (patients with missing facial expressions); and improving the accuracy and timeliness of disease early warning through multimodal compensatory behavior collaborative early warning.

[0049] Example 3: Based on Examples 1 and 2, update the standard set, habitual feature set, and prediction set in Example 1 according to their mutual influence relationships, such as... Figure 3 As shown.

[0050] S301, construct a disease type matrix based on the type of disease;

[0051] Specifically, based on epidemiological data from the NHANES database, combinations with a comorbidity rate >20% are screened, i.e., related disease types associated with the primary disease type are identified. The influence coefficient is then calculated based on the primary disease type and related disease types, using the following formula: ,in, The influence coefficient represents the weighted impact of changes in the main disease type indicator on the risk of related disease types. The percentage increase in risk associated with a specific disease type indicates the proportion of risk increase for that disease type resulting from a change in the main disease type indicator (e.g., a 1 mmol / L increase in blood glucose → a 12% increase in the risk of coronary heart disease). The changes in indicators related to the main disease type. The baseline risk represents the basic risk of associated disease types without the influence of the primary disease type. It records clinical manifestations, the symptoms of associated disease types caused by changes in the primary disease type indicators, and marks the indicator thresholds and risk ratios. It also records the time difference between changes in the primary disease type indicators and the appearance of associated disease type symptoms, reflecting the pathophysiological transmission process. Based on the data recorded above, the system outputs the constructed disease type matrix, which includes the primary disease type, associated disease type, influence coefficient, clinical manifestations, and effect delay fields.

[0052] S302, update the standard set, habitual feature set, and prediction set based on the disease type matrix;

[0053] For updating the standard set, the system continuously monitors the patient's primary disease category indicators and updates the standard set in real time. For updating the habit feature set, a risk coefficient is calculated based on the patient's age, blood lipid level, and medical history. The formula is: Risk Coefficient = Age Weight × Age Coefficient + Blood Lipid Weight × LDL Coefficient + Medical History Weight × Medical History Coefficient, where LDL coefficient is low-density lipoprotein cholesterol, the core substance for the formation of atherosclerotic plaques. High-risk behaviors are marked based on the calculated risk coefficient, and these high-risk behaviors are transferred from normal habits to disease habits, which is the process of updating the habit feature set. For updating the prediction set, the associated disease category features related to the primary disease category are input into the prediction model. In Limited Gradient Boosting (XGBoost), the prediction model is updated as follows: First, based on the patient's current primary condition (e.g., coronary artery disease), physiological indicators (blood glucose fluctuations, ST segment changes) and behavioral characteristics (hand tremor patterns) of related conditions (e.g., diabetes) are extracted. Then, these features are combined with primary condition features (e.g., QTc interval, chest pain compensatory movements) to construct a feature matrix, where the related condition features are weighted (e.g., weight of diabetes-related indicators = 0.3 × coronary artery disease risk coefficient) to reflect their contribution. Next, new data is input into the pre-trained Limited Gradient Boosting (XGBoost) model using incremental learning, adjusting the tree structure (e.g., adding split nodes). The prediction logic is optimized for cases with blood glucose > 11 mmol / L and ST depression ≥ 0.1 mV. The final output is an updated prediction set. For example, when a diabetic patient develops a new coronary heart disease associated feature, the model enhances its warning sensitivity to "hand tremors accompanied by chest compressions" (F1-score increased by 12%) and dynamically expands the threshold range of pathological indicators (e.g., the heart rate warning threshold is reduced by 10% when blood glucose > 9.4 mmol / L). The updated prediction set is output by the model. For patients with multiple coexisting conditions, physiological indicators are identified based on diabetes and coronary heart disease, and independent risk coefficients for diabetes and coronary heart disease are calculated. The diabetes risk coefficient is based on blood glucose level, fluctuation rate, and duration. The calculated risk coefficient for coronary artery disease (CAD) is based on the ST segment depression amplitude and QTc interval deviation. The disease type with the higher risk coefficient is selected as the initial diagnostic direction. If the initial diagnosis is diabetes, its indirect impact on CAD is assessed; if the initial diagnosis is CAD, its interference with diabetes management is assessed. Behavioral characteristics are used for auxiliary verification. Diabetes-related behaviors include hand tremors concentrated in the horizontal plane of the hand (e.g., eating movements), while CAD-related behaviors include hand tremors accompanied by chest compressions (e.g., angina compensatory behavior). If the behavioral characteristics are inconsistent with the initial diagnosis (e.g., high-risk diabetes but hand tremors accompanied by chest compressions), a verification process is triggered, and an alert is issued based on the identified disease type.

[0054] The technical solutions in the above embodiments of this application have at least the following technical effects or advantages: by dynamically updating the standard set, habitual feature set and prediction set through the disease type matrix, the limitations of a single disease type model are avoided, the sensitivity of early warning is improved, a dynamic decision-making framework for collaborative management of multiple disease types is constructed, and the false alarm rate is reduced.

[0055] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying ICU patient abnormalities based on video monitoring, characterized in that, The method comprises the following steps of: S101, collecting historical data, establishing a standard set and a habit feature set according to the collected historical data, and generating a prediction set based on the standard set and the habit feature set; The standard set is the expression, action and corresponding physiological data of the patient in a normal state, the habit feature set includes normal habits and disease habits, the normal habits refer to the habitual expressions and actions of the patient that are unrelated to diseases, the disease habits refer to the habitual expressions and actions of the patient caused by diseases, and the prediction set includes a physiological index threshold and a behavior feature attention area; S102, collecting real-time data, comparing the real-time data with the standard set, the habit feature set and the prediction set to obtain an abnormal coefficient of the data, calculating a comprehensive risk coefficient according to the abnormal coefficient of the data, and triggering an early warning based on the comprehensive risk coefficient; The real-time collected physiological data is compared with the normal physiological data range in the standard set, and a physiological data anomaly index is set according to the deviation degree; the real-time collected expression and action data is compared with the normal expression and action in the standard set, and is matched with the habitual expression and action in the habitual feature set, and an expression and action anomaly index is set according to the anomaly degree; the appearance frequency of the real-time collected habitual expression and action is counted, and is compared with the normal frequency in the habitual feature set, and a frequency anomaly index is set according to the frequency anomaly degree; and a formula for calculating the comprehensive risk coefficient is: , , , , , , , , , , , =1. S103, collecting several types of patient conditions in real time, and screening one type of condition through a two-level recognition method; The two-level recognition method includes physiological index recognition and behavior feature recognition, calculating a risk coefficient of each type of condition according to real-time physiological data of the patient, selecting the type of condition with the highest risk coefficient to enter the behavior feature recognition, and performing behavior feature recognition according to the high-risk condition screened out, the behavior feature recognition is mapping a conflict action trajectory through a three-dimensional space coordinate system, the conflict action is an action related to the high-risk condition of the patient extracted from the action data collected in real time, and the overlap area ratio of the trajectory and a preset attention area of each type of condition is calculated as a matching degree, and when the matching degree of the type of condition exceeds a threshold value, the type of condition is recognized.

2. The ICU patient abnormality recognition method based on video monitoring according to claim 1, wherein S201, when there is an expression-lacking patient in the standard set and the habit feature set, obtaining data of the expression-lacking patient from the standard set and the habit feature set, constructing a database according to a pathological type of the patient, a lacking expression and a physiological index change, recording a patient ID, a compensatory behavior and a corresponding lacking expression as a compensatory behavior library, generating a compensatory mapping table according to the compensatory behavior library, the compensatory mapping table including a lacking expression, a compensatory action, an action description and a function description field, the compensatory behavior refers to a behavior model for replacing or compensating for expression lack for a patient whose facial expression is unclear or cannot provide a clear expression, and the compensatory behavior includes an action type, an occurrence frequency and a duration; The expression-lacking patient refers to a patient whose facial expression is unclear or cannot provide a clear expression; S202, matching real-time detected action data with the compensatory mapping table, calculating an action matching degree and a confidence, and generating an early warning decision according to the confidence. 3.The video monitoring based ICU patient abnormality identification method of claim 2, wherein, The formula for calculating the action matching degree is: wherein, is the dynamic time warping, is the detected action time sequence, is the action feature vector at the i-th time point; is the preset reference action time sequence in the compensation mapping table is the action feature vector at the j-th time point, represents the Euclidean distance, P is a set of all warping paths satisfying the boundary condition, monotonicity and continuity, and the formula for converting the dynamic time warping distance into the action matching degree is: wherein, is the dynamic time warping distance between the current real-time action and the reference action, is the maximum dynamic time warping distance obtained by comparing all actions in the historical data.

4. The method of claim 2, wherein the method further comprises: The formula for calculating the confidence is: wherein, is the weight of the action match degree, is the weight of the physiological shift degree.

5. The method of claim 2, wherein the method further comprises: The generated early warning decision is: setting a threshold range according to the calculated confidence, the threshold range including a highest threshold value and a lowest threshold value, when the confidence is less than the lowest threshold value, the confidence is low, and no early warning is triggered, when the confidence is greater than or equal to the lowest threshold value and less than or equal to the highest threshold value, the confidence is medium, and multi-condition type recognition is started, and when the confidence is greater than the highest threshold value, a compensatory early warning is triggered and pushed to a doctor terminal. 6.The ICU patient anomaly identification method based on video monitoring according to claim 1, characterized in that, S301, constructing a disease category matrix according to the disease category; S302, updating the standard set, the habit feature set and the prediction set according to the disease category matrix.

7. The method of claim 1, wherein the method further comprises: determining a patient abnormality based on the video monitoring. Based on the disease category recognition associated disease category, according to the disease category and associated disease category calculation influence coefficient formula is: Wherein, The influence coefficient, indicates the weighted influence of the main disease category index change on the associated disease category risk, The associated disease category risk increment, The main disease category index change, The baseline risk, indicates the basic risk of the associated disease category without the influence of the main disease category. ​

Images