A comprehensive visual Internet of Things management system

Abstract

This application relates to the technical field of medical information technology and discloses a comprehensive visual Internet of Things management system, including a data acquisition module, a processing module, a risk identification module, and an adjustment module; the data acquisition module is used to collect bed data; the bed data includes monitoring images and pressure sensor data of the bed; the processing module extracts visual features based on the bed data; the visual features include visual skeleton sequences and pressure thermograms; the risk identification module identifies the distortion risk of monitoring equipment based on the visual features; the identification of the distortion risk of monitoring equipment includes identifying the distortion risk of electrocardiogram electrodes and the distortion risk of blood oxygen probes; the adjustment module performs correction and compensation on the monitoring equipment based on the distortion risk of the monitoring equipment. This application realizes the identification and correction of distortion risks of medical monitoring equipment, and realizes the visualization of medical care status and the automation of monitoring.

Description

Technical Field

[0001] The present application relates to the technical field of medical information technology, and in particular to a comprehensive visual Internet of Things management system. Background Art

[0002] In medical scenarios such as postoperative monitoring, frequent adjustments in patient position can lead to poor contact between ECG electrodes and blood oxygen sensors due to body pressure or displacement, resulting in noise or signal loss. Transient signal attenuation caused by changes in body position triggers false alarms, interfering with medical staff's judgment. When monitoring equipment is distorted, medical staff must manually adjust the equipment's position, increasing their workload. Existing technologies achieve more accurate monitoring by increasing alarm sensitivity or filtering the signal, but they cannot fundamentally address the problem of body position interfering with monitoring equipment.

[0003] Traditional monitoring systems lack real-time data processing and transmission capabilities. Existing risk warning systems only issue alerts after a problem with the monitoring equipment occurs, lacking the ability to proactively predict and prevent risks. For example, if a patient's position changes and an electrode becomes dislodged, the system is unable to predict and take appropriate action. Furthermore, existing monitoring systems often require manual intervention when processing distorted data, resulting in a low level of automation.

[0004] For example, a Chinese patent with authorization announcement number CN115394418B discloses a refined status monitoring system for medical equipment based on the Internet of Things. The status monitoring system includes a server, an interaction module, an equipment status monitoring module, a fault reporting module, a positioning module, and several mobile terminals. Each mobile terminal is used to connect to the server to query information and report faults on medical equipment; the positioning module is used to locate the distribution location of medical equipment, the equipment status monitoring module is used to monitor the operating status of medical equipment, the fault reporting module is used to report faults to medical equipment, and the interaction module is used to interact with maintenance personnel based on the data from the fault reporting module; the invention monitors the usage status of medical equipment through the equipment status monitoring module, and can effectively understand the usage status of medical equipment to improve the refined management level of the entire medical equipment.

[0005] For example, a Chinese patent with authorization announcement number CN118762816B discloses an adaptive remote medical monitoring method and system based on the Internet of Things, including creating patient profiles and performing device allocation and configuration; collecting patients' real-time health data and transmitting it to a cloud server for data preprocessing; performing anomaly detection on the preprocessed data and conducting preliminary analysis of the abnormal data; and assessing the patient's health status and formulating response measures. By creating personalized patient profiles, performing intelligent device allocation and configuration, building an anomaly detection algorithm, and conducting preliminary analysis of abnormal data, this invention ensures that monitoring equipment can accurately meet the patient's personalized monitoring needs. This not only improves the efficiency and accuracy of remote medical monitoring, but also enables timely detection of abnormal changes in patients' health status, providing medical staff with a scientific basis for decision-making.

[0006] The above existing technologies all have the problem raised by this background technology: they are unable to solve the problem of interference of the patient's body position on the monitoring equipment.

[0007] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the application and should not be regarded as an admission or any form of suggestion that the information constitutes the prior art already known to ordinary technicians in this field. Summary of the Invention

[0008] The technical problem to be solved by this application is to overcome the defects of the existing technology and provide a comprehensive visual Internet of Things management system to realize the distortion risk identification and correction compensation of medical monitoring equipment, and realize the visualization of medical care status and automatic monitoring.

[0009] To solve the above technical problems, this application provides the following technical solutions:

[0010] A comprehensive visual Internet of Things management system, including a data acquisition module, a processing module, a risk identification module, and a regulation module; wherein:

[0011] The data acquisition module is used to collect bed data; the bed data includes monitoring images and pressure sensor data of the bed;

[0012] The processing module extracts visual features based on the bed data; the visual features include visual skeleton sequences and pressure heat maps;

[0013] The risk identification module identifies the distortion risk of the monitoring device based on the visualization feature; the identification of the distortion risk of the monitoring device includes identifying the distortion risk of the electrocardiogram electrode and the distortion risk of the blood oxygen sensor;

[0014] The adjustment module performs correction compensation on the monitoring device based on the distortion risk of the monitoring device.

[0015] As a preferred solution of the comprehensive visual Internet of Things management system described in this application, wherein: the data acquisition module includes a monitoring unit and a pressure array unit; wherein the monitoring unit is used to collect monitoring images of the hospital bed; the monitoring images of the hospital bed include a planar image and a depth image of the hospital bed;

[0016] The pressure array unit is used to collect pressure sensing data of the hospital bed; the pressure array unit is configured with a piezoresistive sensor array, and the pressure array unit collects pressure sensing data of each position on the hospital bed in real time based on the piezoresistive sensor array.

[0017] As a preferred solution of the comprehensive visual Internet of Things management system described in this application, the processing module includes an image recognition unit; the image recognition unit extracts a visual skeleton sequence based on the monitoring image of the bed; specifically includes:

[0018] Performing coordinate system registration on the depth image and the plane image to obtain a fusion matrix; the fusion matrix includes the pixel value and depth value of each pixel point in the monitoring image of the bed;

[0019] Inputting the fusion matrix into a joint detection model to obtain the three-dimensional coordinates of each joint of the patient and the confidence level of each joint;

[0020] The image recognition unit is configured with a confidence threshold, and constructs a visual skeleton sequence based on the confidence threshold;

[0021] Each element in the visual skeleton sequence corresponds to a joint point of the patient; wherein, if the confidence of any joint point is greater than or equal to the confidence threshold, the value of the corresponding element in the visual skeleton sequence is the three-dimensional coordinate of the joint point; if the confidence of any joint point is less than the confidence threshold, the value of the corresponding element in the visual skeleton sequence is empty.

[0022] As a preferred solution of the comprehensive visual Internet of Things management system described in this application, the processing module further includes a data processing unit; the data processing unit draws a pressure thermodynamic map based on the pressure sensor data of the hospital bed; specifically includes:

[0023] The pressure sensing data at each position on the bed collected by the piezoresistive sensor array are organized into a pressure sensing matrix; each element in the pressure sensing matrix corresponds to one pressure sensing data, and the position of the element in the pressure sensing matrix is ​​the position of the piezoresistive sensor corresponding to the pressure sensing data in the piezoresistive sensor array; the element values ​​in the pressure sensing matrix are normalized; and the pressure sensing matrix is ​​mapped into a pressure thermodynamic map.

[0024] As a preferred solution of the comprehensive visual Internet of Things management system described in this application, the risk identification module includes a first identification unit; the first identification unit is used to identify the distortion risk of ECG electrodes; the ECG electrodes include a standard lead set and a spare lead set; the identification of the distortion risk of ECG electrodes specifically includes:

[0025] performing spatiotemporal alignment of the pressure heatmap with the visual skeleton sequence;

[0026] Inputting the pressure thermogram and the visual skeleton sequence into a trained distortion prediction model; the distortion prediction model calculates and outputs the distortion risk probability of each lead in the standard lead group;

[0027] The first recognition unit is configured with a distortion probability threshold; if the distortion risk probability of any lead is greater than the distortion probability threshold, the corresponding lead is marked as a risky lead.

[0028] As a preferred solution of the comprehensive visual Internet of Things management system described in this application, the distortion prediction model includes an input layer, a feature extraction layer, a multimodal fusion layer, a prediction layer, and an output layer; wherein:

[0029] The input layer is used to receive the pressure heat map and visual feature skeleton as model input;

[0030] The feature extraction layer includes a pressure feature sublayer and a posture feature sublayer. The pressure feature sublayer is used to process the pressure heat map and output a pressure semantic vector. The posture feature sublayer is used to process the visual feature skeleton and output a posture semantic vector.

[0031] The multimodal fusion layer generates a joint feature vector based on the pressure semantic vector and the posture semantic vector;

[0032] The prediction layer calculates the distortion risk vector of the lead based on the joint feature vector;

[0033] The output layer maps the distortion risk vector of the lead to the distortion risk probability of each lead.

[0034] As a preferred solution of the comprehensive visual Internet of Things management system described in this application, the risk identification module further includes a second identification unit; the second identification unit is configured with a risk identification strategy; the second identification unit identifies the distortion risk of the blood oxygen sensor based on any risk identification strategy;

[0035] The risk identification strategy includes a first identification strategy, which specifically includes: if the values ​​of the elements corresponding to the joint points corresponding to the patient's shoulder joints and hip joints in the visual skeleton sequence are not empty, then the patient's torso pitch angle is calculated based on the three-dimensional coordinates of the joint points corresponding to the patient's shoulder joints and hip joints; the second identification unit is also configured with a pitch angle threshold. If the patient's torso pitch angle is less than the pitch angle threshold, there is a risk of distortion of the blood oxygen probe.

[0036] As a preferred solution of the comprehensive visual Internet of Things management system described in the present application, the risk identification strategy also includes a second identification strategy, which specifically includes: if the value of the element corresponding to the joint points corresponding to the patient's two shoulder joints in the visual skeleton sequence is not empty, then the patient's shoulder height difference is calculated based on the three-dimensional coordinates of the joint points corresponding to the patient's two shoulder joints; the shoulder height difference is the difference between the z-axis coordinate of the joint point corresponding to the shoulder joint on the non-probe side and the z-axis coordinate of the joint point corresponding to the shoulder joint on the blood oxygen probe side; the second identification unit is further configured with a height difference threshold. If the patient's shoulder height difference is greater than the height difference threshold, there is a risk of distortion of the blood oxygen probe;

[0037] The risk identification strategy also includes a third identification strategy, which specifically includes: if the value of the element corresponding to the joint point corresponding to at least one relevant joint on the blood oxygen probe side in the visual skeleton sequence is not empty, then the pressure sensing data of the relevant joint is obtained based on the three-dimensional coordinates of the joint point corresponding to the relevant joint and the pressure heat map; the second identification unit is also configured with a pressure threshold. If the pressure sensing data of at least one relevant joint is greater than the pressure threshold, there is a distortion risk in the blood oxygen probe.

[0038] As a preferred solution of the comprehensive visual Internet of Things management system described in this application, the adjustment module includes a first correction unit; the first correction unit corrects and compensates the ECG electrodes based on the distortion risk of the ECG electrodes, specifically including:

[0039] The peak-to-peak amplitude of the QRS wave of each risk lead is monitored; the first correction unit is configured with an amplitude threshold interval; if the peak-to-peak amplitude of the QRS wave of any risk lead is not within the amplitude threshold interval, the corresponding risk lead is marked as a distorted lead; a spare lead corresponding to the distorted lead is selected and enabled from the spare lead group, and the distorted lead is disabled.

[0040] As a preferred solution of the comprehensive visual Internet of Things management system described in this application, the adjustment module further includes a second correction unit; the second correction unit calibrates and compensates the blood oxygen sensor based on the distortion risk of the blood oxygen sensor, specifically including:

[0041] When there is a risk of distortion in the blood oxygen sensor, the DC component and AC component of the red light are separated from the detection signal of the blood oxygen sensor; the DC-AC ratio of the red light is calculated; the second correction unit is configured with an DC-AC ratio reference value and a stability threshold; the ratio of the DC-AC ratio of the red light to the DC-AC ratio reference value is calculated and recorded as the stability of the red light; if the stability of the red light is less than the stability threshold, the emission ratio of the red light and the infrared light is adjusted based on the stability of the red light, thereby correcting and compensating the blood oxygen sensor.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] This application uses the monitoring unit and pressure array unit in the data acquisition module to collect monitoring images and pressure sensor data of the bed, extract visual skeleton sequences and pressure thermograms, and realize multimodal data fusion visualization. It can present the patient's status more comprehensively and intuitively, making it easier for medical staff to remotely view the patient's posture and assist in judging the condition. Utilizing visualization features, different strategies are adopted to separately identify the distortion risks of ECG electrodes and blood oxygen probes, providing a basis for taking measures in advance. Based on the risk identification results, the monitoring equipment is calibrated and compensated to ensure the accuracy of the monitoring data.

[0044] This application realizes an intelligent process from data collection, processing, risk identification to equipment correction and compensation, reduces manual intervention, reduces the risk of monitoring interruption, improves monitoring data quality and system reliability, and provides more reliable support for patients' continuous monitoring and medical decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] To more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0046] Figure 1 A schematic diagram of the structure of a comprehensive visual Internet of Things management system provided by this application;

[0047] Figure 2 A schematic diagram of an application scenario of a comprehensive visual Internet of Things management system provided in this application. DETAILED DESCRIPTION

[0048] The technical solution of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Unless there is a conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0049] This embodiment introduces a comprehensive visual Internet of Things management system. Figure 1 The system includes a data acquisition module, a processing module, a risk identification module, and an adjustment module; wherein:

[0050] The data acquisition module is used to collect bed data; the bed data includes monitoring images and pressure sensor data of the bed;

[0051] The data acquisition module includes a monitoring unit and a pressure array unit; wherein the monitoring unit is used to collect monitoring images of the hospital bed; the monitoring images of the hospital bed include a planar image and a depth image of the hospital bed; in this embodiment, an RGB-D camera is preferably used as the camera configured for the monitoring unit, and the RGB-D camera is installed above the hospital bed, which can collect and synchronously align the RGB planar image and depth image of the hospital bed.

[0052] The pressure array unit is used to collect pressure sensing data of the hospital bed; the pressure array unit is equipped with a piezoresistive sensor array, which is used to collect pressure sensing data of each position on the hospital bed under the pressure of the patient in real time;

[0053] The processing module extracts visual features based on the bed data; the visual features include visual skeleton sequences and pressure heat maps;

[0054] The processing module includes an image recognition unit and a data processing unit;

[0055] The image recognition unit extracts a visual skeleton sequence based on the monitoring image of the hospital bed; specifically, it includes:

[0056] The depth image and the plane image are aligned in a coordinate system to obtain a fusion matrix; the fusion matrix includes the pixel value and depth value of each pixel in the monitoring image of the bed; for example, the pixel points in the depth image are mapped to the coordinate system of the plane image using an intrinsic parameter matrix calibration method, so that the attribute value of each pixel point in the plane image includes a depth value in addition to the RGB pixel value.

[0057] Inputting the fusion matrix into a joint detection model to obtain the three-dimensional coordinates of each joint of the patient and the confidence level of each joint;

[0058] The joint points are characteristic points of the patient's torso, such as the head, neck, shoulders, elbows, wrists, hips, knees, ankles, etc.; the confidence level represents the probability that the three-dimensional coordinates of the joint points are accurate; the higher the confidence level, the higher the probability that the three-dimensional coordinates of the joint points output by the joint point detection model are their true coordinates.

[0059] This embodiment preferably uses the HRNet-3D model (high-resolution network three-dimensional model) as the joint detection model. Based on the input fusion matrix, the HRNet-3D model detects the patient's joints and calculates the three-dimensional coordinates and confidence of each joint. The three-dimensional coordinate accuracy of the joints is at the millimeter level, and the confidence range is 0 to 1. The HRNet-3D model extracts the features of the fusion matrix using four channels. After extracting the features of the R, G, and B channels through a convolutional network, a heat map of each joint is generated. The pixel value of each pixel in the heat map of the joint represents the probability that the joint exists at that location. The HRNet-3D model calculates the confidence of each joint based on the joint heat map. By training the HRNet-3D model with a training set labeled with joint categories, the HRNet-3D model can automatically identify each joint of the patient in the fusion matrix. For example, the HRNet-3D model outputs heat maps of multiple channels, each channel corresponding to a predefined joint point; based on the heat map of each channel, the three-dimensional coordinates and confidence of the corresponding joint point are calculated, and the calculation results of each channel are mapped to the corresponding joint point according to the channel number.

[0060] The image recognition unit is configured with a confidence threshold and constructs a visual skeleton sequence based on the confidence threshold; each element in the visual skeleton sequence corresponds to a joint point of the patient; wherein, if the confidence of any joint point is greater than or equal to the confidence threshold, the value of the corresponding element in the visual skeleton sequence is the three-dimensional coordinate of the joint point; if the confidence of any joint point is less than the confidence threshold, the value of the corresponding element in the visual skeleton sequence is empty.

[0061] The visual skeleton sequence describes the position of the patient's joints in three-dimensional space and can be used to determine the patient's posture. For joints with low confidence, such as those whose confidence falls below the threshold due to insufficient lighting or partial occlusion, the joint detection model's calculated position of the corresponding joint carries a high risk of significant error. Therefore, the corresponding element value in the visual skeleton sequence is set to null, meaning it is not used in determining the patient's posture. Based on the visual skeleton sequence, the patient's joints can be visualized in three dimensions, facilitating remote monitoring of the patient's posture.

[0062] The data processing unit draws a pressure thermodynamic map based on the pressure sensing data of the hospital bed; specifically includes:

[0063] Arranging the pressure sensing data of each position on the bed collected by the piezoresistive sensor array into a pressure sensing matrix; each element in the pressure sensing matrix corresponds to a piece of pressure sensing data, and the position of the element in the pressure sensing matrix is ​​the position of the piezoresistive sensor corresponding to the pressure sensing data in the piezoresistive sensor array;

[0064] Normalize the element values ​​in the pressure sensing matrix;

[0065] Map the pressure sensor matrix into a pressure heatmap. For example, use Matplotlib or Plotly to map the two-dimensional pressure sensor matrix into a pseudo-color image as a pressure heatmap. This pressure heatmap can visually reflect the distribution of pressure exerted by the patient on the bed, facilitating manual assessment of patient posture.

[0066] The risk identification module identifies the distortion risk of the monitoring device based on the visualization feature; the identification of the distortion risk of the monitoring device includes identifying the distortion risk of the electrocardiogram electrode and the distortion risk of the blood oxygen sensor;

[0067] The risk identification module includes a first identification unit and a second identification unit;

[0068] The first identification unit is used to identify the distortion risk of the ECG electrodes; the ECG electrodes include a standard lead group and a spare lead group; this embodiment preferably uses a standard 12-lead system as the standard lead group, and captures the electrical activity of different areas of the heart through leads in 12 different positions to form a complete electrophysiological view; when the ECG electrodes of the standard lead group are compressed by the patient's posture, etc., causing the monitoring data to be distorted, the electrodes in the spare lead group are activated to replace the standard lead group, so that the ECG detection data remains accurate, monitoring interruptions are reduced, and too many leads are activated at the same time, which increases motion artifacts in the monitoring data and affects the quality of the monitoring data. When setting a spare lead for each lead in the standard lead group, the optimal position of the spare lead is explored through repeated experiments, and the correlation between the QPS waveforms of the standard lead and the spare lead is compared. For example, when the spare lead is set at a certain position, if the Pearson correlation coefficient is greater than 0.85, the spare lead can be set at the corresponding position.

[0069] The risk of identifying ECG electrode distortion specifically includes:

[0070] The pressure heat map is spatiotemporally aligned with the visual skeleton sequence; for example, the pressure heat map and the visual skeleton sequence are temporally aligned through timestamps, so that the pressure heat map and the visual skeleton sequence input into the distortion prediction model at the same moment contain the three-dimensional coordinate data and pressure sensing data of the joint points at the same moment; the three-dimensional coordinates in the visual skeleton sequence and the coordinates of the pressure heat map are mapped to the same coordinate system to complete the spatial alignment.

[0071] Inputting the pressure thermogram and the visual skeleton sequence into a trained distortion prediction model; the distortion prediction model calculates and outputs the distortion risk probability of each lead in the standard lead group;

[0072] The first recognition unit is configured with a distortion probability threshold; if the distortion risk probability of any lead is greater than the distortion probability threshold, the corresponding lead is marked as a risk lead;

[0073] The distortion prediction model includes an input layer, a feature extraction layer, a multimodal fusion layer, a prediction layer, and an output layer; wherein:

[0074] The input layer is used to receive the pressure heat map and visual feature skeleton as model input;

[0075] The feature extraction layer includes a pressure feature sublayer and a posture feature sublayer. The pressure feature sublayer is used to process the pressure heat map and output a pressure semantic vector. In this embodiment, the pressure feature sublayer preferably includes a convolutional neural network and a Transformer structure to extract the pressure distribution pattern (such as high-pressure areas and gradient directions) from the pressure heat map and form a 256-dimensional pressure semantic vector.

[0076] The posture feature sublayer is used to process the visual feature skeleton and output a posture semantic vector. In this embodiment, the posture feature sublayer preferably includes a graph convolutional network and a long short-term memory network, which are used to extract the spatiotemporal evolution of the patient's posture (such as side lying angle and joint movement speed) from the visual feature skeleton and form a 256-dimensional posture semantic vector.

[0077] The multimodal fusion layer generates a joint feature vector based on the pressure semantic vector and the posture semantic vector. For example, through a bidirectional cross-attention mechanism, it establishes an implicit association between the pressure distribution pattern and the spatiotemporal evolution of posture, identifying key combination patterns that cause lead distortion. For example, it explores how pressure distribution and specific postures synergize to cause lead distortion, and explores which pressure patterns under specific postures exacerbate the risk of lead distortion, thereby forming a 256-dimensional joint feature vector.

[0078] The prediction layer calculates the distortion risk vector of the lead based on the joint feature vector. In this embodiment, the prediction layer preferably includes a multi-layer perceptron for processing the joint feature vector and forming a 12-dimensional distortion risk vector, where the dimension of the distortion risk vector is equal to the number of leads in the standard lead set.

[0079] The output layer maps the distortion risk vector of each lead into the distortion risk probability of each lead. The output layer uses a Sigmoid activation function to map the distortion risk vector into an independent distortion risk probability for each lead.

[0080] The second identification unit is configured with a risk identification strategy; the second identification unit identifies the distortion risk of the blood oxygen sensor based on any one of the risk identification strategies;

[0081] The risk identification strategy includes a first identification strategy, which specifically includes: if the values ​​of the elements corresponding to the joint points corresponding to the patient's shoulder joints and hip joints in the visual skeleton sequence are not empty, then the patient's trunk pitch angle is calculated based on the three-dimensional coordinates of the joint points corresponding to the patient's shoulder joints and hip joints; the second identification unit is also configured with a pitch angle threshold. If the patient's trunk pitch angle is less than the pitch angle threshold, there is a risk of distortion of the blood oxygen sensor. For example, the values ​​of the elements corresponding to the patient's left shoulder, right shoulder, left hip, and right hip in the visual skeleton sequence are not empty, that is, the confidence levels of the joint points corresponding to the shoulders and hips are greater than or equal to the confidence threshold; the trunk pitch angle is calculated based on the three-dimensional coordinates of the joint points corresponding to the shoulders and hips;

[0082] The preferred calculation formula for the trunk pitch angle in this embodiment is as follows:

[0083] ;

[0084] in, represents the patient's trunk pitch angle, is the inverse tangent function; is the coordinate of the joint point corresponding to the left shoulder along the z-axis; 、 are the coordinates of the joint point corresponding to the left shoulder along the x-axis and y-axis respectively; is the coordinate of the joint point corresponding to the left hip along the z-axis; 、 are the coordinates of the joint points corresponding to the left hip along the x-axis and y-axis respectively; the positive direction of the z-axis is vertical to the horizontal plane upward, that is, is the distance between the joint points corresponding to the left shoulder and left hip on the horizontal plane, is the height difference between the joint points corresponding to the left shoulder and the left hip; when the patient's posture gradually transitions from semi-recumbent to supine to prone, the trunk pitch angle gradually decreases; when the trunk pitch angle is less than the pitch angle threshold, the patient's posture is prone. The prone posture puts pressure on the limb on the side of the blood oxygen sensor, resulting in reduced blood perfusion and the risk of distortion of the blood oxygen sensor detection data. Preferably, It can also represent the midpoint coordinates of the joints corresponding to the shoulders. Indicates the midpoint coordinates of the joints corresponding to the two hips, making the calculation of the torso pitch angle more accurate.

[0085] The risk identification strategy also includes a second identification strategy, which specifically includes: if the values ​​of the elements corresponding to the joint points corresponding to the patient's two shoulder joints in the visual skeleton sequence are not empty, then calculating the patient's shoulder height difference based on the three-dimensional coordinates of the joint points corresponding to the patient's two shoulder joints; the shoulder height difference is the difference between the z-axis coordinates of the joint points corresponding to the shoulder joint on the non-probe side and the z-axis coordinates of the joint points corresponding to the shoulder joint on the blood oxygen sensor side; the second identification unit is also configured with a height difference threshold. If the patient's shoulder height difference is greater than the height difference threshold, there is a risk of distortion of the blood oxygen sensor. For example, if the blood oxygen sensor is connected to the patient's left hand finger, the left shoulder is the blood oxygen sensor-side shoulder joint and the right shoulder is the non-probe-side shoulder joint. The z-axis coordinate of the joint point corresponding to the left shoulder is subtracted from the z-axis coordinate of the joint point corresponding to the right shoulder to obtain the shoulder height difference; if the shoulder height difference is greater than the height difference threshold, the patient is lying on his left side, which puts pressure on the left hand connected to the blood oxygen sensor, reduces blood perfusion, and causes a risk of distortion of the blood oxygen sensor detection data.

[0086] The risk identification strategy also includes a third identification strategy, which specifically includes: if the value of the element corresponding to the joint point of at least one relevant joint on the blood oxygen sensor side in the visual skeleton sequence is not empty, then the pressure sensor data of the relevant joint is obtained based on the three-dimensional coordinates of the joint point and the pressure heat map; the second identification unit is also configured with a pressure threshold. If the pressure sensor data of at least one relevant joint exceeds the pressure threshold, the blood oxygen sensor is at risk of distortion. For example, if the blood oxygen sensor is connected to the fingers of the patient's left hand, the relevant joints include the left shoulder, left elbow, and left wrist. Because the pressure heat map and the visual skeleton sequence are spatially and temporally aligned, the position of any relevant joint in the pressure heat map can be located based on its three-dimensional coordinates, and its pressure sensor data can be read from the pressure heat map. If the pressure sensor data of at least one of the left shoulder, left elbow, and left wrist exceeds the pressure threshold, the left hand is at risk of compression, causing the blood oxygen sensor detection data to be distorted.

[0087] The adjustment module performs correction compensation on the monitoring device based on the distortion risk of the monitoring device.

[0088] The adjustment module includes a first correction unit and a second correction unit;

[0089] The first correction unit corrects and compensates the ECG electrodes based on the distortion risk of the ECG electrodes, specifically including:

[0090] The peak-to-peak amplitude of the QRS wave of each risk lead is monitored; the first correction unit is configured with an amplitude threshold interval; if the peak-to-peak amplitude of the QRS wave of any risk lead is not within the amplitude threshold interval, the corresponding risk lead is marked as a distorted lead; a backup lead corresponding to the distorted lead is selected and activated from the backup lead group, and the distorted lead is deactivated. For example, when a patient is in the left lateral decubitus position, changes in chest pressure distribution lead to poor electrode contact or signal distortion. At this time, based on the distortion prediction model and the distortion probability threshold, the left chest lead (V4-V6) located on the lateral wall of the left ventricle of the heart is determined to be a risk lead, and the peak-to-peak amplitude of its QRS wave is not within the amplitude threshold interval, then the corresponding backup lead is activated to replace the left chest lead.

[0091] The second correction unit corrects and compensates the blood oxygen sensor based on the distortion risk of the blood oxygen sensor, specifically including:

[0092] When there is a risk of distortion in the blood oxygen sensor, the distortion verification is performed on the blood oxygen sensor, specifically including: separating the DC component and AC component of the red light from the detection signal of the blood oxygen sensor; calculating the AC-DC ratio of the red light, that is, the ratio of the AC component to the DC component of the red light; the second correction unit is configured with an AC-DC ratio reference value and a stability threshold; calculating the ratio of the AC-DC ratio of the red light to the AC-DC ratio reference value, which is recorded as the stability of the red light; if the stability of the red light is less than the stability threshold, adjusting the emission ratio of the red light and the infrared light based on the stability of the red light, thereby correcting and compensating the blood oxygen sensor.

[0093] The Spo2 oximeter calculates blood oxygen saturation based on the differences in the absorption characteristics of oxyhemoglobin and deoxyhemoglobin at different wavelengths. Specifically, the Spo2 oximeter calculates blood oxygen saturation based on the cross-to-direct ratio of the collected red light to the cross-to-direct ratio of the collected infrared light. When local blood flow decreases and the deoxyhemoglobin concentration increases in the extremity where the Spo2 oximeter is located, the AC component of the red light is significantly attenuated, resulting in distortion in the blood oxygen saturation calculation. The Spo2 oximeter dynamically adjusts the emission ratio of red and infrared light, increasing the red light power to compensate for signal loss. This offsets absorbance deviations caused by low blood perfusion and brings the calculated blood oxygen saturation closer to the true value. For example, the base emission ratio (power ratio) of red and infrared light is set to 1:11. If the Spo2 oximeter distorts, the emission ratio is adjusted based on the stability of the red light. The lower the red light stability, the higher the emission ratio. The maximum emission ratio is limited to 2:1 to prevent the red light power from exceeding safety limits.

[0094] The application scenarios of the comprehensive visual Internet of Things management system introduced in this embodiment are as follows: Figure 2 shown.

[0095] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] The above describes the embodiments of the present application in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose and scope of protection of this application, all of which are protected by this application.

Claims

1. A comprehensive visual Internet of Things management system, characterized by: It includes data acquisition module, processing module, risk identification module and adjustment module; among which: The data acquisition module is used to collect bed data; the bed data includes monitoring images and pressure sensor data of the bed; The processing module extracts visual features based on the bed data; the visual features include visual skeleton sequences and pressure heat maps; The risk identification module identifies the distortion risk of the monitoring device based on the visualization feature; the identification of the distortion risk of the monitoring device includes identifying the distortion risk of the electrocardiogram electrode and the distortion risk of the blood oxygen sensor; The adjustment module performs correction compensation on the monitoring device based on the distortion risk of the monitoring device; The processing module includes an image recognition unit; the image recognition unit extracts a visual skeleton sequence based on the monitoring image of the bed; specifically includes: Performing coordinate system registration on the depth image and the plane image to obtain a fusion matrix; the fusion matrix includes the pixel value and depth value of each pixel point in the monitoring image of the bed; Inputting the fusion matrix into a joint detection model to obtain the three-dimensional coordinates of each joint of the patient and the confidence level of each joint; The image recognition unit is configured with a confidence threshold, and constructs a visual skeleton sequence based on the confidence threshold; Each element in the visual skeleton sequence corresponds to a joint point of the patient; wherein, if the confidence of any joint point is greater than or equal to the confidence threshold, the value of the corresponding element in the visual skeleton sequence is the three-dimensional coordinate of the joint point; if the confidence of any joint point is less than the confidence threshold, the value of the corresponding element in the visual skeleton sequence is empty; The processing module further includes a data processing unit; the data processing unit draws a pressure thermodynamic map based on the pressure sensing data of the hospital bed; specifically includes: The pressure sensing data collected by the piezoresistive sensor array at each position on the bed are organized into a pressure sensing matrix; each element in the pressure sensing matrix corresponds to a piece of pressure sensing data, and the position of the element in the pressure sensing matrix is ​​the position of the piezoresistive sensor corresponding to the pressure sensing data in the piezoresistive sensor array; the element values ​​in the pressure sensing matrix are normalized; and the pressure sensing matrix is ​​mapped into a pressure heat map; The risk identification module includes a first identification unit; the first identification unit is used to identify the distortion risk of the ECG electrodes; the ECG electrodes include a standard lead group and a spare lead group; the identification of the distortion risk of the ECG electrodes specifically includes: performing spatiotemporal alignment of the pressure heatmap with the visual skeleton sequence; Inputting the pressure thermogram and the visual skeleton sequence into a trained distortion prediction model; the distortion prediction model calculates and outputs the distortion risk probability of each lead in the standard lead group; The first recognition unit is configured with a distortion probability threshold; if the distortion risk probability of any lead is greater than the distortion probability threshold, the corresponding lead is marked as a risky lead.

2. A comprehensive visual Internet of Things management system according to claim 1, characterized in that: The data acquisition module includes a monitoring unit and a pressure array unit; wherein the monitoring unit is used to collect monitoring images of the bed; the monitoring images of the bed include a planar image and a depth image of the bed; The pressure array unit is used to collect pressure sensing data of the hospital bed; the pressure array unit is configured with a piezoresistive sensor array, and the pressure array unit collects pressure sensing data of each position on the hospital bed in real time based on the piezoresistive sensor array.

3. A comprehensive visual Internet of Things management system according to claim 2, characterized in that: The distortion prediction model includes an input layer, a feature extraction layer, a multimodal fusion layer, a prediction layer, and an output layer; wherein: The input layer is used to receive the pressure heat map and visual skeleton sequence as model input; The feature extraction layer includes a pressure feature sublayer and a posture feature sublayer. The pressure feature sublayer is used to process the pressure heat map and output a pressure semantic vector. The posture feature sublayer is used to process the visual skeleton sequence and output a posture semantic vector. The multimodal fusion layer generates a joint feature vector based on the pressure semantic vector and the posture semantic vector; The prediction layer calculates the distortion risk vector of the lead based on the joint feature vector; The output layer maps the distortion risk vector of the lead to the distortion risk probability of each lead.

4. A comprehensive visual Internet of Things management system according to claim 3, characterized in that: The risk identification module further includes a second identification unit; the second identification unit is configured with a risk identification strategy; The second identification unit identifies the distortion risk of the blood oxygen sensor based on any risk identification strategy; The risk identification strategy includes a first identification strategy, which specifically includes: if the values ​​of the elements corresponding to the joint points corresponding to the patient's shoulder joints and hip joints in the visual skeleton sequence are not empty, then calculating the patient's trunk pitch angle based on the three-dimensional coordinates of the joint points corresponding to the patient's shoulder joints and hip joints; The second identification unit is further configured with a pitch angle threshold. If the patient's torso pitch angle is less than the pitch angle threshold, there is a risk of distortion of the blood oxygen sensor.

5. A comprehensive visual Internet of Things management system according to claim 4, characterized in that: The risk identification strategy also includes a second identification strategy, which specifically includes: if the value of the element corresponding to the joint points corresponding to the patient's two shoulder joints in the visual skeleton sequence is not empty, then calculating the patient's shoulder height difference based on the three-dimensional coordinates of the joint points corresponding to the patient's two shoulder joints; the shoulder height difference is the difference between the z-axis coordinate of the joint point corresponding to the shoulder joint on the non-probe side and the z-axis coordinate of the joint point corresponding to the shoulder joint on the blood oxygen probe side; the second identification unit is also configured with a height difference threshold, if the patient's shoulder height difference is greater than the height difference threshold, then there is a risk of distortion of the blood oxygen probe; The risk identification strategy also includes a third identification strategy, which specifically includes: if the value of the element corresponding to the joint point corresponding to at least one relevant joint on the blood oxygen probe side in the visual skeleton sequence is not empty, then the pressure sensing data of the relevant joint is obtained based on the three-dimensional coordinates of the joint point corresponding to the relevant joint and the pressure heat map; the second identification unit is also configured with a pressure threshold. If the pressure sensing data of at least one relevant joint is greater than the pressure threshold, there is a distortion risk in the blood oxygen probe.

6. A comprehensive visual Internet of Things management system according to claim 5, characterized in that: The adjustment module includes a first correction unit; the first correction unit corrects and compensates the ECG electrode based on the distortion risk of the ECG electrode, specifically including: The peak-to-peak amplitude of the QRS wave of each risk lead is monitored; the first correction unit is configured with an amplitude threshold interval; if the peak-to-peak amplitude of the QRS wave of any risk lead is not within the amplitude threshold interval, the corresponding risk lead is marked as a distorted lead; a spare lead corresponding to the distorted lead is selected and enabled from the spare lead group, and the distorted lead is disabled.

7. A comprehensive visual Internet of Things management system according to claim 6, characterized in that: The adjustment module further includes a second correction unit; the second correction unit performs correction compensation on the blood oxygen sensor based on the distortion risk of the blood oxygen sensor, specifically including: When there is a risk of distortion in the blood oxygen sensor, the DC component and AC component of the red light are separated from the detection signal of the blood oxygen sensor; the DC-AC ratio of the red light is calculated; the second correction unit is configured with an DC-AC ratio reference value and a stability threshold; the ratio of the DC-AC ratio of the red light to the DC-AC ratio reference value is calculated and recorded as the stability of the red light; if the stability of the red light is less than the stability threshold, the emission ratio of the red light and the infrared light is adjusted based on the stability of the red light, thereby correcting and compensating the blood oxygen sensor.

Images