Comprehensive visual Internet of Things management system

Abstract

The invention relates to the technical field of medical information technology, and discloses a comprehensive visual Internet of Things management system, which comprises a data acquisition module, a processing module, a risk identification module and an adjustment module, the data acquisition module is used for acquiring sickbed data; the sickbed data comprises a monitoring image and pressure sensing data of a sickbed; the processing module extracts visual features based on the sickbed data; the visual features comprise a visual skeleton sequence and a pressure thermodynamic diagram; the risk identification module identifies the distortion risk of the monitoring equipment based on the visual features; the identification of the distortion risk of the monitoring equipment comprises the identification of the distortion risk of an electrocardio electrode and the distortion risk of a blood oxygen probe; and the adjusting module corrects and compensates the monitoring equipment based on the distortion risk of the monitoring equipment. Distortion risk identification and correction compensation of the medical monitoring equipment are realized, and medical care state visualization and monitoring automation are realized.

Description

Technical Field

[0001] This application relates to the technical field of medical information technology, and particularly relates to a comprehensive visualization Internet of Things management system. Background Art

[0002] In medical scenarios such as postoperative monitoring, frequent position changes of patients can cause poor contact of electrocardiogram electrodes, blood oxygen probes, etc. due to body compression or displacement, resulting in noise or signal loss. The instantaneous signal attenuation caused by position changes triggers false alarms, interfering with the judgment of medical staff. When the monitoring device is distorted, medical staff need to manually adjust the position of the device, increasing the workload. Existing technologies achieve more accurate monitoring by increasing the alarm sensitivity or filtering the signals, but cannot fundamentally solve the problem of the interference of body position on the monitoring device.

[0003] Traditional monitoring systems lack real-time data processing and transmission, and existing risk warning systems alarm after problems occur in the monitoring device, lacking the ability of active prediction and prevention of risks. For example, when the position of a patient changes and causes the electrode to fall off, the system cannot predict in advance and take corresponding measures. In addition, when dealing with distorted data, existing monitoring systems often require manual intervention and have a low degree of automation.

[0004] As disclosed in the Chinese patent with the authorization announcement number CN115394418B, a refined status monitoring system for medical devices based on the Internet of Things is disclosed. The status monitoring system includes a server, and also includes an interaction module, a device status monitoring module, a fault reporting module, a positioning module, and several mobile terminals. Each mobile terminal is used to connect to the server to query information and report faults of medical devices; the positioning module is used to locate the distribution positions of medical devices, the device status monitoring module is used to monitor the operating status of medical devices, the fault reporting module is used to report faults of medical devices, and the interaction module is used to interact with maintenance personnel according to the data of the fault reporting module; through the device status monitoring module to monitor the usage status of medical devices, the invention can effectively understand the usage status of medical devices to improve the refined management level of the entire medical device.

[0005] As disclosed in the Chinese patent with the authorization announcement number CN118762816B, a method and system for adaptive remote medical monitoring based on the Internet of Things are provided, including creating a patient file and performing device allocation and configuration; collecting real-time health data of the patient and transmitting it to the cloud server for data preprocessing; performing anomaly detection on the preprocessed data and conducting a preliminary analysis of the abnormal data; evaluating the patient's health status and formulating response measures. By creating personalized patient files, performing intelligent device allocation and configuration, constructing anomaly detection algorithms, and conducting a preliminary analysis of abnormal data, the invention ensures that the monitoring devices can accurately meet the personalized monitoring needs of patients, not only improving the efficiency and accuracy of remote medical monitoring, but also being able to promptly detect abnormal changes in the patient's health status, providing a scientific decision-making basis for medical staff.

[0006] The above existing technologies all have the problems raised in this background technology: they cannot solve the interference problem of the patient's body position on the monitoring device.

[0007] The information disclosed in this background technology section is only intended to increase the overall understanding of the present application and should not be regarded as an admission or an indication in any form that this information constitutes the prior art already known to those of ordinary skill in the art. Summary of the Invention

[0008] The technical problem to be solved by the present application is to overcome the defects of the prior art and provide a comprehensive visualization Internet of Things management system to realize the identification and correction compensation of the distortion risk of medical monitoring devices, and to realize the visualization of medical staff status and the automation of monitoring.

[0009] To solve the above technical problems, the present application provides the following technical solutions:

[0010] A comprehensive visualization Internet of Things management system includes a data acquisition module, a processing module, a risk identification module, and an adjustment module; wherein:

[0011] The data acquisition module is used to acquire hospital bed data; the hospital bed data includes the monitoring images of the hospital bed and pressure sensing data;

[0012] The processing module extracts visualization features based on the hospital bed data; the visualization features include a visual skeleton sequence and a pressure heat map;

[0013] The risk identification module identifies the distortion risk of the monitoring device based on the visualization features; the identification of the distortion risk of the monitoring device includes identifying the distortion risk of electrocardiogram electrodes and the distortion risk of blood oxygen probes;

[0014] The adjustment module performs correction compensation on the monitoring device based on the distortion risk of the monitoring device.

[0015] As a preferred solution of the comprehensive visual IoT management system described in the present application, wherein: the data acquisition module includes a monitoring unit and a pressure array unit; wherein, the monitoring unit is used to collect monitoring images of the hospital bed; the monitoring images of the hospital bed include a planar image and a depth image of the hospital bed;

[0016] The pressure array unit is used to collect pressure sensing data of the hospital bed; the pressure array unit is configured with a piezoresistive sensor array, and the pressure array unit collects pressure sensing data of each position on the hospital bed in real time based on the piezoresistive sensor array.

[0017] As a preferred solution of the comprehensive visual IoT management system described in the present application, wherein: the processing module includes an image recognition unit; the image recognition unit extracts a visual skeleton sequence based on the monitoring images of the hospital bed; specifically including:

[0018] Perform coordinate registration on the depth image and the planar image to obtain a fusion matrix; the fusion matrix includes the pixel value and depth value of each pixel point in the monitoring image of the hospital bed;

[0019] Input the fusion matrix into a joint point detection model to obtain the three-dimensional coordinates of each joint point of the patient and the confidence of each joint point;

[0020] The image recognition unit is configured with a confidence threshold, and constructs a visual skeleton sequence based on the confidence threshold;

[0021] Each element in the visual skeleton sequence corresponds to a joint point of the patient; wherein, if the confidence of any joint point is greater than or equal to the confidence threshold, the value of the corresponding element in the visual skeleton sequence is the three-dimensional coordinates of the joint point; if the confidence of any joint point is less than the confidence threshold, the value of the corresponding element in the visual skeleton sequence is empty.

[0022] As a preferred solution of the comprehensive visual IoT management system described in the present application, wherein: the processing module further includes a data processing unit; the data processing unit draws a pressure heat map based on the pressure sensing data of the hospital bed; specifically including:

[0023] Organize the pressure sensing data of each position on the hospital bed collected by the piezoresistive sensor array into a pressure sensing matrix; each element in the pressure sensing matrix corresponds to a pressure sensing data, and the position of the element in the pressure sensing matrix is the position of the piezoresistive sensor corresponding to the pressure sensing data in the piezoresistive sensor array; perform normalization processing on the element values in the pressure sensing matrix; map the pressure sensing matrix into a pressure heat map.

[0024] As a preferred solution of the comprehensive visual Internet of Things management system described in this application, where: the risk identification module includes a first identification unit; the first identification unit is used to identify the distortion risk of the electrocardiogram electrodes; the electrocardiogram electrodes include a standard lead group and a spare lead group; identifying the distortion risk of the electrocardiogram electrodes specifically includes:

[0025] Perform spatio-temporal alignment of the pressure heat map and the visual skeleton sequence;

[0026] Input the pressure heat map and the visual skeleton sequence into the trained distortion prediction model; the distortion prediction model calculates and outputs the distortion risk probability of each lead in the standard lead group;

[0027] The first identification unit is configured with a distortion probability threshold; if the distortion risk probability of any lead is greater than the distortion probability threshold, the corresponding lead is marked as a risk lead.

[0028] As a preferred solution of the comprehensive visual Internet of Things management system described in this application, where: the distortion prediction model includes an input layer, a feature extraction layer, a multi-modal fusion layer, a prediction layer, and an output layer; where:

[0029] The input layer is used to receive the pressure heat map and the visual feature skeleton as model inputs;

[0030] The feature extraction layer includes a pressure feature sub-layer and a pose feature sub-layer; among them, the pressure feature sub-layer is used to process the pressure heat map and output a pressure semantic vector; the pose feature sub-layer is used to process the visual feature skeleton and output a pose semantic vector;

[0031] The multi-modal fusion layer generates a joint feature vector based on the pressure semantic vector and the pose semantic vector;

[0032] The prediction layer calculates the distortion risk vector of the lead based on the joint feature vector;

[0033] The output layer maps the distortion risk vector of the lead into the distortion risk probability of each lead.

[0034] As a preferred solution of the comprehensive visual Internet of Things management system described in this application, where: the risk identification module further includes a second identification unit; the second identification unit is configured with a risk identification strategy; the second identification unit identifies the distortion risk of the blood oxygen probe based on any one of the risk identification strategies;

[0035] The risk identification strategy includes a first identification strategy, which specifically includes: if the values of the elements corresponding to the joint points of the patient's bilateral shoulder joints and bilateral hip joints in the visual skeleton sequence are not empty, calculate the torso pitch angle of the patient based on the three-dimensional coordinates of the joint points of the patient's bilateral shoulder joints and bilateral hip joints; the second identification unit is also configured with a pitch angle threshold, and if the torso pitch angle of the patient is less than the pitch angle threshold, there is a risk of distortion of the blood oxygen probe.

[0036] As a preferred solution of the comprehensive visualization Internet of Things management system described in the present application, wherein: the risk identification strategy further includes a second identification strategy, which specifically includes: if the value of the element corresponding to the joint point of the patient's bilateral shoulder joints in the visual skeleton sequence is not empty, calculate the shoulder height difference of the patient based on the three-dimensional coordinates of the joint points of the patient's bilateral shoulder joints; the shoulder height difference is the difference between the z-axis coordinate of the joint point corresponding to the shoulder joint on the non-probe side and the z-axis coordinate of the joint point corresponding to the shoulder joint on the blood oxygen probe side; the second identification unit is also configured with a height difference threshold, and if the shoulder height difference of the patient is greater than the height difference threshold, there is a risk of distortion of the blood oxygen probe;

[0037] The risk identification strategy further includes a third identification strategy, which specifically includes: if the values of the elements corresponding to the joint points of at least one relevant joint on the blood oxygen probe side in the visual skeleton sequence are not empty, obtain the pressure sensing data of the relevant joint based on the three-dimensional coordinates of the joint points corresponding to the relevant joint and the pressure thermogram; the second identification unit is also configured with a pressure threshold, and if the pressure sensing data of at least one relevant joint is greater than the pressure threshold, there is a risk of distortion of the blood oxygen probe.

[0038] As a preferred solution of the comprehensive visualization Internet of Things management system described in the present application, wherein: the adjustment module includes a first correction unit; the first correction unit corrects and compensates the electrocardiogram electrode based on the distortion risk of the electrocardiogram electrode, specifically including:

[0039] Monitor the peak-to-peak amplitude of the QRS wave of each risk lead; the first correction unit is configured with an amplitude threshold interval; if the peak-to-peak amplitude of the QRS wave of any risk lead is not within the amplitude threshold interval, mark the corresponding risk lead as a distorted lead; select and enable the spare lead corresponding to the distorted lead from the spare lead group, and deactivate the distorted lead.

[0040] As a preferred solution of the comprehensive visualization Internet of Things management system described in the present application, wherein: the adjustment module further includes a second correction unit; the second correction unit corrects and compensates the blood oxygen probe based on the distortion risk of the blood oxygen probe, specifically including:

[0041] When there is a risk of distortion in the blood oxygen probe, the DC component and the AC component of the red light are separated from the detection signal of the blood oxygen probe; the AC / DC ratio of the red light is calculated; the second correction unit is configured with an AC / DC ratio reference value and a stability threshold; the ratio of the AC / DC ratio of the red light to the AC / DC ratio reference value is calculated and denoted as the stability of the red light; if the stability of the red light is less than the stability threshold, the emission ratio of the red light and the infrared light is adjusted based on the stability of the red light, so as to correct and compensate the blood oxygen probe.

[0042] Compared with the prior art, the beneficial effects achieved by this application are as follows:

[0043] Through the monitoring unit and the pressure array unit in the data acquisition module of this application, the monitoring images and pressure sensing data of the hospital bed are collected, and the visual skeleton sequence and the pressure heat map are extracted to realize the visualization of multi-modal data fusion, which can present the patient's state more comprehensively and intuitively, facilitate the medical staff to remotely view the patient's posture, and assist in judging the condition. Using the visualization features, different strategies are adopted to respectively identify the distortion risks of the electrocardiogram electrode and the blood oxygen probe, providing a basis for taking measures in advance. The monitoring equipment is corrected and compensated according to the risk identification results to ensure the accuracy of the monitoring data.

[0044] This application realizes an intelligent process from data acquisition, processing, risk identification to equipment correction and compensation, reduces manual intervention, reduces the risk of monitoring interruption, improves the quality of monitoring data and the reliability of the system, and provides more reliable support for the continuous monitoring of patients and medical decision-making. Description of the Drawings

[0045] In order to more clearly illustrate the technical solutions of the embodiments of this application, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. Among them:

[0046] Figure 1 It is a schematic structural diagram of a comprehensive visualization Internet of Things management system provided by this application;

[0047] Figure 2 It is a schematic application scenario diagram of a comprehensive visualization Internet of Things management system provided by this application. Detailed Embodiments

[0048] The technical solutions of this application will be described in detail below through the drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. Without conflict, the technical features in the embodiments of this application and the embodiments can be combined with each other.

[0049] This embodiment introduces a comprehensive visualization Internet of Things management system. Referring to Figure 1 , the system includes a data acquisition module, a processing module, a risk identification module, and an adjustment module; among them:

[0050] The data acquisition module is used to acquire hospital bed data; the hospital bed data includes the monitoring image of the hospital bed and the pressure sensing data.

[0051] The data acquisition module includes a monitoring unit and a pressure array unit; among them, the monitoring unit is used to acquire the monitoring image of the hospital bed; the monitoring image of the hospital bed includes the planar image and the depth image of the hospital bed; in this embodiment, an RGB-D camera is preferably used as the camera configured in the monitoring unit, and the RGB-D camera is installed above the hospital bed, and can acquire and synchronously align the RGB planar image and the depth image of the hospital bed.

[0052] The pressure array unit is used to acquire the pressure sensing data of the hospital bed; the pressure array unit is configured with a piezoresistive sensor array for real-time acquisition of the pressure sensing data of each position on the hospital bed under the pressure of the patient.

[0053] The processing module extracts visualization features based on the hospital bed data; the visualization features include a visual skeleton sequence and a pressure heat map.

[0054] The processing module includes an image recognition unit and a data processing unit.

[0055] The image recognition unit extracts a visual skeleton sequence based on the monitoring image of the hospital bed; specifically including:

[0056] Perform coordinate registration on the depth image and the planar image to obtain a fusion matrix; the fusion matrix includes the pixel value and the depth value of each pixel point in the monitoring image of the hospital bed; for example, use the method of intrinsic matrix calibration to map the pixel points in the depth image to the coordinate system of the planar image, so that the attribute value of each pixel point in the planar image includes not only the RGB pixel value but also the depth value.

[0057] Input the fusion matrix into the joint point detection model to obtain the three-dimensional coordinates of each joint point of the patient and the confidence of each joint point.

[0058] The joint points are the feature points of the patient's torso, such as the head, neck, shoulders, elbows, wrists, hips, knees, ankles, etc.; the confidence represents the probability that the three-dimensional coordinates of the joint point are accurate; the higher the confidence, the higher the probability that the three-dimensional coordinates of the joint point output by the joint point detection model are its true coordinates.

[0059] In this embodiment, the HRNet-3D model (High-Resolution Network 3D model) is preferably used as the joint point detection model. The HRNet-3D model detects the joint points of the patient based on the input fusion matrix and calculates the three-dimensional coordinates and confidence levels of each joint point. Among them, the accuracy of the three-dimensional coordinates of the joint points is at the millimeter level, and the value range of the confidence level is from 0 to 1. The HRNet-3D model extracts the features of the fusion matrix through four channels. Among them, after extracting the features of the R, G, and B channels through the convolutional network, a heat map of each joint point is generated. The pixel value of each pixel in the heat map of the joint point represents the probability that the joint point exists at that position; the HRNet-3D model calculates the confidence level of each joint point based on the heat map of the joint point. By training the HRNet-3D model with a training set labeled with joint point categories, the HRNet-3D model can automatically identify each joint point of the patient in the fusion matrix. For example, the HRNet-3D model outputs heat maps of multiple channels, and each channel corresponds to a predefined joint point; based on the heat map of each channel, the three-dimensional coordinates and confidence level of the corresponding joint point are calculated, and the calculation results of each channel are mapped to the corresponding joint point according to the channel number.

[0060] The image recognition unit is configured with a confidence threshold and constructs a visual skeleton sequence based on the confidence threshold; each element in the visual skeleton sequence corresponds to a joint point of the patient; among them, if the confidence level of any joint point is greater than or equal to the confidence threshold, the value of the corresponding element in the visual skeleton sequence is the three-dimensional coordinates of the joint point; if the confidence level of any joint point is less than the confidence threshold, the value of the corresponding element in the visual skeleton sequence is empty.

[0061] The visual skeleton sequence describes the positions of the joint points of the patient in three-dimensional space and can be used for the posture judgment of the patient. For joint points with low confidence levels, for example, due to insufficient light, local occlusion, etc., the confidence levels of some joint points are lower than the confidence threshold, then there is a high risk of obvious errors in the positions of the corresponding joint points calculated by the joint point detection model. Therefore, the value of the corresponding element in the visual skeleton sequence is set to empty, that is, it does not participate in the posture judgment of the patient. Based on the visual skeleton sequence, three-dimensional visualization processing of the joint points of the patient can be performed, which is convenient for remotely viewing the posture of the patient.

[0062] The data processing unit draws a pressure heat map based on the pressure sensing data of the hospital bed; specifically including:

[0063] Organize the pressure sensing data of each position on the hospital bed collected by the piezoresistive sensor array into a pressure sensing matrix; each element in the pressure sensing matrix corresponds to a pressure sensing data, and the position of the element in the pressure sensing matrix is the position of the piezoresistive sensor corresponding to the pressure sensing data in the piezoresistive sensor array;

[0064] Normalize the element values in the pressure sensing matrix;

[0065] Map the pressure sensing matrix into a pressure heat map. For example, map the two-dimensional pressure sensing matrix into a pseudo-color map as the pressure heat map through Matplotlib or Plotly. The pressure heat map can intuitively reflect the distribution of the pressure exerted by the patient on the hospital bed, facilitating manual auxiliary judgment of the patient's posture.

[0066] Based on the visualization features, the risk identification module identifies the distortion risks of the monitoring device; identifying the distortion risks of the monitoring device includes identifying the distortion risks of electrocardiogram electrodes and blood oxygen probes;

[0067] The risk identification module includes a first identification unit and a second identification unit;

[0068] The first identification unit is used to identify the distortion risks of electrocardiogram electrodes; the electrocardiogram electrodes include a standard lead group and a spare lead group; in this embodiment, the standard 12-lead system is preferably used as the standard lead group, and the electrical activities of different regions of the heart are captured through 12 leads at different positions to form a complete electrophysiological view; when the electrocardiogram electrodes in the standard lead group are distorted due to being pressed by the patient's posture, etc., the electrodes in the spare lead group are enabled to replace the standard lead group, so that the electrocardiogram detection data remains accurate, reducing monitoring interruptions, and avoiding the increase of motion artifacts in the monitoring data when too many leads are enabled simultaneously, affecting the quality of the monitoring data. When setting spare leads for the leads in each standard lead group, explore the optimal positions of the spare leads through repeated experiments, and compare the correlation of the QPS waveforms of the standard leads and the spare leads. For example, when the spare lead is set at a certain position and the Pearson correlation coefficient is greater than 0.85, the spare lead can be set at the corresponding position.

[0069] Specifically, identifying the distortion risks of electrocardiogram electrodes includes:

[0070] Perform spatio-temporal alignment of the pressure heat map and the visual skeleton sequence; for example, align the pressure heat map and the visual skeleton sequence in time through timestamps, so that the pressure heat map and the visual skeleton sequence input into the distortion prediction model at the same moment contain the three-dimensional coordinate data of the joint points and the pressure sensing data at the same moment; map the three-dimensional coordinates in the visual skeleton sequence and the coordinates of the pressure heat map into the same coordinate system to complete the spatial alignment.

[0071] Input the pressure heat map and the visual skeleton sequence into the trained distortion prediction model; the distortion prediction model calculates and outputs the distortion risk probability of each lead in the standard lead group;

[0072] The first recognition unit is configured with a distortion probability threshold; if the distortion risk probability of any lead is greater than the distortion probability threshold, the corresponding lead is marked as a risk lead;

[0073] The distortion prediction model includes an input layer, a feature extraction layer, a multi-modal fusion layer, a prediction layer, and an output layer; where:

[0074] The input layer is used to receive the pressure heat map and the visual feature skeleton as model inputs;

[0075] The feature extraction layer includes a pressure feature sub-layer and a pose feature sub-layer; among them, the pressure feature sub-layer is used to process the pressure heat map and output a pressure semantic vector; in this embodiment, it is preferably that the pressure feature sub-layer includes a convolutional neural network and a Transformer structure, which are used to extract the pressure distribution pattern (such as high-pressure area, gradient direction) from the pressure heat map and form a 256-dimensional pressure semantic vector;

[0076] The pose feature sub-layer is used to process the visual feature skeleton and output a pose semantic vector; in this embodiment, it is preferably that the pose feature sub-layer includes a graph convolutional network and a long short-term memory network, which are used to extract the spatio-temporal evolution of the patient's pose (such as lateral lying angle, joint movement speed) from the visual feature skeleton and form a 256-dimensional pose semantic vector;

[0077] The multi-modal fusion layer generates a joint feature vector based on the pressure semantic vector and the pose semantic vector; for example, through a bidirectional cross-attention mechanism, an implicit association between the pressure distribution pattern and the spatio-temporal evolution of the pose is established to identify the key combined patterns that cause lead distortion, such as exploring how the pressure distribution cooperates with a specific pose to trigger lead distortion, and exploring which pressure patterns under a specific pose exacerbate the lead distortion risk, and forming a 256-dimensional joint feature vector;

[0078] The prediction layer calculates the distortion risk vector of the lead based on the joint feature vector; in this embodiment, it is preferably that the prediction layer includes a multi-layer perceptron, which is used to process the joint feature vector and form a 12-dimensional distortion risk vector, and the dimension of the distortion risk vector is equal to the number of leads in the standard lead group;

[0079] The output layer maps the distortion risk vector of the lead into the distortion risk probability of each lead. The output layer uses a Sigmoid activation function to map the distortion risk vector into the independent distortion risk probability of each lead.

[0080] The second recognition unit is configured with a risk recognition strategy; the second recognition unit identifies the distortion risk of the blood oxygen probe based on any one of the risk recognition strategies;

[0081] The risk identification strategy includes a first identification strategy, which specifically includes: if the values of the elements corresponding to the joint points of the patient's bilateral shoulder joints and bilateral hip joints in the visual skeleton sequence are not empty, calculate the trunk pitch angle based on the three-dimensional coordinates of the joint points corresponding to the patient's bilateral shoulder joints and bilateral hip joints; the second identification unit is also configured with a pitch angle threshold. If the trunk pitch angle of the patient is less than the pitch angle threshold, there is a risk of distortion of the blood oxygen probe. For example, the values of the elements corresponding to the patient's left shoulder, right shoulder, left hip, and right hip in the visual skeleton sequence are not empty, that is, the confidence levels of the joint points corresponding to both shoulders and both hips are greater than or equal to the confidence level threshold; calculate the trunk pitch angle based on the three-dimensional coordinates of the joint points corresponding to both shoulders and both hips;

[0082] In this embodiment, the preferred calculation formula for the trunk pitch angle is as follows:

[0083] ;

[0084] Wherein, represents the trunk pitch angle of the patient, is the arctangent function; is the coordinate of the joint point corresponding to the left shoulder along the z-axis; and are the coordinates of the joint point corresponding to the left shoulder along the x-axis and y-axis respectively; is the coordinate of the joint point corresponding to the left hip along the z-axis; and are the coordinates of the joint point corresponding to the left hip along the x-axis and y-axis respectively; wherein, the positive direction of the z-axis is vertically upward from the horizontal plane, that is is the distance between the joint points corresponding to the left shoulder and the left hip on the horizontal plane, is the height difference between the joint points corresponding to the left shoulder and the left hip; when the patient's posture gradually transitions from semi-reclining to supine to prone, the trunk pitch angle gradually decreases; when the trunk pitch angle is less than the pitch angle threshold, the patient's posture is prone. The prone posture exerts pressure on the limb end on the side of the blood oxygen probe, resulting in reduced blood perfusion and a risk of distortion of the detection data of the blood oxygen probe. Preferably, can also represent the midpoint coordinates of the joint points corresponding to both shoulders, and at the same time represents the midpoint coordinates of the joint points corresponding to both hips, thereby making the calculation of the trunk pitch angle more accurate.

[0085] The risk identification strategy further includes a second identification strategy, which specifically includes: if the values of the elements corresponding to the joint points of the patient's bilateral shoulder joints in the visual skeleton sequence are non-empty, calculate the shoulder height difference of the patient based on the three-dimensional coordinates of the joint points corresponding to the patient's bilateral shoulder joints; the shoulder height difference is the difference between the z-axis coordinate of the joint point corresponding to the shoulder joint on the non-probe side and the z-axis coordinate of the joint point corresponding to the shoulder joint on the blood oxygen probe side; the second identification unit is further configured with a height difference threshold, if the shoulder height difference of the patient is greater than the height difference threshold, there is a risk of distortion of the blood oxygen probe. For example, if the blood oxygen probe is connected to the patient's left hand finger, the left shoulder is the shoulder joint on the blood oxygen probe side, and the right shoulder is the shoulder joint on the non-probe side. Subtract the z-axis coordinate of the joint point corresponding to the left shoulder from the z-axis coordinate of the joint point corresponding to the right shoulder to obtain the shoulder height difference; if the shoulder height difference is greater than the height difference threshold, the patient lies on the left side, which compresses the left hand connected to the blood oxygen probe at this time, reducing blood perfusion and causing a risk of distortion of the detection data of the blood oxygen probe.

[0086] The risk identification strategy further includes a third identification strategy, which specifically includes: if the values of the elements corresponding to the joint points of at least one relevant joint on the blood oxygen probe side in the visual skeleton sequence are non-empty, obtain the pressure sensing data of the relevant joint based on the three-dimensional coordinates of the joint points corresponding to the relevant joint and the pressure thermogram; the second identification unit is further configured with a pressure threshold, if the pressure sensing data of at least one relevant joint is greater than the pressure threshold, there is a risk of distortion of the blood oxygen probe. For example, if the blood oxygen probe is connected to the patient's left hand finger, the relevant joints include the left shoulder, left elbow, and left wrist; since the pressure thermogram and the visual skeleton sequence are spatio-temporally aligned, the position of any relevant joint in the pressure thermogram can be located based on its three-dimensional coordinates, and the pressure sensing data can be read from the pressure thermogram accordingly. If the pressure sensing data of at least one of the left shoulder, left elbow, and left wrist is greater than the pressure threshold, there is a risk of compression of the left hand, causing a risk of distortion of the detection data of the blood oxygen probe.

[0087] The adjustment module corrects and compensates the monitoring device based on the distortion risk of the monitoring device.

[0088] The adjustment module includes a first correction unit and a second correction unit;

[0089] The first correction unit corrects and compensates the electrocardiogram electrode based on the distortion risk of the electrocardiogram electrode, specifically including:

[0090] Monitor the peak-to-peak amplitude of the QRS wave of each risk lead; the first correction unit is configured with an amplitude threshold range; if the peak-to-peak amplitude of the QRS wave of any risk lead is not within the amplitude threshold range, mark the corresponding risk lead as a distorted lead; select and enable the spare lead corresponding to the distorted lead from the spare lead group, and deactivate the distorted lead. For example, when the patient lies on the left side, the change in thoracic pressure distribution causes poor electrode contact or signal distortion. At this time, based on the distortion prediction model and the distortion probability threshold, the left chest leads (V4-V6) located on the left ventricular lateral wall of the heart are determined as risk leads, and the peak-to-peak amplitude of their QRS waves is not within the amplitude threshold range, then enable the corresponding spare lead to replace the left chest leads.

[0091] The second correction unit corrects and compensates the blood oxygen probe based on the distortion risk of the blood oxygen probe, specifically including:

[0092] When there is a distortion risk in the blood oxygen probe, perform distortion verification on the blood oxygen probe, specifically including: separating the DC component and the AC component of the red light from the detection signal of the blood oxygen probe; calculating the AC / DC ratio of the red light, that is, the ratio of the AC component to the DC component of the red light; the second correction unit is configured with an AC / DC ratio reference value and a stability threshold; calculate the ratio of the AC / DC ratio of the red light to the AC / DC ratio reference value, denoted as the stability of the red light; if the stability of the red light is less than the stability threshold, adjust the emission ratio of the red light to the infrared light based on the stability of the red light, so as to correct and compensate the blood oxygen probe.

[0093] The blood oxygen probe calculates the blood oxygen saturation based on the difference in the absorption characteristics of different wavelength lights by oxyhemoglobin and deoxyhemoglobin. Specifically, the blood oxygen saturation is calculated by the AC / DC ratio of the collected red light and the AC / DC ratio of the infrared light. When the local blood flow in the limb where the blood oxygen probe is located decreases and the concentration of deoxyhemoglobin increases, the AC component of the red light attenuates significantly, resulting in distorted calculation of the blood oxygen saturation. Dynamically adjust the emission ratio of the red light to the infrared light, enhance the red light power to compensate for its signal loss, so as to offset the absorbance deviation caused by reduced low blood perfusion, and make the calculation result of the blood oxygen saturation closer to the true value. For example, set the basic emission ratio (power ratio) of the red light to the infrared light to 1:11. When the blood oxygen probe is distorted, adjust the emission ratio based on the stability of the red light, and the lower the stability of the red light, the higher the emission ratio. Limit the maximum emission ratio to 2:1 to prevent the red light power from exceeding the safety limit.

[0094] The application scenarios of the comprehensive visualization Internet of Things management system introduced in this embodiment are as Figure 2 shown.

[0095] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an all-hardware embodiment, an all-software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0096] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Those of ordinary skill in the art, under the inspiration of the present application and without departing from the purpose and scope protected by the present application, can still make many forms, and all of these are within the protection scope of the present application.

Claims

1. A comprehensive visual Internet of Things management system, characterized in that: It includes a data acquisition module, a processing module, a risk identification module, and an adjustment module; among them: The data acquisition module is used to acquire hospital bed data; the hospital bed data includes the monitoring image of the hospital bed and the pressure sensing data; The processing module extracts visual features based on the hospital bed data; the visual features include a visual skeleton sequence and a pressure heat map; The risk identification module identifies the distortion risk of the monitoring device based on the visual features; identifying the distortion risk of the monitoring device includes identifying the distortion risk of the electrocardiogram electrode and the distortion risk of the blood oxygen probe; The adjustment module corrects and compensates the monitoring device based on the distortion risk of the monitoring device.

2. The comprehensive visual IoT management system according to claim 1, characterized in that: The data acquisition module includes a monitoring unit and a pressure array unit; among them, the monitoring unit is used to acquire the monitoring image of the hospital bed; the monitoring image of the hospital bed includes a planar image and a depth image of the hospital bed; The pressure array unit is used to acquire the pressure sensing data of the hospital bed; the pressure array unit is configured with a piezoresistive sensor array, and the pressure array unit acquires the pressure sensing data of each position on the hospital bed in real time based on the piezoresistive sensor array.

3. The comprehensive visualization Internet of Things management system according to claim 2, characterized in that: The processing module includes an image recognition unit; the image recognition unit extracts a visual skeleton sequence based on the monitoring image of the hospital bed; specifically including: Performing coordinate registration on the depth image and the planar image to obtain a fusion matrix; the fusion matrix includes the pixel value and the depth value of each pixel point in the monitoring image of the hospital bed; Inputting the fusion matrix into a joint point detection model to obtain the three-dimensional coordinates of each joint point of the patient and the confidence level of each joint point; The image recognition unit is configured with a confidence threshold, and constructs a visual skeleton sequence based on the confidence threshold; Each element in the visual skeleton sequence corresponds to a joint point of the patient; among them, if the confidence level of any joint point is greater than or equal to the confidence threshold, the value of the corresponding element in the visual skeleton sequence is the three-dimensional coordinates of the joint point; if the confidence level of any joint point is less than the confidence threshold, the value of the corresponding element in the visual skeleton sequence is empty.

4. The comprehensive visualization Internet of Things management system according to claim 3, characterized in that: The processing module further includes a data processing unit; the data processing unit draws a pressure heat map based on the pressure sensing data of the hospital bed; specifically including: Sorting the pressure sensing data of each position on the hospital bed acquired by the piezoresistive sensor array into a pressure sensing matrix; each element in the pressure sensing matrix corresponds to a pressure sensing data, and the position of the element in the pressure sensing matrix is the position of the piezoresistive sensor corresponding to the pressure sensing data in the piezoresistive sensor array; performing normalization processing on the element values in the pressure sensing matrix; mapping the pressure sensing matrix into a pressure heat map.

5. The all-round visual IoT management system according to claim 4, characterized in that: The risk identification module includes a first identification unit; the first identification unit is used to identify the distortion risk of the electrocardiogram electrode; the electrocardiogram electrode includes a standard lead group and a spare lead group; identifying the distortion risk of the electrocardiogram electrode specifically includes: Performing spatio-temporal alignment on the pressure heat map and the visual skeleton sequence; Inputting the pressure heat map and the visual skeleton sequence into a trained distortion prediction model; the distortion prediction model calculates and outputs the distortion risk probability of each lead in the standard lead group; The first recognition unit is configured with a distortion probability threshold; if the distortion risk probability of any lead is greater than the distortion probability threshold, the corresponding lead is marked as a risk lead.

6. The comprehensive visual IoT management system according to claim 5, wherein: The distortion prediction model includes an input layer, a feature extraction layer, a multi-modal fusion layer, a prediction layer, and an output layer; where: The input layer is used to receive the pressure heat map and the visual feature skeleton as the model input; The feature extraction layer includes a pressure feature sub-layer and a pose feature sub-layer; wherein, the pressure feature sub-layer is used to process the pressure heat map and output a pressure semantic vector; the pose feature sub-layer is used to process the visual feature skeleton and output a pose semantic vector; The multi-modal fusion layer generates a joint feature vector based on the pressure semantic vector and the pose semantic vector; The prediction layer calculates the distortion risk vector of the lead based on the joint feature vector; The output layer maps the distortion risk vector of the lead into the distortion risk probability of each lead.

7. The comprehensive visualization Internet of Things management system according to claim 6, characterized in that: The risk recognition module further includes a second recognition unit; the second recognition unit is configured with a risk recognition strategy; The second recognition unit identifies the distortion risk of the blood oxygen probe based on any one of the risk recognition strategies; The risk recognition strategy includes a first recognition strategy, and the first recognition strategy specifically includes: if the values of the elements corresponding to the joint points of the patient's bilateral shoulder joints and bilateral hip joints in the visual skeleton sequence are non-empty, then calculate the torso pitch angle of the patient based on the three-dimensional coordinates of the joint points corresponding to the patient's bilateral shoulder joints and bilateral hip joints; The second recognition unit is further configured with a pitch angle threshold, and if the torso pitch angle of the patient is less than the pitch angle threshold, there is a distortion risk for the blood oxygen probe.

8. The comprehensive visualization Internet of Things management system according to claim 7, characterized in that: The risk recognition strategy further includes a second recognition strategy, and the second recognition strategy specifically includes: if the values of the elements corresponding to the joint points of the patient's bilateral shoulder joints in the visual skeleton sequence are non-empty, then calculate the shoulder height difference of the patient based on the three-dimensional coordinates of the joint points corresponding to the patient's bilateral shoulder joints; the shoulder height difference is the difference between the z-axis coordinates of the joint points corresponding to the shoulder joint on the non-probe side and the z-axis coordinates of the joint points corresponding to the shoulder joint on the blood oxygen probe side; the second recognition unit is further configured with a height difference threshold, and if the shoulder height difference of the patient is greater than the height difference threshold, there is a distortion risk for the blood oxygen probe; The risk recognition strategy further includes a third recognition strategy, and the third recognition strategy specifically includes: if the values of the elements corresponding to the joint points of at least one relevant joint on the blood oxygen probe side in the visual skeleton sequence are non-empty, then obtain the pressure sensing data of the relevant joint based on the three-dimensional coordinates of the joint points corresponding to the relevant joint and the pressure heat map; the second recognition unit is further configured with a pressure threshold, and if the pressure sensing data of at least one relevant joint is greater than the pressure threshold, there is a distortion risk for the blood oxygen probe.

9. The all-round visual Internet of Things management system according to claim 8, characterized in that: The adjustment module includes a first correction unit; the first correction unit performs correction compensation on the electrocardiogram electrode based on the distortion risk of the electrocardiogram electrode, specifically including: Monitor the peak-to-peak amplitude of the QRS wave of each risk lead; the first correction unit is configured with an amplitude threshold range; if the peak-to-peak amplitude of the QRS wave of any risk lead is not within the amplitude threshold range, mark the corresponding risk lead as a distorted lead; select and enable the spare lead corresponding to the distorted lead from the spare lead group, and deactivate the distorted lead.

10. A comprehensive visualization Internet of Things management system as described in claim 9, characterized in that: The adjustment module further includes a second correction unit; the second correction unit corrects and compensates the blood oxygen probe based on the distortion risk of the blood oxygen probe, specifically including: When there is a distortion risk in the blood oxygen probe, separate the DC component and the AC component of the red light from the detection signal of the blood oxygen probe; calculate the AC / DC ratio of the red light; the second correction unit is configured with an AC / DC ratio reference value and a stability threshold; calculate the ratio of the AC / DC ratio of the red light to the AC / DC ratio reference value, denoted as the stability of the red light; if the stability of the red light is less than the stability threshold, adjust the emission ratio of the red light and the infrared light based on the stability of the red light, so as to correct and compensate the blood oxygen probe.

Images