Target recognition method and system in emergency intensive care unit based on Internet of Things

Abstract

The present invention relates to the field of target recognition technology, and in particular, to a method and system for target recognition in an emergency intensive care unit based on the Internet of Things. The method comprises the following steps: using a camera to capture a video stream in the emergency intensive care unit at a fixed frame rate and extracting image frames; using a YOLOv7 model to perform real-time target detection on each frame in the image frame, identifying the target patient, associating targets in consecutive frames of the video stream using the DeepSORT algorithm, and tracking the position of the target patient in combination with the Kalman filter, and constructing a four-dimensional spatiotemporal-semantic collaborative cost matrix for optimization in the process of tracking the position of the target patient; using a 3D CNN to analyze the action sequence of the target patient in the video stream, and combining the physiological data of the target patient to determine abnormalities. The design of the present invention enhances the detection capability of moving targets in complex scenarios in the YOLOv7 model by introducing a cross-frame attention mechanism and an occlusion density-guided feature fusion strategy.

Description

Technical Field

[0001] The present invention relates to the technical field of target recognition, and in particular to a target recognition method and system in an emergency intensive care unit based on the Internet of Things. Background Art

[0002] The emergency intensive care unit (ICU) is a rapidly changing, densely populated environment with a high density of personnel and equipment. In such an environment, traditional monitoring systems struggle to effectively distinguish between medical staff and patients, especially when there are frequent human occlusions and rapid movement, which can easily lead to target loss or misidentification. Emergency scenarios require extremely fast response times, and any delay can have serious consequences. Single video stream analysis may not provide sufficient information to fully understand the patient's condition. In high-density target scenarios, efficient utilization of computing resources is necessary to ensure stable system operation. Traditional methods can lead to excessive computational loads due to the simultaneous presence of a large number of targets, affecting the system's response speed and accuracy. Therefore, this paper proposes an IoT-based target recognition method and system for ICU ICUs. Summary of the Invention

[0003] The purpose of the present invention is to provide a target recognition method and system for an emergency intensive care unit based on the Internet of Things, so as to solve the problems of inaccurate target recognition, unstable tracking and untimely emergency response in the emergency intensive care unit due to the complex environment, high real-time requirements and difficulty in multimodal data fusion raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides an Internet of Things-based target recognition method for an emergency intensive care unit, comprising the following steps:

[0005] S1, collects the video stream in the emergency intensive care unit through a camera at a fixed frame rate;

[0006] S2, preprocessing the collected video stream and extracting image frames;

[0007] S3: Use the YOLOv7 model to perform real-time target detection on each image frame to identify the target patient. Use the DeepSORT algorithm to correlate targets in consecutive frames of the video stream and combine it with the Kalman filter to track the target patient's position. During the tracking process, a four-dimensional spatiotemporal-semantic collaborative cost matrix is ​​constructed for optimization.

[0008] S4. Analyze the action sequence of the target patient in the video stream through 3D CNN and judge abnormalities based on the physiological data of the target patient.

[0009] As a further improvement of the present technical solution, in S3, a YOLOv7 model is used to perform real-time target detection on each frame in the image frame to identify the target patient, including the following steps:

[0010] S3.1. Input the image frame into the YOLOv7 model for forward propagation, and output the location, confidence, and category score of each predicted box of the target patient;

[0011] S3.2. Use non-maximum suppression to remove overlapping prediction boxes and filter out targets with confidence higher than the threshold a.

[0012] S3.3. Among all the prediction boxes that remain after screening, further analyze the category information of each prediction box, identify the target patients, and output the final test result list.

[0013] As a further improvement of this technical solution, in S3.1, the image frame is input into the YOLOv7 model for forward propagation, and the position, confidence, and category score of each predicted box of the target patient are output, including the following steps:

[0014] S3.11. In the backbone of the YOLOv7 model, for each residual block, the feature maps corresponding to the past three frames are retained during forward propagation to build a feature buffer pool. The feature map of the current frame is dot-producted with the feature maps of the previous frames to obtain a similarity matrix. The historical features are weighted and summed using the attention weights.

[0015] S3.12. Use a depth camera to obtain depth information within the emergency intensive care unit and calculate the degree of occlusion of the local area based on the depth map;

[0016] S3.13. Use the PANet structure of the YOLOv7 model to perform multi-scale feature fusion. Input the multi-scale fused features into the detection head of the YOLOv7 model, and output a prediction tensor containing the bounding box offset, target confidence, and category score. Use the anchor box mechanism to decode the offset into the actual box position in the image coordinate space, and calculate the final confidence value by combining the target confidence and category score.

[0017] As a further improvement of the present technical solution, in S3.3, among all the prediction boxes retained after screening, further analyzing the category information of each prediction box to identify the target patient includes the following steps:

[0018] S3.31, traverse each prediction box and extract category information;

[0019] S3.32, mapping the category ID to a semantic label using a predefined category mapping table;

[0020] S3.33, filter out the prediction boxes whose categories are patients;

[0021] S3.34. Number the identified target patients and assign a unique ID to each identified patient;

[0022] S3.35. Output the identified target patient list as the final test result list.

[0023] As a further improvement of the present technical solution, in S3, the DeepSORT algorithm is used to associate targets in consecutive frames of the video stream, and the Kalman filter is combined to track the position of the target patient, including the following steps:

[0024] S3.4. Input the detection box coordinates output by YOLOv7 into the DeepSORT model. For each object detected by the YOLOv7 model in each frame, crop the bounding box area of ​​the object and use the ReID sub-network to extract the appearance ReID embedding feature vector of the object.

[0025] S3.5. Initialize the Kalman filter for each detected target to predict its state vector in the next frame;

[0026] S3.6. Use the state estimate obtained by Kalman filter prediction and the cosine distance between the appearance ReID embedding feature vectors to construct a four-dimensional spatiotemporal-semantic collaborative cost matrix.

[0027] S3.7. Perform minimum weight matching on the four-dimensional spatiotemporal-semantic collaborative cost matrix based on the Hungarian algorithm, associate the detection results in the current frame and the previous frame of the video stream, and pre-filter matching candidate pairs through the medical behavior causal graph;

[0028] S3.8. Output the tracking result with the target patient ID.

[0029] As a further improvement to this technical solution, in S3.6, the state estimate obtained by Kalman filter prediction is combined with the cosine distance between ReID embedding vectors to construct a four-dimensional spatiotemporal-semantic collaborative cost matrix, including the following steps:

[0030] S3.61. Combine the texture features of the patient's hospital gown and the device connection status to generate a hybrid ReID feature. Calculate the cosine similarity between the current frame and the target in the historical trajectory. The initial weight is set to b.

[0031] S3.62. Calculate the Mahalanobis distance between targets using the Kalman filter prediction results;

[0032] S3.63, checking whether the target detected in the current frame carries a specific medical device, and comparing the device connection status with the device connection status in the historical trajectory. If the same device is detected in both the current frame and the historical trajectory, increasing the confidence of the matching pair;

[0033] S3.64, combining the RFID wristband positioning signal, if the Euclidean distance between the detection frame center and the RFID coordinates is less than 0.5 meters, then increase the confidence level of the matching pair;

[0034] S3.65, when it is detected that the patient's continuous frame acceleration is greater than the threshold x, reduce the appearance weight and increase the motion weight;

[0035] S3.66. Based on the results of the above steps, a four-dimensional cost matrix is ​​constructed. The rows in the cost matrix represent each detection box in the current frame, and the columns in the cost matrix represent each tracking trajectory in the previous frame.

[0036] As a further improvement of the present technical solution, in S3.7, minimum weight matching is performed on the four-dimensional spatiotemporal-semantic collaborative cost matrix based on the Hungarian algorithm, the detection results in the current frame and the previous frame of the video stream are associated, and matching candidate pairs are pre-filtered through the medical behavior causal graph, including the following steps:

[0037] S3.71. Construct a causal map of medical behavior in the emergency intensive care unit to analyze the action intentions of medical staff in real time;

[0038] S3.72. For the target patient carrying emergency equipment, construct an independent sub-cost matrix;

[0039] S3.73. Based on the conventional Hungarian algorithm, a detection frame in the current frame and a tracking track in the previous frame are jointly verified in terms of spatial position, device status, and vital signs.

[0040] S3.74, dynamically set the matching threshold based on the regional congestion;

[0041] S3.75. Output the final matching result with priority control.

[0042] As a further improvement of the present technical solution, in S4, the action sequence of the target patient in the video stream is analyzed by 3D CNN, and abnormalities are determined in combination with the physiological data of the target patient, including the following steps:

[0043] S4.1. Extract a 3D space-time cube of continuous time windows from the video stream based on the tracking ID of the target patient, and synchronously collect the physiological data of the target patient in the corresponding time period;

[0044] S4.2. Using the 3D space-time cube as input to the 3D CNN model, extracting video features in spatial and temporal dimensions through the convolutional layer of the 3D CNN model, and outputting a target patient motion feature vector;

[0045] S4.3. Use continuous wavelet transform to convert the physiological data of the target patient into a time-frequency graph, use the time-frequency graph as the input of the 2DCNN model, extract frequency domain features, and introduce a causal mask;

[0046] S4.4. Use the cross-attention mechanism to calculate the joint feature vector of the target patient's motion features and frequency domain features, perform abnormality classification based on the joint feature vector through the MLP classifier, and trigger the alarm mechanism.

[0047] As a further improvement of the present technical solution, in S4.4, a cross-attention mechanism is used to calculate a joint feature vector of the target patient's motion features and frequency domain features, including the following steps:

[0048] S4.41. Use cosine similarity to measure the cosine value of the angle between the target patient's motion feature vector and the frequency domain feature vector to calculate the similarity score;

[0049] S4.42. Applying a Softmax function to the similarity score to convert the similarity score into a probability distribution;

[0050] S4.43. Based on the similarity score, perform weighted summation on the action feature vector and the frequency domain feature vector to form a new joint feature vector.

[0051] On the other hand, the present invention provides an emergency intensive care unit target identification system based on the Internet of Things, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-mentioned emergency intensive care unit target identification method based on the Internet of Things.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] 1. This IoT-based target recognition method and system for emergency ICUs enhances the detection capability of moving targets (especially patients) in complex scenarios within the YOLOv7 model by introducing a cross-frame attention mechanism and an occlusion density-guided feature fusion strategy. This effectively mitigates the problems of missed and false detections caused by factors such as medical staff occlusion and lighting variations. Furthermore, by combining spatial information acquired by a depth camera with a medical device status verification mechanism, a four-dimensional spatiotemporal-semantic collaborative cost matrix is ​​constructed, and high-precision target tracking is achieved through an improved DeepSORT algorithm. Furthermore, the use of a medical behavior causal graph for matching pre-filtering further enhances the tracking system's logical reasoning and anti-interference capabilities, ensuring that the ID continuity and tracking stability of key targets (such as critically ill patients) are maintained even in high-density or dynamic scenarios such as emergency care.

[0054] 2. This IoT-based method and system for identifying objects in emergency intensive care units (ICUs) utilizes a 3D CNN to extract temporal features of patient movements. Combined with the time-frequency features generated by continuous wavelet transform of physiological data (heart rate, blood pressure, and blood oxygen), this multimodal information is fused and analyzed using a cross-attention mechanism, enabling a more comprehensive understanding of the patient's health status. In particular, the introduction of a causal masking mechanism prevents future information leakage and dynamically adjusts the length of historical dependencies based on the patient's status, thereby enhancing the system's responsiveness to emergencies. Finally, by setting an associated weight threshold to trigger an alarm mechanism, real-time perception and intelligent judgment of abnormal patient behavior (such as falls and violent struggles) and sudden changes in physiological indicators are achieved, providing a reliable basis for timely clinical intervention and improving the efficiency and safety of emergency ICU care. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 The figure is a flow chart of the overall method of the present invention. DETAILED DESCRIPTION

[0056] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0057] Example 1: Please refer to Figure 1 As shown, this embodiment provides a method for identifying targets in an emergency intensive care unit based on the Internet of Things, comprising the following steps:

[0058] S1. Capture the video stream in the ICU with a fixed frame rate (≥30fps) using a camera. Use high-definition network cameras and deploy multiple monitoring nodes in the ICU. Set a fixed frame rate (≥30fps) to ensure the continuity and accuracy of motion capture. The video data is transmitted in real time to the edge computing device or cloud server via the local area network.

[0059] S2. Preprocess the captured video stream (including image enhancement (such as denoising, contrast adjustment, and illumination compensation)) and extract image frames (each frame is input into the YOLOv7 model as an independent image for analysis).

[0060] S3: Use the YOLOv7 model to perform real-time target detection on each frame of the image, identify the target patient, associate the targets in consecutive frames of the video stream using the DeepSORT algorithm, and combine it with the Kalman filter to track the target patient's position;

[0061] In this embodiment, the YOLOv7 model consists of an optimized backbone network (CSPDarknet), a cross-frame attention module, a feature pyramid network, a path aggregation network, a detection head, and an anchor box mechanism, aiming to achieve efficient and accurate object detection;

[0062] The YOLOv7 model is used to perform real-time target detection on each image frame to identify the target patient, including the following steps:

[0063] S3.1. Input the image frame into the YOLOv7 model for forward propagation, and output the location, confidence, and category score of each predicted box of the target patient;

[0064] Among them, this solution is a feature enhancement scheme that jointly optimizes spatiotemporal attention and occlusion density. Specifically, the cross-frame attention mechanism uses temporal information to enhance the current frame features, solving the problem of single-frame detection being sensitive to instantaneous occlusions, and combining the occlusion density of the depth sensor to guide feature fusion. The feature enhancement scheme that jointly optimizes spatiotemporal attention and occlusion density effectively solves problems such as dense targets, frequent occlusions, and complex semantics. This scheme combines the local occlusion density perception mechanism with the global spatiotemporal attention mechanism, can dynamically perceive the occlusion intensity between targets, and perform specific compensation for occluded areas during the feature extraction process, improving the expressiveness of target boundaries and detailed features. At the same time, cross-frame spatiotemporal attention strengthens the modeling of target identity consistency between consecutive frames, giving the model strong robustness and re-identification capabilities against short-term occlusions and the reappearance of targets after occlusion. In addition, this scheme can effectively model the behavioral semantics and dynamic change characteristics of the target, helping to distinguish the differences in micro-movements between medical staff and patients and improve the accuracy of abnormal state recognition. Overall, this feature enhancement mechanism significantly improves the stability, accuracy, and practicality of multi-target recognition and tracking systems in complex medical scenarios.

[0065] The image frame is input into the YOLOv7 model for forward propagation, and the location, confidence, and category score of each predicted box of the target patient are output. The following steps are included:

[0066] S3.11. In the YOLOv7 backbone, for each residual block, the feature maps corresponding to the past three frames (layers C3 / C4 / C5) are retained during forward propagation. A feature buffer pool is constructed (each time a new frame is processed, the feature maps corresponding to each selected layer are added to the corresponding buffer pool). The feature map of the current frame is dot-producted with the feature maps of the previous frames to obtain a similarity matrix (this operation seeks the correlation or consistency of feature maps between different frames). The historical features are weighted and summed using attention weights to enhance the key feature representation of the moving object in the current frame. Specifically, a cross-frame attention module (CFAM) is introduced in each residual block of the YOLOv7 backbone network (CSPDarknet).

[0067] S3.12. Depth information within the emergency ICU is acquired using a depth camera synchronized with the RGB camera. The occlusion level of the local area is calculated based on the depth map (based on neighborhood depth differences or gradient changes). In areas with high occlusion (where multiple medical staff gather to interfere), low-level details are retained to maintain positioning accuracy. In areas with low occlusion, high-level semantic features are emphasized to enhance classification capabilities.

[0068] S3.13. Use the PANet structure of the YOLOv7 model to perform multi-scale feature fusion (P3, P4, P5) to enhance the detection performance of small and large objects. The multi-scale fused features are input into the detection head of the YOLOv7 model (the detection head is a collection of one or more network layers that receive feature maps from the backbone network (Backbone) and the Feature Pyramid Network (FPN)). The output is a prediction tensor containing the bounding box offset, object confidence, and category score. The anchor box mechanism is used to decode the offset into the actual box position in the image coordinate space. The final confidence value is calculated by combining the object confidence and category score to obtain high-quality and accurate prediction boxes.

[0069] S3.2. Use non-maximum suppression to remove overlapping prediction boxes and filter out targets with confidence higher than the threshold a.

[0070] S3.3. Further analyze the category information of each prediction box among all the prediction boxes retained after screening, identify the target patients, and output the final test result list;

[0071] Among all the prediction boxes retained after screening, the category information of each prediction box is further analyzed to identify the target patient, including the following steps:

[0072] S3.31, traverse each prediction box and extract category information;

[0073] S3.32, mapping the category ID to a semantic label using a predefined category mapping table;

[0074] S3.33. Filter out the prediction boxes whose category is patient, set a conditional judgment, and only keep the prediction boxes whose category is patient;

[0075] S3.34. Number the identified target patients and assign a unique ID to each identified patient to facilitate subsequent tracking and behavioral analysis;

[0076] S3.35. Output the identified target patient list as the final test result list.

[0077] Furthermore, the DeepSORT algorithm is used to associate the targets in the consecutive frames of the video stream, and the Kalman filter is combined to track the target patient position, including the following steps:

[0078] S3.4. Input the detection box coordinates output by YOLOv7 into the DeepSORT model. For each object detected by the YOLOv7 model in each frame, crop the bounding box area of ​​the object and use the ReID sub-network (a modified version of MobileNet or ResNet-18) to extract the object's appearance ReID embedding feature vector. This vector has good discriminability and is used to measure the appearance similarity between different objects.

[0079] S3.5. Initialize the Kalman filter for each detected target to predict its state vector in the next frame;

[0080] S3.6. Using the state estimate obtained by Kalman filter prediction and the cosine distance between the appearance ReID embedding feature vectors, a four-dimensional spatiotemporal-semantic collaborative cost matrix is ​​constructed for target matching.

[0081] This solution breaks through the traditional two-dimensional (appearance + motion) cost calculation and introduces device connection status and spatial verification. In complex, high-density emergency monitoring scenarios, targets are prone to occlusion, intersection, and aggregation. The traditional two-dimensional (appearance + motion) matching mechanism is difficult to cope with these challenges. The construction of a four-dimensional spatiotemporal-semantic collaborative cost matrix significantly improves the accuracy of target association by introducing more dimensions of information (device status, spatial verification, etc.). The four-dimensional spatiotemporal-semantic collaborative cost matrix solves the ID jump problem caused by target occlusion and aggregation in emergency scenarios by integrating the four dimensions of spatiotemporal (Kalman prediction), appearance (ReID), device status (semantics), and spatial verification (RFID).

[0082] Using the state estimate predicted by the Kalman filter and the cosine distance between the ReID embedding vectors, a four-dimensional spatiotemporal-semantic collaborative cost matrix is ​​constructed, which includes the following steps:

[0083] S3.61. Combine the texture features of the patient's gown and the device connection status (whether an IV tube or ECG monitor is present) to generate a hybrid ReID feature. Calculate the cosine similarity between the current frame and the target in the historical trajectory, with an initial weight of b (0.5 in this example).

[0084] S3.62. Calculate the Mahalanobis distance between targets using the Kalman filter prediction results (the Kalman filter state vector contains the target's position and velocity).

[0085] S3.63. Check whether the target detected in the current frame is carrying a specific medical device (infusion tube) and compare it with the device connection status in the historical trajectory. If the same device is detected in both the current frame and the historical trajectory, increase the confidence score of the matching pair with a weight of +0.2;

[0086] S3.64. Combined with the RFID wristband positioning signal, if the Euclidean distance between the detection frame center and the RFID coordinates is less than 0.5 meters, increase the confidence level of the matching pair (if the condition is met, the confidence level is multiplied by 1.2);

[0087] S3.65. When the patient's continuous frame acceleration is detected to be greater than the threshold x (i.e., the acceleration of the target patient when falling), reduce the appearance weight to 0.3 and increase the motion weight to 0.5, prioritizing motion continuity to complete target association in the emergency state;

[0088] S3.66. Based on the results of the above steps, a four-dimensional cost matrix is ​​constructed. The rows in the cost matrix represent each detection box in the current frame, and the columns in the cost matrix represent each tracking track in the previous frame.

[0089] Among them, each element represents the total cost between a pair of candidate matches (i.e., a detection box in the current frame and a tracking track in the previous frame);

[0090] S3.7. Perform minimum weight matching on the four-dimensional spatiotemporal-semantic collaborative cost matrix based on the Hungarian algorithm, associate the detection results in the current frame of the video stream with the previous frame, and pre-filter matching candidate pairs using the medical behavior causal graph (e.g., blocking associations that violate clinical operation logic). Optimize the accuracy of target ID assignment (if a match is successful, the matching target ID is retained; if a detection box is not matched, it is initialized as a new target and assigned a new ID; if a trajectory is not matched for N consecutive frames, it is marked as "Lost" or "Deleted").

[0091] Furthermore, this solution converts clinical operation logic into matching rules through the medical behavior causal map, solves the problem of insufficient understanding of medical scenarios by general algorithms (target aggregation caused by sudden emergency treatment), separates the matching paths of emergency targets and regular targets through a hierarchical two-way mechanism, and ensures zero-delay tracking of critical patients; in the emergency monitoring scenario, the behavior and location of patients, medical staff, and equipment follow the clinical operation logic (patients are usually located in the bed area, and medical staff need to be close to the equipment cabinet before operating the equipment). If it is detected that the predicted location of patient A suddenly appears next to the equipment cabinet (the distance from the bed is >2 meters), and medical staff B is operating the equipment, the matching of patient A and the equipment cabinet area is blocked to avoid misassociation. If patient C's vital signs are abnormal (blood oxygen drops sharply) and there is a defibrillator nearby If medical staff D is present in the device, patient C is preferentially associated with medical staff D. Pre-filtering eliminates obviously unreasonable candidate pairs, reducing the number of matching pairs that the Hungarian algorithm needs to process, accelerating the matching process and reducing resource consumption (especially in high-density target scenarios). In highly dynamic scenarios such as emergency care (where multiple people surround a patient), traditional algorithms are prone to ID jumps due to target aggregation. The causal graph maintains the ID continuity of key targets (patients) through clinical rule constraints. Relying solely on the cost matrix of appearance (ReID) and motion (Kalman filtering) may produce logically conflicting matches (such as patients instantly "teleporting" to unreasonable areas). The pre-filtering stage directly eliminates candidate pairs that violate medical behavior rules, generating a cost matrix that has been cleansed by clinical logic, improving the input quality of the Hungarian algorithm.

[0092] Based on the Hungarian algorithm, the minimum weight matching of the four-dimensional spatiotemporal-semantic collaborative cost matrix is ​​performed, the detection results in the current frame of the video stream are associated with the previous frame, and the matching candidate pairs are pre-filtered through the medical behavior causal graph, including the following steps:

[0093] S3.71. Build a causal graph of medical behavior in the emergency intensive care unit (ICU) to analyze the medical staff's action intentions in real time. If a medical staff member is detected holding a syringe (visual recognition) and moving toward a bed (trajectory prediction), the association between the medical staff member and the corresponding bed patient is prioritized. If the patient's vital signs suddenly change (blood oxygen level drops), the ID is forced to be retained and the matching priority is increased.

[0094] Define nodes (target type, action, device, region) and edges (causality, dependency):

[0095] The causal graph of this embodiment is:

[0096] “Medical staff holding a syringe → approaching the bed → matching the patient in the bed”;

[0097] “Defibrillator is unlocked → ECG monitoring abnormality is detected → Prioritize matching with nearby patients”;

[0098] "Patient's blood oxygen level drops → visual features blur → forced ID retention and increased matching weight";

[0099] S3.72. For target patients carrying emergency equipment (defibrillators) or with abnormal vital signs, construct an independent sub-cost matrix and prioritize high-weight matching to ensure zero-delay tracking of critical targets.

[0100] S3.73. Based on the conventional Hungarian algorithm, a detection box in the current frame and a tracking trajectory in the previous frame are jointly verified in terms of spatial position, device status, and vital signs. Only logical matching pairs (patient-bed binding, medical staff-equipment operation permission matching) are allowed to enter the cost matrix calculation, blocking associations that violate clinical rules (patients instantly "teleporting" to non-bed areas, unauthorized personnel accessing high-risk equipment). Specifically, if patient A's predicted position conflicts with the bed topology (instantly moving to the next bed), the abnormal match is blocked. If medical staff B's ​​movement direction conflicts with the equipment access logic (walking towards the exit without returning the equipment), manual review is triggered.

[0101] S3.74. Dynamically set the matching threshold based on the area crowding (calculated by the depth sensor). In high-density areas (>3 people / ㎡), the appearance similarity threshold is relaxed (0.7→0.5), focusing on motion continuity. In low-density areas, a strict threshold (0.8) is used to avoid false associations at long distances.

[0102] S3.75, output the final matching result with priority control;

[0103] S3.8. Output the tracking result with the target patient ID.

[0104] S4, using 3D CNN to analyze the target patient's action sequence in the video stream and determine abnormalities based on the target patient's physiological data;

[0105] In this embodiment, the motion sequence of the target patient in the video stream is analyzed by 3D CNN, and abnormalities are determined based on the physiological data of the target patient, including the following steps:

[0106] S4.1. Based on the target patient's tracking ID, extract a 3D space-time cube of a continuous time window (16 frames / 2 seconds in this example) from the video stream. Simultaneously collect the target patient's physiological data (heart rate, blood pressure, blood oxygen level) within the corresponding time period and normalize the video stream.

[0107] S4.2. Using the 3D space-time cube as the input of the 3D CNN model, extracting video features in the spatial and temporal dimensions through the convolutional layer of the 3D CNN model, and outputting a target patient motion feature vector representing the target patient's behavior pattern within the time window;

[0108] S4.3. Use continuous wavelet transform (CWT) to convert the target patient's physiological data (heart rate, blood pressure, blood oxygen) into a time-frequency graph. This graph is used as input to the 2D CNN model to extract frequency domain features. A causal mask is introduced to ensure that predictions rely solely on historical information to prevent future data leakage. If the patient is in a stable state (normal heart rate), a standard causal mask is used. If a sudden abnormality is detected (a sudden increase in heart rate), the mask restriction is temporarily relaxed to allow the model to reference earlier historical data (features from the past 5 minutes), enhancing its ability to respond to emergencies.

[0109] S4.4. Use a cross-attention mechanism to calculate a joint feature vector of the target patient's motion and frequency domain features (unifying the motion and frequency domain features into the same dimensional space). Based on this joint feature vector, perform abnormality classification using an MLP classifier (specifically, input the fused joint feature vector into a multi-layer perceptron (MLP), which, after nonlinear transformation, outputs the predicted probabilities of various abnormal states, thereby identifying abnormal events). This triggers an alarm mechanism.

[0110] The cross-attention mechanism is used to calculate the joint feature vector of the target patient's motion features and frequency domain features, including the following steps:

[0111] S4.41. Use cosine similarity to measure the cosine value of the angle between the target patient's motion feature vector and the frequency domain feature vector to calculate the similarity score. The range is between [-1, 1]. The larger the value, the more similar the two vectors are.

[0112] S4.42. Apply the Softmax function to the similarity scores to convert them into a probability distribution so that each score falls within the range [0, 1] and the sum of all scores is 1.

[0113] S4.43. Based on the similarity score (weight after Softmax), the action feature vector and the frequency domain feature vector are weighted and summed to form a new joint feature vector. The final joint feature vector contains the comprehensive information of the action feature and the frequency domain feature.

[0114] This embodiment is for detecting falls of patients in the emergency intensive care unit:

[0115] Motion features are captured by analyzing a spatiotemporal cube spanning 16 consecutive frames (approximately 0.5 seconds) using a 3D CNN, capturing the patient's continuous motion patterns, including sudden trunk tilt (hip angle change >60°), lower limb instability (knee acceleration >3m / s²), and rapid head approaching the ground. The spatiotemporal features extracted by the 3D convolution kernel are then matched against a pre-defined "fall template" (a limb movement frequency of 2-5Hz during a fall) using a cosine similarity match (>0.85 triggers an alarm).

[0116] Physiological Data Collaboration:

[0117] Heart rate data is collected simultaneously at the moment of the fall (via an ECG monitor). A sudden increase of more than 30 bpm within 6 seconds is detected, combined with transient blood pressure fluctuations (a drop in systolic blood pressure of more than 20 mmHg during the same time period or within the next few minutes). The ECG signal is converted into a time-frequency graph using a wavelet transform. The frequency domain features (high-frequency energy spikes) extracted by the 2D CNN are then cross-attentionally combined with the motion features. If the correlation weight exceeds a threshold (0.7), the event is considered a fall.

[0118] Among them, when the patient is obscured by medical staff, the system uses the cross-frame attention mechanism (CFAM) to fuse the unobstructed features of the previous three frames to maintain tracking continuity. If the patient's position suddenly deviates from the bed area (moves to the side of the equipment cabinet), the medical causal graph will block the false association between "fall" and "normal walking" and trigger an alarm only when abnormal acceleration occurs in the bed area bound to the patient ID.

[0119] Example 2: This example provides an emergency intensive care unit target identification system based on the Internet of Things, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the steps of the above-mentioned emergency intensive care unit target identification method based on the Internet of Things.

[0120] The basic principles, main features, and advantages of the present invention are shown and described above. It should be understood by those skilled in the art that the present invention is not limited to the above-described embodiments. The above-described embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention, and such changes and modifications fall within the scope of the invention claimed.

Claims

1. A target recognition method in an emergency intensive care unit based on the Internet of Things, characterized in that: The following steps are involved: S1, collects the video stream in the emergency intensive care unit through a camera at a fixed frame rate; S2, preprocessing the collected video stream and extracting image frames; S3: Use the YOLOv7 model to perform real-time target detection on each image frame to identify the target patient. Use the DeepSORT algorithm to correlate targets in consecutive frames of the video stream and combine it with the Kalman filter to track the target patient's position. During the tracking process, a four-dimensional spatiotemporal-semantic collaborative cost matrix is ​​constructed for optimization. S4, using 3D CNN to analyze the target patient's action sequence in the video stream and determine abnormalities based on the target patient's physiological data; The process of analyzing the target patient's action sequence in the video stream using 3D CNN and combining it with the target patient's physiological data to determine abnormalities includes the following steps: S4.

1. Extract a 3D space-time cube of continuous time windows from the video stream based on the tracking ID of the target patient, and synchronously collect the physiological data of the target patient in the corresponding time period; S4.

2. Using the 3D space-time cube as input to the 3D CNN model, extracting video features in spatial and temporal dimensions through the convolutional layers of the 3D CNN model, and outputting a target patient motion feature vector. S4.

3. Use continuous wavelet transform to convert the target patient's physiological data into a time-frequency map. Use the time-frequency map as the input of the 2D CNN model to extract frequency domain features and introduce a causal mask. S4.

4. Use the cross-attention mechanism to calculate the joint feature vector of the target patient's motion features and frequency domain features, perform abnormal classification through the MLP classifier based on the joint feature vector, and trigger the alarm mechanism; Furthermore, the cross-attention mechanism is used to calculate the joint feature vector of the target patient's motion features and frequency domain features, including the following steps: S4.

41. Use cosine similarity to measure the cosine value of the angle between the target patient's motion feature vector and the frequency domain feature vector to calculate the similarity score; S4.

42. Applying a Softmax function to the similarity score to convert the similarity score into a probability distribution; S4.

43. Based on the similarity score, perform weighted summation on the action feature vector and the frequency domain feature vector to form a new joint feature vector.

2. The method for identifying targets in an emergency intensive care unit based on the Internet of Things according to claim 1, wherein: In S3, the YOLOv7 model is used to perform real-time target detection on each frame of the image frame to identify the target patient, including the following steps: S3.

1. Input the image frame into the YOLOv7 model for forward propagation, and output the location, confidence, and category score of each predicted box of the target patient; S3.

2. Use non-maximum suppression to remove overlapping prediction boxes and filter out targets with confidence higher than the threshold a. S3.

3. Among all the prediction boxes retained after screening, further analyze the category information of each prediction box, identify the target patients, and output the final test result list.

3. The method for identifying targets in an emergency intensive care unit based on the Internet of Things according to claim 2, wherein: In S3.1, the image frame is input into the YOLOv7 model for forward propagation, and the position, confidence, and category score of each predicted box of the target patient are output, including the following steps: S3.

11. In the backbone of the YOLOv7 model, for each residual block, the feature maps corresponding to the past three frames are retained during forward propagation to build a feature buffer pool. The feature map of the current frame is dot-producted with the feature maps of the previous frames to obtain a similarity matrix. The historical features are weighted and summed using the attention weights. S3.

12. Use a depth camera to obtain depth information within the emergency intensive care unit and calculate the degree of occlusion of the local area based on the depth map; S3.

13. Use the PANet structure of the YOLOv7 model to perform multi-scale feature fusion. Input the multi-scale fused features into the detection head of the YOLOv7 model, and output a prediction tensor containing the bounding box offset, target confidence, and category score. Use the anchor box mechanism to decode the offset into the actual box position in the image coordinate space, and calculate the final confidence value by combining the target confidence and category score.

4. The method for identifying targets in an emergency intensive care unit based on the Internet of Things according to claim 2, wherein: In S3.3, among all the prediction boxes retained after screening, the category information of each prediction box is further analyzed to identify the target patient, including the following steps: S3.31, traverse each prediction box and extract category information; S3.32, mapping the category ID to a semantic label using a predefined category mapping table; S3.33, filter out the prediction box whose category is patient; S3.

34. Number the identified target patients and assign a unique ID to each identified patient; S3.

35. Output the identified target patient list as the final test result list.

5. The method for identifying targets in an emergency intensive care unit based on the Internet of Things according to claim 2, wherein: In S3, the DeepSORT algorithm is used to associate targets in consecutive frames of the video stream, and the Kalman filter is used to track the position of the target patient, including the following steps: S3.

4. Input the detection box coordinates output by YOLOv7 into the DeepSORT model. For each object detected by the YOLOv7 model in each frame, crop the bounding box area of ​​the object and use the ReID sub-network to extract the appearance ReID embedding feature vector of the object. S3.

5. Initialize the Kalman filter for each detected target to predict its state vector in the next frame; S3.

6. Use the state estimate obtained by Kalman filter prediction and the cosine distance between the appearance ReID embedding feature vectors to construct a four-dimensional spatiotemporal-semantic collaborative cost matrix. S3.

7. Perform minimum weight matching on the four-dimensional spatiotemporal-semantic collaborative cost matrix based on the Hungarian algorithm, associate the detection results in the current frame and the previous frame of the video stream, and pre-filter matching candidate pairs through the medical behavior causal graph; S3.

8. Output the tracking result with the target patient ID.

6. The method for identifying targets in an emergency intensive care unit based on the Internet of Things according to claim 5, characterized in that: In S3.6, the state estimate obtained by Kalman filter prediction is combined with the cosine distance between ReID embedding vectors to construct a four-dimensional spatiotemporal-semantic collaborative cost matrix, including the following steps: S3.

61. Combine the texture features of the patient's hospital gown and the device connection status to generate a hybrid ReID feature. Calculate the cosine similarity between the current frame and the target in the historical trajectory. The initial weight is set to b. S3.

62. Calculate the Mahalanobis distance between targets using the Kalman filter prediction results; S3.63, checking whether the target detected in the current frame carries a specific medical device, and comparing the device connection status with the device connection status in the historical trajectory. If the same device is detected in both the current frame and the historical trajectory, increasing the confidence of the matching pair; S3.64, combining the RFID wristband positioning signal, if the Euclidean distance between the detection frame center and the RFID coordinates is less than 0.5 meters, then increase the confidence level of the matching pair; S3.65, when it is detected that the patient's continuous frame acceleration is greater than the threshold x, reduce the appearance weight and increase the motion weight; S3.

66. Based on the results of the above steps, a four-dimensional cost matrix is ​​constructed. The rows in the cost matrix represent each detection box in the current frame, and the columns in the cost matrix represent each tracking trajectory in the previous frame.

7. The method for identifying targets in an emergency intensive care unit based on the Internet of Things according to claim 5, characterized in that: In S3.7, the minimum weight matching of the four-dimensional spatiotemporal-semantic collaborative cost matrix is ​​performed based on the Hungarian algorithm, the detection results in the current frame and the previous frame of the video stream are associated, and the matching candidate pairs are pre-filtered through the medical behavior causal graph, including the following steps: S3.

71. Construct a causal map of medical behavior in the emergency intensive care unit to analyze the action intentions of medical staff in real time; S3.

72. For the target patient carrying emergency equipment, construct an independent sub-cost matrix; S3.

73. Based on the conventional Hungarian algorithm, a detection frame in the current frame and a tracking trajectory in the previous frame are jointly verified in terms of spatial position, device status, and vital signs. S3.74, dynamically set the matching threshold based on the regional congestion; S3.

75. Output the final matching result with priority control.

8. An object recognition system for an emergency intensive care unit based on the Internet of Things, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor executes the computer program to implement the steps of the method for identifying targets in an emergency intensive care unit based on the Internet of Things as described in any one of claims 1 to 7.

Images