A machine vision-based medical monitoring method, medium, and device

Abstract

The application aims to provide a medical monitoring method, medium and device based on machine vision, which distributes edge intelligent monitoring terminals in key medical areas, analyzes video streams in real time, and automatically checks the compliance of medical processes. The scheme realizes full-time, full-coverage automatic real-time monitoring, solves the defects of poor real-time performance and inability to synchronize early warning in operation of manual spot checks in the prior art; at the same time, objective visual data replaces paper signature confirmation, eliminates the problems of after-signing and selective recording, and improves the authenticity and accuracy of compliance management data.

Description

Technical Field

[0001] This invention relates to the field of data risk management technology, specifically to the optimization of process compliance management in the medical field. Background Technology

[0002] In the healthcare field, process compliance management is particularly crucial. It focuses on whether medical procedures are performed in accordance with established procedures, steps, and standards. The core value of process compliance management lies in the fact that medical procedures are highly specialized and risky; any omission or error in the sequence of any step can directly lead to patient safety incidents, such as incorrect surgical sites, infection outbreaks, or medication errors.

[0003] In existing technologies, compliance management of medical processes mainly relies on two methods. The first method is manual on-site spot checks, where hospital quality control departments or reviewers regularly or irregularly visit key areas such as operating rooms, wards, and emergency rooms to observe and record the operational behaviors of medical staff and determine whether they comply with standardized procedures. The second method is manual signature confirmation, which involves setting up paper record forms at key operational nodes, requiring medical staff to sign and confirm after performing a certain operation.

[0004] The aforementioned existing technical solutions have certain systemic defects. Firstly, they lack real-time performance. Manual on-site inspections and post-event video review are both offline processes, unable to provide simultaneous monitoring and early warning at the moment of the operation. When violations occur, the system cannot stop them immediately, only allowing for post-event accountability, by which time adverse consequences have often already occurred. Secondly, the authenticity of the data is questionable. Paper-based signature confirmation methods suffer from issues of "post-event signature" and "selective recording." These defects collectively result in existing compliance management methods failing to meet the needs of modern hospital's refined management in terms of accuracy, timeliness, and comprehensiveness. Summary of the Invention

[0005] The purpose of this invention is to provide a machine vision-based medical monitoring method, medium, and device. Edge intelligent monitoring terminals are deployed in a distributed manner in key medical areas to analyze video streams in real time and automatically verify the compliance of medical procedures. This solution achieves automated real-time monitoring with full coverage and all-time monitoring capabilities, overcoming the shortcomings of existing technologies such as poor real-time performance of manual spot checks and the inability to provide simultaneous early warnings during operations. Furthermore, by replacing paper signatures with objective visual data, it eliminates the problems of post-signature additions and selective recording, improving the authenticity and accuracy of compliance management data.

[0006] In a first aspect, the present invention provides a medical monitoring method based on machine vision, comprising the following steps:

[0007] Read real-time video information from a camera device deployed within a medical facility;

[0008] Identify medical targets in the real-time video information;

[0009] Based on the medical target and its movement trajectory, the medical behavior in the real-time video information is analyzed;

[0010] The parsed medical behaviors are compared one by one with the behaviors in the preset process rule base to determine whether there are any behavioral errors or omissions.

[0011] If a behavioral error or omission occurs, an alarm signal will be generated in real time and pushed to the designated terminal.

[0012] As a preferred embodiment of the present invention, in the step of identifying medical targets in the real-time video information, a target detection model is invoked to identify and detect the video information. The target detection model incorporates the CBAM attention mechanism, and the medical targets include medical personnel and medical items.

[0013] As a preferred embodiment of the present invention, in the step of identifying medical targets in the real-time video information, the target detection model outputs structured information of the medical targets, including the category label, bounding box coordinates and confidence score of each medical target.

[0014] As a preferred embodiment of the present invention, the step of parsing the medical behavior in the real-time video information based on the medical target and its movement trajectory specifically includes:

[0015] Establish an independent tracking thread for each medical staff member and extract the skeletal key point sequence of that medical staff member within a preset time window;

[0016] Information about the medical items surrounding the medical personnel is encoded into a spatial relationship feature vector;

[0017] The skeleton keypoint sequence and the spatial relationship feature vector are input together into the spatiotemporal graph convolutional network to extract the spatiotemporal features of the behavior;

[0018] The spatiotemporal features of this behavior are input into a classifier, and the output recognition result is used as the medical behavior.

[0019] As a preferred embodiment of the present invention, in the step of inputting the spatiotemporal features of the behavior into a classifier and outputting the recognition result as a medical behavior, the classifier outputs the probability distribution of the behavior belonging to each category, and selects the category with the highest probability as the recognition result.

[0020] As a preferred embodiment of the present invention, the step of comparing the parsed medical behavior with the behaviors in the preset process rule base one by one to determine whether there are behavioral errors or omissions specifically includes:

[0021] The multiple medical actions performed by each medical staff member within a continuous time period are arranged into a sequence of actions in chronological order;

[0022] The behavior sequence is compared with the standard process in the preset process rule base, which includes the necessary set of behaviors, the corresponding execution order, and the time constraints between each behavior.

[0023] If an unauthorized action is performed, a necessary action is not performed, or the order of actions is incorrect, it will be deemed non-compliant.

[0024] As a preferred embodiment of the present invention, in the step of comparing the behavior sequence with the standard process in the preset process rule library, specifically, the medical scene area in the real-time video information is identified, and the corresponding process rule is retrieved from the process rule library based on the medical scene area as the standard process.

[0025] As a preferred embodiment of the present invention, in the step of generating alarm signals in real time and pushing them to designated terminals...

[0026] An alarm event record is generated, and the generation of the alarm event record is processed locally, with face and private parts being anonymized for confidentiality.

[0027] The patient's identity information is replaced with an anonymous identifier, and the mapping relationship between the anonymous identifier and the original identity is stored locally only;

[0028] Alarm event records containing anonymous identifiers and information anonymized data are encrypted and encapsulated, and then sent to the cloud via an encrypted transmission protocol.

[0029] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described above.

[0030] In a third aspect, the present invention provides an electronic device, including a processor and a memory; the processor is connected to the memory;

[0031] The memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the method described above.

[0032] In summary, the present invention has the following beneficial effects:

[0033] 1. The camera equipment collects video information in medical facilities in real time, automatically identifies medical targets, analyzes medical behaviors, and compares them one by one with the behaviors in the process rule base. It can promptly detect behavioral errors or omissions, and immediately generate alarm signals to push to designated terminals when an anomaly occurs. This realizes automated intelligent monitoring of the entire medical operation process, shortens the time for anomaly detection, reduces the cost of manual inspection, and improves the level of medical safety.

[0034] 2. The target detection model that integrates the CBAM attention mechanism is used to identify and detect video information. The model can pay more attention to the key feature regions related to medical personnel and medical items, significantly enhance the detection capability of small-sized medical items and targets in complex backgrounds, and improve the accuracy and recall rate of medical target recognition.

[0035] 3. The target detection model outputs structured information of medical targets, including category labels, bounding box coordinates, and confidence scores. This achieves a standardized expression of medical target information, enabling the subsequent behavior analysis module to directly process data in a unified format, reducing redundant calculations and format conversion steps.

[0036] 4. An independent tracking thread is established for each medical personnel and the skeleton key point sequence is extracted. The information of surrounding medical items is encoded into spatial relationship feature vectors, and then jointly input into a spatiotemporal graph convolutional network to extract spatiotemporal features. This achieves deep modeling of the relationship between human actions and object interactions, which can accurately identify complex and subtle medical operation behaviors and significantly improve the accuracy and robustness of medical behavior analysis.

[0037] 5. The classifier outputs the probability distribution of the behavior belonging to each category and selects the category with the highest probability as the recognition result, providing a reliable decision basis for medical behavior recognition, effectively reducing the risk of behavior category confusion and misjudgment, improving the stability and credibility of behavior analysis results in complex real-world scenarios, and laying a solid foundation for subsequent compliance determination.

[0038] 6. By comparing the sequence of actions of medical personnel over a continuous period of time with the standard procedures in the preset process rule base, it can comprehensively detect various non-compliant situations such as execution of prohibited behaviors, failure to execute necessary behaviors, and incorrect execution order of behaviors. It also combines time constraints for judgment, avoiding false alarms and false negatives caused by simple rule judgment.

[0039] 7. The system identifies medical scenario areas and retrieves corresponding process rules from the process rule base as standard processes, enabling automatic adaptation of behavior comparison rules. This ensures that different medical scenarios can be matched with accurate and applicable standard processes, avoiding rule mismatch and the tedious process of manually selecting rules.

[0040] 8. During the alarm event recording generation stage, facial and private parts are anonymized and protected, and patient identity information is replaced with anonymous identifiers stored locally. The processed records are then encrypted, packaged, and sent to the cloud via an encrypted transmission protocol. This effectively prevents the leakage of patient privacy information during transmission and storage, balancing alarm real-time performance and data security. Attached Figure Description

[0041] Figure 1 A schematic diagram of an example environment 100 in which several embodiments of this specification may be implemented is shown;

[0042] Figure 2 A schematic diagram of method 200, in which several embodiments of this specification may be implemented, is shown;

[0043] Figure 3 A schematic diagram of the structure of an electronic device according to some embodiments of this specification is shown. Detailed Implementation

[0044] The present invention will be further described in detail below with reference to the accompanying drawings.

[0045] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0046] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0047] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this specification. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0048] Figure 1 A schematic diagram of an exemplary application environment 100 in which several embodiments of the present disclosure may be implemented is shown.

[0049] like Figure 1 As shown, environment 100 may include camera device 101, local processing system 102 and cloud processing system 103.

[0050] Camera device 101 is deployed in key monitoring areas of medical institutions, including operating rooms, intensive care units, emergency rooms, sterilization supply centers, and pharmacies. Camera device 101 is an intelligent camera with an integrated artificial intelligence acceleration chip, possessing local image acquisition and real-time inference capabilities. Camera device 101 is connected to local processing system 102 via wired or wireless network.

[0051] The local processing system 102 is deployed in the computer room or low-voltage room within the medical institution, and is an edge computing cluster consisting of one or more high-performance servers. The local processing system 102 is responsible for receiving structured behavioral data from the camera device 101, running a medical-specific visual recognition model and a process compliance verification engine, and generating monitoring records and alarm information. The local processing system 102 is connected to the cloud processing system 103 via a dedicated encrypted communication link.

[0052] The cloud processing system 103 is deployed in a remote data center and can serve as a cloud computing service platform. For example, the cloud processing system 103 can be responsible for model training and distribution across campuses, maintenance and updating of the fault knowledge base, aggregation and analysis of monitoring data from multiple campuses, and automatic generation and export of review logs.

[0053] It should be understood that the architecture and functionality in example environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure. Embodiments of this disclosure can also be applied to other environments with different structures and functionalities.

[0054] like Figure 2 As shown, Figure 2 This is a flowchart illustrating a machine vision-based medical monitoring method 200.

[0055] In box 201, method 200 is used for hardware deployment and video acquisition, reading real-time video information from a camera device deployed in a medical facility.

[0056] Box 201 aims to enable real-time acquisition and localized preprocessing of on-site video streams by distributing intelligent monitoring terminals in key areas of medical institutions, providing high-quality visual data input for subsequent steps.

[0057] Specifically, camera equipment 101 is deployed in key monitoring areas of medical institutions. These areas may include operating rooms, intensive care units, central sterile supply departments, and pharmacies. Camera equipment 101 is an intelligent monitoring terminal integrating a 4K high-definition camera, infrared fill light, AI acceleration chip, and edge computing module. Camera equipment 101 can be connected to the local processing system 102 via a gigabit industrial-grade switch supporting PoE power supply, ensuring stable transmission of video data.

[0058] For example, the internal structure of the camera device 101 includes an optical imaging module, an image sensor, an AI inference chip, and a local storage unit. The optical imaging module uses a high-resolution fixed-focus or motorized zoom lens, capable of automatically adjusting the focal length according to the monitoring distance and field of view. The image sensor supports clear imaging in low-light environments and, in conjunction with an infrared fill light, can operate normally at night or in the absence of light. The AI ​​inference chip is a neural network processor designed specifically for edge computing, capable of performing real-time inference operations on video frames locally on the device, without needing to upload the raw video to a remote server. The local storage unit uses solid-state storage media for caching short-term video data and structured analysis results.

[0059] When deploying camera equipment 101, the installation location and angle are determined based on the characteristics of the monitored area. For example, in an operating room, camera equipment 101 is installed at the four corners of the ceiling to ensure no blind spots in key areas such as the operating table, anesthesia machine, instrument table, and medicine cart, with the lens facing the center of the surgical area. For example, in a patient ward, camera equipment 101 is installed on the ceiling directly above the bed, with the lens pointing vertically downwards to cover the entire bed and surrounding monitoring equipment area. For example, in the handwashing area of ​​a sterilization supply center, camera equipment 101 is installed on the wall directly in front of the handwashing sink, with the lens aimed at the handwashing sink and hand dryer area to ensure clear capture of the details of medical staff's hand operations.

[0060] After the camera device 101 is started, it continuously captures on-site video footage at a frame rate of no less than 25 frames per second. The video stream is compressed using H.265 or H.264 encoding format and transmitted to the local processing system 102 via an industrial-grade switch. During transmission, the video data only circulates within the local area network between the camera device 101 and the local processing system 102, without passing through the external public network, ensuring data security.

[0061] Through the above methods, frame 201 achieves comprehensive coverage and continuous monitoring of key areas in medical institutions. Compared to traditional manual inspection methods, this step eliminates the need for dedicated personnel on-site to achieve 24 / 7 video acquisition, significantly expanding the monitoring range and duration. The AI ​​inference chip built into camera device 101 enables front-end processing of video analysis, providing a low-latency, high-frame-rate video data source for real-time target detection and behavior recognition in subsequent steps. Simultaneously, the design of transmitting video data only within the local area network lays the foundation for the data security of the entire system.

[0062] In box 202, method 200 is used for target detection in medical scenarios, that is, to identify medical targets in the real-time video information.

[0063] Box 202 is designed to input the real-time video stream acquired by Box 201 into a medical scene-specific target detection model to identify medical targets in the scene and provide structured target information for subsequent behavior analysis and process compliance detection.

[0064] Specifically, after receiving the real-time video stream transmitted from box 201, the local processing system 102 sends each frame of video to the object detection engine. This object detection engine, for example, can be built based on an improved YOLOv8 network, which introduces the CBAM attention mechanism into the original YOLOv8 backbone network. The CBAM attention mechanism, through the cascaded combination of channel attention modules and spatial attention modules, enables the model to focus more on feature regions in the image related to medical objects, thereby enhancing the detection capability of small medical objects.

[0065] In this paper, medical targets include medical personnel and medical items. The target detection model is trained using a dedicated medical scene dataset. The training data was constructed as follows: over 10,000 hours of video footage from operating rooms were collected, and keyframes in the videos were annotated frame by frame by professionals. The annotated categories included more than fifty types of medical objects such as scalpels, forceps, gauze, gloves, hemostats, and needle holders. During training, various training strategies, such as Mosaic data augmentation, MixUp augmentation, and label smoothing, were employed to improve the model's adaptability to different lighting conditions, shooting angles, and degrees of occlusion.

[0066] When the object detection model runs on the local processing system 102, it performs the following processing flow for each frame of video.

[0067] First, the input image is scaled to a fixed size, and multi-scale feature maps are extracted through the YOLOv8 backbone network.

[0068] Subsequently, feature maps of different scales are fused in the feature pyramid network, enabling the model to detect both large and small objects simultaneously.

[0069] Subsequently, the CBAM attention mechanism is introduced to recalibrate the channel and spatial dimensions of the fused feature map, thereby enhancing the response to key regions.

[0070] Finally, the bounding box coordinates, class label, and confidence score of each target are output through the detection head.

[0071] For example, in a video feed from an operating room, the object detection model can identify three medical personnel in green surgical gowns, positioned on the left, right, and in front of the anesthesia machine, respectively. The model also identifies a scalpel, two forceps, and three pieces of gauze on the operating table; hemostats and needle holders on the instrument table; and a syringe and IV tubing next to the anesthesia machine. Each identification result is accompanied by bounding box coordinates and a confidence score; results with a confidence score higher than a preset threshold are retained and output.

[0072] For example, in video footage from the handwashing area of ​​a disinfection supply center, the object detection model can identify the presence of a healthcare worker in the scene, as well as environmental objects such as handwashing sinks, hand sanitizer bottles, paper dryers, and trash cans. The model can also identify detailed features such as whether the healthcare worker is wearing gloves, the color of the gloves, and the integrity of the gloves.

[0073] For example, in video footage of an area where emergency medical equipment is stored, the target detection model can identify the location and quantity of emergency medical equipment such as defibrillators, ventilators, ambulances, and oxygen cylinders. The model can also make a preliminary judgment on the working status of the equipment by recognizing the colors of indicator lights on the equipment panels and the content displayed on the screen.

[0074] The object detection model's performance was validated on a medical-specific test set, with the average accuracy meeting requirements. The detection accuracy for small objects also showed improvement compared to the original YOLOv8 network. On a local processing system 102 equipped with a Jetson Orin NX edge computing device, the model achieved a real-time inference speed of 30 frames per second, with a single-frame processing latency of less than 50 milliseconds, meeting the requirements for real-time video streaming processing.

[0075] The structured target information output by the target detection model, including the category label, bounding box coordinates, and confidence score of each target, is encapsulated in a standardized data format and transmitted to the subsequent behavior analysis module for further processing.

[0076] Through the above methods, box 202 achieves automated identification of medical targets in healthcare scenarios, namely various objects and personnel. Compared to manual visual observation, the identification speed and coverage of this step far exceed human limits. The deep learning-based target detection model, specially trained on large-scale healthcare scenario data, can accurately distinguish between different categories of targets such as surgical instruments, medicines, medical equipment, and medical personnel, providing a reliable structured data foundation for subsequent behavior recognition and process compliance detection. The introduction of the CBAM attention mechanism significantly enhances the detection capability of small objects such as suture needles and small gauze, avoiding behavior recognition errors caused by missed detection of small targets.

[0077] In box 203, method 200 is used to analyze medical behavior in real-time video information based on medical targets and their movement trajectories.

[0078] Box 203 aims to utilize the structured target information output by Box 202, including the category label, bounding box coordinates, confidence score, and positional change data between consecutive frames for each medical target, to extract semantically meaningful medical behaviors, providing behavioral-level input for subsequent compliance determination.

[0079] Specifically, the behavior analysis module of the local processing system 102 receives a structured data stream from the target detection engine. This data stream is organized in time series, recording all detected target objects and their spatial location information in each frame. The behavior analysis module first establishes an independent tracking thread for each medical worker object, and uses Kalman filtering or Hungarian matching algorithms to achieve target association across frames, thereby generating a continuous movement trajectory for each medical worker object.

[0080] The behavior analysis module integrates a medical behavior recognition model based on a spatiotemporal graph convolutional network. This model takes the skeletal key point sequence of medical personnel and the spatial relationships of surrounding medical objects as input, and captures the spatiotemporal dependencies between different parts of the human body and between people and objects through graph convolution operations, ultimately outputting the category of medical behavior performed in the current time period.

[0081] This medical behavior recognition model is trained using a large-scale dataset of medical scene behaviors. The training data covers common behaviors in several typical medical scenarios, such as operating rooms, sterilization supply centers, and intensive care units, including more than sixty medical behaviors such as hand disinfection, putting on gloves, handling instruments, passing instruments, suturing, and writing medical records. Each behavior is annotated by professional medical personnel in the video footage, and the annotations include the start and end times of the behavior, the behavior category, and the key objects involved in the behavior.

[0082] For example, the behavior analysis module performs the following processing flow on the movement trajectory of each medical staff object.

[0083] First, extract the sequence of key skeleton points of the object within the current time window, including the two-dimensional coordinates of multiple key points such as the head, shoulders, and elbows.

[0084] Subsequently, information about medical objects surrounding the object, including object category, relative position, and relative velocity, is encoded into a spatial relationship feature vector.

[0085] Subsequently, the skeleton keypoint sequence and spatial relationship feature vector are input into the spatiotemporal graph convolutional network. The network extracts the spatiotemporal features of the behavior through the alternating stacking of multiple layers of graph convolution and temporal convolution.

[0086] Finally, the probability distribution of the current behavior belonging to each category is output through a fully connected layer and a Softmax classifier, and the category with the highest probability is selected as the recognition result.

[0087] For example, in an operating room scenario, the behavior parsing module tracks a medical staff member's hand bounding box as it moves from the instrument table area to the operating table area, while simultaneously detecting an overlap between the hand and the bounding box of a scalpel. Combining this with the medical staff member's skeletal keypoint sequence, the spatiotemporal graph convolutional network identifies a pattern in the trajectory of their right wrist: grasping from the instrument table, lifting it, and moving it towards the operating table, thus interpreting the behavior as picking up a scalpel.

[0088] For example, in the handwashing area of ​​a disinfection supply center, the behavior analysis module tracked the movement trajectory of a healthcare worker's hand skeletal key points while they were in front of the handwashing sink, revealing a periodic reciprocating movement pattern. Simultaneously, visual features of the faucet being on and regional characteristics of water splashing were detected. After comprehensively analyzing this information, the spatiotemporal graph convolutional network determined that the behavior was handwashing.

[0089] For example, in an intensive care unit, the behavior analysis module tracks a nurse carrying an IV bag towards a patient's bed, with key points on her arm skeletal structure showing a trajectory of raising and moving towards the IV stand. Simultaneously, it detects that the relative positions of the IV bag and the IV stand gradually approach each other until they overlap. The spatiotemporal graph convolutional network interprets this behavior as changing an IV bag.

[0090] The processing performance of the behavior analysis module has been optimized. On the local processing system 102 equipped with Jetson Orin NX, the behavior analysis module can simultaneously perform real-time behavior recognition on multiple healthcare personnel objects, with ideal control over recognition latency and recognition accuracy for each object.

[0091] The medical behavior data output by the behavior analysis module includes the behavior category, start and end times of the behavior, the target object number involved in the behavior, and the spatial area where the behavior occurred. This data is encapsulated into a standardized message format and transmitted to the behavior comparison module in box 204 for further compliance determination.

[0092] Using the above method, box 203 automatically extracts semantically meaningful medical behaviors from raw video information. The dual-modal feature fusion strategy based on skeleton keypoints and spatial relationships enables the model to maintain stable recognition performance when facing medical personnel of different body types, attire, and shooting angles.

[0093] It should be noted that in real-world medical scenarios, multiple medical personnel often exist simultaneously, and this method supports simultaneous monitoring of multiple individuals. The system assigns a unique identifier to each medical personnel object in the screen and establishes an independent tracking thread for each, generating their movement trajectory and behavior analysis results separately. The threads do not interfere with each other, thus enabling the simultaneous processing of multiple medical personnel in the same monitoring screen, and also allowing continuous tracking of the complete behavior process of the same medical personnel across screens, meeting the monitoring needs of multiple personnel operating in parallel in real-world medical environments.

[0094] In box 204, method 200 is used to compare the parsed medical behavior with the behaviors in the preset process rule base one by one to determine whether there are any behavior errors or omissions.

[0095] Box 204 aims to use the medical behavior data output from Box 203 to systematically compare it with predefined standard operating procedures, and to identify deviations in actual operation, including performing prohibited behaviors, failing to perform necessary behaviors, and incorrect behavior execution order.

[0096] Specifically, the behavior comparison module of the local processing system 102 receives a medical behavior data stream from the behavior parsing module. This data stream contains a series of medical behaviors performed by each medical staff member within a continuous time window, arranged in chronological order to form a behavior sequence.

[0097] The behavior comparison module maintains a process rule base. This rule base is pre-created and is typically built by engineers based on the medical institution management standards issued by the National Health Commission and the internal operating procedures provided by the hospital. Each process rule corresponds to a complete medical operation process, specifying the necessary set of behaviors that the process must include, the execution order of the behaviors, and the time constraints between the behaviors.

[0098] The process rules are represented using a directed acyclic graph. Each node in the graph represents a standard medical action, each directed edge represents the order of actions, and the weight of the edge represents the maximum allowed time interval. In addition, each node can be assigned preconditions and postconditions. Preconditions refer to the states that must be met before the action is executed, and postconditions refer to the states that should be achieved after the action is executed.

[0099] The behavior comparison module performs the following processing flow for the behavior sequence of each medical staff member. First, based on the current medical scenario area of ​​the individual (e.g., operating room, sterilization supply center, or intensive care unit), it retrieves the corresponding process rule from the process rule base. Second, it aligns and matches the actual behavior sequence with the node sequence in the rule graph, using a dynamic time warping algorithm to handle the scaling differences between the actual and standard behaviors on the time axis. Then, it checks item by item whether the actual behavior sequence contains all the necessary behaviors stipulated by the rules, whether there are any additional behaviors not defined in the rules, whether the execution order of the behaviors conforms to the edge directions in the rule graph, and whether the time interval between adjacent behaviors exceeds the maximum allowable value stipulated by the rules. Finally, it generates a compliance judgment conclusion based on the above check results.

[0100] For example, in an operating room scenario, a surgical hand disinfection procedure rule is defined in the process rule library. This rule specifies the necessary sequence of actions as follows: turn on the tap, rinse hands up to wrists, take hand sanitizer, rub hands together for at least fifteen seconds, rinse clean, turn off the tap, take paper towels, dry hands, and discard the paper towels. The behavior comparison module compares the actual behavior sequence parsed from box 203 with this rule. If the action of taking hand sanitizer is missing from the actual behavior sequence, it is determined to be a behavior omission, specifically a missing handwashing step. If the order of taking hand sanitizer after turning off the tap is reversed, it is determined to be an incorrect behavior order. If the duration of rubbing hands together in the actual behavior sequence is only eight seconds, failing to meet the minimum requirement of fifteen seconds stipulated in the rule, it is determined to be a failure to meet the behavior execution standard.

[0101] For example, in the intensive care unit, a set of handover process rules is defined in the process rule base. These rules stipulate that the following actions must be performed during handover: verifying patient identity information, handing over vital sign data, handing over medication records, handing over changes to medical orders, and handing over special precautions. The behavior comparison module compares the actual behavior sequence parsed from box 203 with these rules. If the action of handing over medication records is missing from the actual behavior sequence, it is determined to be a behavior omission. If all necessary actions in the actual behavior sequence are performed in the correct order, it is determined to be compliant.

[0102] The process rule base supports version management and dynamic updates. When medical industry standards or hospital internal procedures change, administrators can upload new versions of the process rules through the management interface of the cloud processing system 103. The system automatically compares the differences between the old and new versions and, after review and confirmation, distributes the new rules to all connected local processing systems 102. Upon receiving the new rules, each local processing system 102 will automatically implement the new rules for judgment at the start of the next monitoring cycle.

[0103] The behavior comparison module outputs the results of its violations in the form of structured violation event records. Each violation event record includes the following fields: unique violation event ID, violation type, violation level, violation occurrence time, violation location, object ID involved in the violation, description of the behavior involved in the violation, description of the standard procedure requirements, description of the actual behavior, start and end frame numbers of the video clip used as evidence of the violation, and associated target detection data and behavior analysis data. Violation event records are simultaneously written to the local database and reported to the cloud processing system 103.

[0104] Through the methods described above, box 204 achieves automated and standardized compliance assessment of medical operation processes. Compared to traditional manual spot checks and post-audit methods, this step can detect deviations in real time during the operation and accurately pinpoint the deviations to specific behavioral stages. The process rule representation method based on directed acyclic graphs can accurately describe the behavioral dependencies and temporal constraints in complex medical processes, avoiding the limitations of simple list-based rules in expressing sequential requirements. The dynamically updated rule base management mechanism ensures that the system can reflect the latest medical regulatory requirements in a timely manner, maintaining the effectiveness and accuracy of compliance assessments in the long term.

[0105] In box 205, method 200 is used for cloud data processing and hospital review indicator generation, and pushes alarm signals in real time. That is, if a behavioral error or omission occurs, an alarm signal is generated in real time and pushed to the designated terminal. Privacy protection is also implemented.

[0106] Box 205 aims to address two issues: first, when Box 204 determines that a behavioral error or omission has occurred, the system can generate an alarm signal in real time and push it to the designated terminal, so that managers can be notified and intervene immediately; second, it implements privacy protection requirements throughout the entire data flow process to ensure that the personal information of patients and medical staff is not leaked.

[0107] Specifically, after completing the compliance assessment, the behavior comparison module in the local processing system 102 immediately generates an alarm event record if it confirms a behavioral error or omission, and sends it to the cloud processing system 103 via an encrypted channel. The real-time alarm service of the cloud processing system 103 parses the received event record and generates an alarm signal containing the violation type, violation level, occurrence time, occurrence area, anonymous ID of the involved party, and a screenshot of the on-site video. Subsequently, the alarm signal is sent to designated terminals via a message push service. Designated terminals include the smart display screen at the nurse station, the on-duty computer in the quality control department, and the mobile phones and tablets of relevant management personnel. The alarm signal is simultaneously pushed out via pop-up reminders, SMS notifications, and voice broadcasts to ensure that relevant responsible persons can quickly be informed and respond.

[0108] For example, when a violation of aseptic technique occurs in the operating room, an alarm signal is pushed to the smart display screen in the corridor outside the operating room and the surgeon's mobile terminal within hundreds of milliseconds after the violation is determined. For example, when a cleaning procedure is missed in the sterilization supply center, an alarm signal is simultaneously pushed to the head nurse's mobile phone and the quality control department's on-duty computer, along with a specific description of the missed procedure and the standard procedure requirements.

[0109] Regarding privacy protection, the local processing system 102 performs multiple anonymization processes when generating and reporting data. For video screenshots carried in violation event records, a face detection algorithm is used to locate the facial areas of all medical staff and patients in the image, and Gaussian blurring is applied to these areas. For image areas involving patient body parts, solid-color rectangles are used for overall occlusion. Identification information such as names, employee numbers, and patient hospital numbers in the local database are replaced with randomly generated meaningless anonymous identifiers. The mapping table between these anonymous identifiers and the original identities is stored only in the local processing system 102 and is not uploaded to the cloud with the data. For example, in the hand hygiene monitoring scenario, the reported data only includes the anonymized medical staff role category, department number, time of occurrence, and behavior judgment result, and does not contain any fields that can identify individuals.

[0110] The anonymized data is encrypted and encapsulated locally using the SM4 symmetric encryption algorithm, and an integrity check code is calculated using the SM3 hash algorithm before being sent to the cloud processing system 103 via an encrypted transmission protocol. The cloud processing system 103 decrypts and verifies the integrity of the received data; data packets that fail verification are discarded. When storing data, the cloud processing system 103 dynamically executes field-level anonymization strategies based on the query user's role; users with different permissions can only view data at their corresponding anonymization level. The default data retention period is three years. Data exceeding the retention period is automatically transferred to cold storage media by scheduled archiving tasks and destroyed according to the medical institution's data management system.

[0111] The raw video stream data is always stored in the local storage unit of the local processing system 102 and is not uploaded to the cloud processing system 103. Access to the raw video must be authorized through local management terminal authentication and secondary authorization. Access operations automatically overlay a watermark containing the accessor's account, access time, and purpose onto the video screen to prevent unauthorized copying and distribution of video data. The cloud processing system 103 only retains the anonymized structured data and analysis results.

[0112] Through the above methods, frame 205 enables real-time alerts for behavioral errors and omissions, allowing administrators to intervene and handle violations immediately upon occurrence, effectively preventing the continuation and escalation of violations. Simultaneously, by anonymizing data before reporting, encrypting and verifying its integrity during transmission, dynamically desensitizing data during storage, and implementing tiered access control, a multi-layered privacy protection mechanism covering the entire data lifecycle is constructed. This minimizes the risk of privacy breaches for patients and medical staff while achieving intelligent monitoring of hospital review indicators.

[0113] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0114] Figure 3 A block diagram of an electronic device 500 that can implement various embodiments of the present disclosure is shown. For example... Figure 3 As shown, device 500 includes a processor 501, which can perform various appropriate actions and processes based on computer program instructions loaded into random access memory (RAM) 503 according to computer program instructions stored in read-only memory (ROM) 502. RAM 503 may also store various programs and data required for the operation of device 500. The processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0115] The various processes and procedures described above, such as method 200, can be executed by processor 501. For example, in some embodiments, method 200 may be implemented as a software program tangibly contained in a machine-readable medium. In some embodiments, part or all of the software program may be loaded and / or installed on device 500 via ROM 502. When the software program is loaded into RAM 503 and executed by processor 501, one or more actions of method 200 described above may be performed.

[0116] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0117] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. Furthermore, although operations are depicted in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0118] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A medical monitoring method based on machine vision, characterized in that, Includes the following steps: Read real-time video information from a camera device deployed within a medical facility; Identify medical targets in the real-time video information; Based on the medical target and its movement trajectory, the medical behavior in the real-time video information is analyzed; The parsed medical behaviors are compared one by one with the behaviors in the preset process rule base to determine whether there are any behavioral errors or omissions. If a behavioral error or omission occurs, an alarm signal will be generated in real time and pushed to the designated terminal.

2. The medical monitoring method based on machine vision according to claim 1, characterized in that, In the step of identifying medical targets in the real-time video information, a target detection model is invoked to identify and detect the video information. The target detection model incorporates the CBAM attention mechanism, and the medical targets include medical personnel and medical items.

3. The medical monitoring method based on machine vision according to claim 2, characterized in that, In the step of identifying medical targets in the real-time video information, the target detection model outputs structured information of the medical targets, including the category label, bounding box coordinates, and confidence score of each medical target.

4. The medical monitoring method based on machine vision according to claim 3, characterized in that, In the step of parsing the medical behavior in the real-time video information based on the medical target and its movement trajectory, the specific steps are as follows: Establish an independent tracking thread for each medical staff member and extract the skeletal key point sequence of that medical staff member within a preset time window; Information about the medical items surrounding the medical personnel is encoded into a spatial relationship feature vector; The skeleton keypoint sequence and the spatial relationship feature vector are input together into the spatiotemporal graph convolutional network to extract the spatiotemporal features of the behavior; The spatiotemporal features of this behavior are input into a classifier, and the output recognition result is used as the medical behavior.

5. A medical monitoring method based on machine vision according to claim 4, characterized in that, In the step of inputting the spatiotemporal features of the behavior into a classifier and outputting the recognition result as a medical behavior, the classifier outputs the probability distribution of the behavior belonging to each category, and selects the category with the highest probability as the recognition result.

6. The medical monitoring method based on machine vision according to claim 1, characterized in that, The step of comparing the parsed medical behavior with behaviors in a preset process rule base one by one to determine whether there are any behavioral errors or omissions is as follows: The multiple medical actions performed by each medical staff member within a continuous time period are arranged into a sequence of actions in chronological order; The behavior sequence is compared with the standard process in the preset process rule base, which includes the necessary set of behaviors, the corresponding execution order, and the time constraints between each behavior. If an unauthorized action is performed, a necessary action is not performed, or the order of actions is incorrect, it will be deemed non-compliant.

7. A medical monitoring method based on machine vision according to claim 6, characterized in that, In the step of comparing the behavior sequence with the standard process in the preset process rule base, specifically, the medical scene area in the real-time video information is identified, and the corresponding process rule is retrieved from the process rule base based on the medical scene area as the standard process.

8. A medical monitoring method based on machine vision according to claim 1, characterized in that, In the step of generating alarm signals in real time and pushing them to designated terminals An alarm event record is generated, and the generation of the alarm event record is processed locally, with face and private parts being anonymized for confidentiality. The patient's identity information is replaced with an anonymous identifier, and the mapping relationship between the anonymous identifier and the original identity is stored locally only; Alarm event records containing anonymous identifiers and information anonymized data are encrypted and encapsulated, and then sent to the cloud via an encrypted transmission protocol.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

10. An electronic device, comprising a processor and a memory; the processor being connected to the memory; The memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to perform the method as described in any one of claims 1 to 8.

Images