Nurse intervention process intelligent monitoring method combined with deep learning

Abstract

The application relates to the technical field of artificial intelligence, and discloses a nurse intervention process intelligent monitoring method combined with deep learning, which aims to solve the fragmentation and superficiality of nurse intervention process monitoring in the prior art. The method comprises the following steps: deploying a multi-modal perception network to collect multi-source heterogeneous data streams in real time; constructing a deep behavior recognition model based on a space-time attention mechanism to perform feature extraction and fusion, and generating a structured semantic description; establishing a compliance verification engine driven by a clinical intervention knowledge graph to identify and mark potential illegal operations or omitted steps; and based on a reinforcement learning adaptive alarm strategy generation module, dynamically generating and pushing differentiated intervention reminders or alarm information. The application can realize accurate, continuous and whole-process identification and understanding of nurse intervention behavior, significantly improve the intelligent level and judgment accuracy of monitoring, dynamically adjust the alarm mode and intensity according to the environmental context, and improve the practicability and acceptability of the system.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to an intelligent monitoring method for nurse intervention processes that combines deep learning. Background Technology

[0002] With the accelerated development of smart healthcare systems, the standardization and intelligence of clinical nursing work are increasingly becoming core aspects of improving hospital service quality. Nursing intervention, as a key behavior in the patient care process, covers multiple dimensions such as vital sign monitoring, medication administration, risk warning response, and humanistic care. The timeliness, standardization, and completeness of its operation are directly related to patient safety and treatment outcomes.

[0003] Traditional monitoring of nurse intervention processes mainly relies on manual inspections, paper records, or simple electronic check-in systems, which have significant limitations: First, monitoring methods lack real-time and continuous capabilities, making it difficult to track nurses' actual operational behaviors throughout the entire process in the ward; second, existing systems cannot effectively identify the semantic content of intervention actions, only recording timestamps or location information, and cannot determine whether the operation conforms to clinical standards; third, there is a lack of deep integration mechanisms between multi-source heterogeneous data (such as video streams, wearable device signals, and electronic medical orders), resulting in fragmented behavioral understanding, high false alarm rates, and poor scenario adaptability; finally, in complex clinical environments, existing methods are not robust enough to interference factors such as occlusion, changes in lighting, and multi-person interaction, making it difficult to support the needs of high-precision and high-reliability intelligent monitoring.

[0004] The aforementioned problems make it difficult for the current technology system to achieve refined, automated, and intelligent monitoring of nurses' intervention processes, and there is an urgent need for a new generation of monitoring methods that can integrate multimodal perception and deep learning reasoning capabilities. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and to provide an intelligent monitoring method for nurse intervention processes that combines deep learning, which can effectively solve the problems in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an intelligent monitoring method for nurse intervention processes combined with deep learning, the method comprising the following steps: Step S110: Deploy a multimodal sensing network to collect multi-source heterogeneous data streams in real time during the nurse intervention process. The multi-source heterogeneous data streams include at least the ward environment video stream, the nurse's wearable device signal, and the electronic medical order data in the hospital information system. Step S120: Construct a deep behavior recognition model based on spatiotemporal attention mechanism, extract and fuse features from the multi-source heterogeneous data stream, and generate a structured semantic description of the nurse's intervention behavior. The structured semantic description includes at least the behavior category, the execution object, the operation standardization score, and the timestamp. Step S130: Establish a compliance verification engine driven by clinical intervention knowledge graph, compare and reason with the structured semantic description in real time with the preset clinical nursing standard knowledge graph, and identify and mark potential violations or omissions. Step S140: The reinforcement learning-based adaptive alarm strategy generation module dynamically generates and pushes differentiated intervention reminders or alarm information based on the severity level, frequency of occurrence, and current ward environment context of the violation or omission step.

[0007] Preferably, in step S110, deploying the multimodal sensing network specifically includes: deploying network cameras with infrared illumination and wide dynamic range functions in key areas of the ward to collect 1080P resolution video streams at a rate of 25 frames per second; equipping nurses with smart name tags integrating inertial measurement units and heart rate sensors to collect nurses' three-axis acceleration, three-axis angular velocity, and heart rate variability signals in real time at a sampling frequency of 10Hz; and acquiring electronic medical orders, nursing plans, and patient vital sign history records related to the current ward and patient in real time through the hospital information system data interface in an event-driven manner, forming text and numerical data streams.

[0008] Furthermore, in step S110, the multi-source heterogeneous data stream is preprocessed and spatiotemporally aligned: a YOLOv5-based target detection algorithm is applied to the video stream to locate and track the bounding boxes of nurses, patients, and key medical equipment (such as infusion pumps and monitors) in real time; low-pass filtering and noise reduction are performed on the wearable device signal, and its time series is precisely aligned with the timestamp of the video stream using a dynamic time warping algorithm; the electronic medical order data is parsed into a structured triplet form and associated with the start time of events detected in the video stream.

[0009] Preferably, in step S120, constructing a deep behavior recognition model based on a spatiotemporal attention mechanism includes: designing a two-stream neural network architecture, wherein the spatial stream takes a preprocessed sequence of video keyframes as input, and the temporal stream takes a sequence of continuous optical flow maps as input; introducing a multi-head self-attention module into the two-stream neural network, wherein the calculation formula of the multi-head self-attention module is:

[0010] in, These represent the query, key, and value matrices, respectively. The dimension of the key vector; this module enables the model to adaptively focus on the spatial regions and temporal segments in the video most relevant to the nurse's operation; at the same time, the preprocessed and aligned wearable device signals (such as arm movement trajectory and body posture angle) are feature-encoded through an independent one-dimensional convolutional neural network, and the encoded feature vector is concatenated with the fused features of the two-stream network in the feature dimension.

[0011] Furthermore, in step S120, generating a structured semantic description of the nurse's intervention behavior specifically involves: inputting the concatenated multimodal feature vector into a fully connected layer classifier, which outputs a probability distribution corresponding to a preset nurse intervention behavior category library (such as "intravenous injection", "vital sign measurement", "turning over and patting the back", "medication verification"); simultaneously, through a parallel regression network branch, outputting a standardization score of the current operation relative to the standard operation template, with the score ranging from 0 to 1; the structured semantic description is finally encapsulated in JSON format, including the behavior category identifier, confidence level, standardization score, associated patient ID, device ID used, and start and end timestamps accurate to milliseconds.

[0012] Preferably, in step S130, establishing a compliance verification engine driven by a clinical intervention knowledge graph includes: pre-constructing a knowledge graph centered on clinical nursing standards. The nodes of this knowledge graph include nursing operations, medications, devices, patient status, and contraindications. Edges represent the sequence of operation steps, the applicability of operations with medications / devices, the matching relationship between operations and patient status, and the mutual exclusion relationship between operations. After receiving the structured semantic description output in step S120, the compliance verification engine maps it to entities and relationships in the knowledge graph and performs graph traversal and rule reasoning.

[0013] Furthermore, in step S130, the logic for identifying and marking potential violations or omissions includes: a timing compliance check, verifying whether the currently identified operation is within a reasonable time window specified by the electronic medical order, and whether the time interval between multiple operation steps conforms to the sequential constraints defined in the knowledge graph; a contextual correlation check, verifying whether the medication or device used in the current operation matches the patient's current medical order and allergy history recorded in the knowledge graph; and a completeness check, based on the operation subgraph necessary for a complete intervention defined in the knowledge graph, checking whether all necessary steps in the current intervention session have been identified and recorded, and marking any missing nodes.

[0014] Furthermore, in step S140, the reinforcement learning-based adaptive alarm policy generation module includes: defining a state space. Its elements are triples <violation type, environmental context, historical frequency>, where the environmental context includes the ward's busyness level and the patient's critical condition level; an action space is defined. It includes different levels of alarm actions, such as "silent recording," "pop-up notification at the nurses' station," "sending a text message to the head nurse," and "triggering an audible and visual alarm"; design a reward function. The function rewards positive actions when they effectively correct violations without excessively interfering with normal nursing care, and rewards negative actions when false alarms or missed alarms occur. A deep Q-network is optimized through a combination of offline training and online fine-tuning, enabling it to learn how to respond to a given state. Choose the optimal alarm action. To maximize long-term cumulative rewards.

[0015] Furthermore, in step S140, the dynamic generation and push of differentiated intervention reminders or alarm information are specifically manifested as follows: for medium-risk violations such as "omission of medication verification", a slight vibration is generated on the smart name tag worn by the nurse as a reminder; for high-risk violations such as "performing a certain operation on a contraindicated patient", a red highlighted alarm information is immediately pushed simultaneously on the central monitoring screen of the nurse station and the head nurse's mobile terminal, and the relevant medicine cabinet is automatically locked; all alarm events, triggering strategies, processing results and subsequent nurse feedback are recorded in the blockchain evidence storage module to ensure the immutability of the audit trajectory.

[0016] The technical effects and advantages of the present invention in the above technical solution are as follows: By deeply integrating multimodal data such as video, wearable device signals, and electronic medical orders, and using spatiotemporal attention mechanisms for feature extraction and alignment, we have achieved accurate, continuous, and full-process recognition and understanding of nurses' intervention behaviors, from surface actions to deep semantics, overcoming the fragmentation and superficiality problems of traditional monitoring methods.

[0017] By introducing a clinical nursing standards knowledge graph as prior knowledge to drive compliance verification, and placing the behavioral recognition results in a rich clinical context for logical reasoning, it is possible to systematically identify complex violation scenarios such as timing errors, context mismatches, and missing steps, significantly improving the intelligence level and accuracy of monitoring.

[0018] By adopting an adaptive alarm strategy based on reinforcement learning, the alarm method and intensity can be dynamically adjusted according to the severity of the violation, environmental context and historical patterns. This achieves a balance between precise intervention and minimizing workflow disruption, improving the system's practicality and acceptability.

[0019] The entire solution addresses common challenges in clinical environments such as occlusion, lighting changes, and multi-person interaction. Through multimodal data complementarity and robust deep learning model design, it enhances the system's environmental adaptability and reliability, providing a feasible technical foundation for high-quality intelligent nursing supervision. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall technical solution architecture of the intelligent monitoring method for nurse intervention process combined with deep learning proposed in this invention. Figure 2 This is a schematic diagram of the core principle framework of the deep behavior recognition model based on spatiotemporal attention mechanism in this invention; Figure 3 This is a flowchart illustrating the logical process of multimodal sensing data acquisition and preprocessing in this invention. Figure 4 This is a schematic diagram of the multi-level interaction relationships and data flow of the compliance verification engine driven by the clinical intervention knowledge graph in this invention; Figure 5 This is a schematic diagram of the principle framework of the adaptive alarm strategy generation module based on reinforcement learning in this invention. Detailed Implementation

[0021] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the specific embodiments according to the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.

[0022] Example 1 In the internal medicine wards of large general hospitals, especially in intensive care units and postoperative observation areas, nurses' interventions directly affect patients' lives, safety, and recovery quality. Traditional monitoring methods rely on manual spot checks, post-event review of video recordings or paper records, which are inefficient, have narrow coverage, and struggle to detect complex violations in real time. This embodiment addresses this scenario by deploying and running an intelligent monitoring method for nurse intervention processes that incorporates deep learning, enabling real-time, accurate, and intelligent monitoring of the entire process from receiving medical orders to completing interventions.

[0023] See Figure 1 The overall system architecture in this embodiment consists of a multimodal perception network, a deep behavior recognition model based on spatiotemporal attention mechanism, a compliance verification engine driven by a clinical intervention knowledge graph, and an adaptive alarm strategy generation module based on reinforcement learning. Its core workflow begins with the real-time acquisition and fusion of multi-source heterogeneous data, which is then parsed into structured semantics by a deep learning model, followed by compliance reasoning through the knowledge graph, and finally dynamically generated and executed differentiated alarm strategies based on the reasoning results.

[0024] First, a multimodal sensing network is deployed to collect multi-source heterogeneous data streams during nurse interventions in real time. The implementation involves deployment and collection at three levels. At the ward environment video stream acquisition level, network cameras equipped with infrared illumination and wide dynamic range capabilities are deployed in key areas such as patient bedsides, treatment cart parking areas, medication preparation stations, and corridor entrances and exits. These cameras collect 1080P resolution video streams at a constant rate of 25 frames per second, ensuring clear capture of nurses' facial orientation, fine motor skills, body posture, and interactions with patients and medical equipment even under varying day / night lighting conditions, backlighting, or partial shadows. The video stream is continuously transmitted to the central processing server via a dedicated hospital network using a real-time transmission protocol.

[0025] At the level of signal acquisition for wearable devices for nurses, each on-duty nurse is equipped with a smart badge integrating an inertial measurement unit and a heart rate sensor. This badge continuously collects the nurse's three-axis acceleration, three-axis angular velocity, and heart rate variability signals based on photoplethysmography (PPG) at a sampling frequency of 10 Hz. The three-axis acceleration data is used to infer the nurse's movement trajectory, gait characteristics, and sudden changes in posture; the three-axis angular velocity data is used to calculate the body's pitch, roll, and yaw angles to determine whether the nurse is standing, bending over, or sitting; and the heart rate variability signal serves as a physiological indicator reflecting the nurse's workload and stress state. All signals are synchronized in real-time via Bluetooth Low Energy to the nurse's mobile terminal or a fixed gateway in the ward, and then aggregated to a central server.

[0026] At the data access level of the hospital information system, electronic data streams related to the currently monitored ward and patients are acquired in real time through application programming interfaces (APIs) that conform to medical information exchange standards, in an event-driven manner. When a nurse scans a patient's wristband or medication barcode using a personal digital assistant, or when a doctor issues a new medical order at their workstation, the system immediately triggers data capture. The acquired data includes at least: the patient's basic information, details of currently valid electronic medical orders, a list of nursing plan tasks, the patient's recent vital signs records, allergy history, and medication records. This data constitutes a mixed text and numerical data stream, providing crucial clinical context information for subsequent behavioral understanding.

[0027] See Figure 3Before entering the core analysis module, the acquired multi-source heterogeneous data streams undergo rigorous preprocessing and spatiotemporal alignment to ensure the effectiveness of subsequent fusion and analysis. The video stream preprocessing first applies a target detection algorithm based on the YOLOv5 architecture to analyze each frame in real time, locating and continuously tracking the bounding boxes of nurses, patients, and key medical equipment in the frame. Key medical equipment includes, but is not limited to, infusion pumps, monitors, syringe pumps, and ventilators. The system assigns a unique identifier to each tracked target and records its pixel coordinates, size, and confidence level. Simultaneously, it calculates the dense optical flow field between consecutive frames, generating an optical flow map sequence to characterize the target's motion pattern.

[0028] Preprocessing of wearable device signals includes data cleaning and initial feature extraction. The raw triaxial acceleration and angular velocity signals are first passed through a Butterworth low-pass digital filter with a cutoff frequency of 5 Hz to eliminate high-frequency noise and power frequency interference. Subsequently, outlier segments caused by brief poor device contact or severe impacts are identified and removed through sliding window variance calculation. For heart rate variability signals, time-domain and frequency-domain indices of adjacent heartbeat interval sequences are calculated. The preprocessed wearable signal time series is precisely aligned with the time reference of the video stream using a dynamic time warping algorithm. This algorithm finds the optimal matching path by non-linearly bending the time axes of the two sequences, thereby eliminating time offsets caused by device startup delays, network transmission jitter, or sampling rate differences, ensuring that the "raising hand" action corresponds precisely to the frames of arm movement in the video.

[0029] Preprocessing of textual and numerical data such as electronic medical orders focuses on structuring and association. The system parses unstructured medical order text into structured triples in the form of "operation-object-parameter" using named entity recognition and relation extraction techniques. For example, "500 ml of 0.9% sodium chloride injection intravenously, 40 drops per minute" is parsed as the operation "intravenous drip," the object "0.9% sodium chloride injection," the parameters "volume 500 ml," and "drip rate 40 drops / minute." This structured data is then associated with the start times of relevant events detected in the video stream. The association logic is based on the principles of temporal proximity and object consistency; for example, the time point of a video event scanning a bottle of medicine's barcode is bound to the corresponding medical order entry for that medicine.

[0030] After data preprocessing and alignment, the process proceeds to step S120, which involves constructing a deep behavior recognition model based on a spatiotemporal attention mechanism. This model extracts and fuses features from the fused multimodal data, ultimately generating a structured semantic description of the nurse's intervention behavior. (See also...) Figure 2At the heart of this model is a meticulously designed two-stream neural network architecture specifically designed for processing visual information. The spatial stream network takes a pre-processed sequence of keyframes extracted from the video stream as input. These keyframes are extracted from the original video stream at fixed intervals or based on scene change detection, and are then scaled and normalized. Spatial stream networks typically employ residual networks pre-trained on large image datasets as their backbone, extracting static appearance features related to the nurse's actions in each frame, such as the shape of the instruments held in the hands, the color of the medication packaging, and patient body parts.

[0031] The temporal flow network takes a sequence of computed continuous optical flow maps as input. Optical flow maps represent the motion vectors of pixels across consecutive frames. The temporal flow network employs a three-dimensional convolutional neural network to capture the dynamic evolution of actions, such as the speed of a syringe piston, the trajectory of a swab, and the direction of force applied when turning a patient. Spatiotemporal information is crucial for distinguishing behaviors that appear similar but have different dynamics.

[0032] This two-stream neural network innovatively introduces a multi-head self-attention module. This module enables the model to adaptively focus on the spatial regions and temporal segments in the video sequence most relevant to the current recognition task, rather than processing all information equally. The calculation process is as follows: For the input feature sequence, a query matrix, a key matrix, and a value matrix are generated through linear transformation. The attention weights are obtained by calculating the dot product of the query and all keys, scaling by the square root of the key vector dimension, and then applying a normalized exponential function. The final output is a weighted sum of the value matrices, and the weights are the calculated attention scores. The specific calculation formula is as follows:

[0033] in, Represents the query matrix. Represents the bond matrix. Representative value matrix, Let be the dimension of the key vector. By having multiple attention heads work in parallel and fusing the results, the model can simultaneously focus on information from different representation subspaces. For example, one head might focus on the nurse's hand area, while another head focuses on the monitor screen the nurse is looking at, thus providing a more comprehensive understanding of complex operational scenarios.

[0034] Meanwhile, the pre-processed and aligned wearable device signals undergo deep feature encoding via a separate one-dimensional convolutional neural network. This network, composed of multiple stacked one-dimensional convolutional layers, pooling layers, and fully connected layers, is capable of extracting high-level abstract features representing movement patterns, body stability, and operational rhythm from temporal signals such as acceleration and angular velocity. For example, patterns such as "holding an object steadily," "rapidly shaking an ampoule," and "walking briskly" can be encoded from acceleration signals.

[0035] Finally, the feature vector fused from the visual two-stream network and the feature vector encoded from the wearable signal are concatenated along the feature dimension to form a unified multimodal feature representation. This fused feature vector is input into a fully connected layer classifier. The classifier outputs a probability distribution vector whose dimension corresponds to a predefined library of nurse intervention behavior categories. The behavior category library is based on clinical nursing routines and includes dozens of common operations such as "intravenous injection," "subcutaneous injection," "vital sign measurement," "oral care," "turning and back percussion," "airway suctioning," "medication verification," "infusion adjustment," and "wound dressing change." The model outputs the most likely behavior category to which the current video segment belongs and its confidence level.

[0036] In parallel, a regression network branch receives the same fused feature vector and outputs a prescriptive score between 0 and 1. This score reflects the degree to which the currently identified action matches the standard operating procedure template in terms of trajectory, force, rhythm, and angle. The closer the score is to 1, the more standardized the operation. The standard operating procedure template is obtained by collecting a large number of standard operating procedure videos and signals from experienced nurses, and then modeling them after de-personalization processing.

[0037] The final output of step S120 is a structured semantic description encapsulated in JSON format. This description is a structured data object that must contain the following fields: behavior category identifier, behavior confidence score, operational standardization score, associated patient unique identifier, medical device identifier used, and behavior start and end timestamps accurate to milliseconds. This structured description transforms the nurse's continuous, multimodal raw observation data into semantic events that are understandable and reasonable by computers.

[0038] Next, proceed to step S130, which involves establishing a compliance verification engine driven by a clinical intervention knowledge graph. At the core of this engine is a pre-built knowledge graph based on clinical nursing standards and best practices. See [link / reference] Figure 4 This knowledge graph is a semantic network where nodes represent clinical entities, including various nursing procedures, medications, medical devices, patient pathophysiological states, diagnostic results, allergens, contraindications, etc. Edges represent relationships between entities, primarily including: the sequential relationship of procedures, the applicability of procedures to required medications or devices, the matching or contraindication relationship between procedures and specific patient states (such as "post-operative" or "fasting"), the mutual exclusion relationship between different procedures, and the incompatibility relationship between medications. The knowledge graph is constructed by integrating knowledge from national nursing operation standards, hospital internal regulations, medication instructions, clinical pathway guidelines, and medical textbooks, and formally represented using ontology modeling tools, supporting efficient graph querying and logical reasoning.

[0039] After receiving the continuous stream of structured semantic descriptions from step S120, the compliance verification engine immediately initiates multi-level compliance verification logic. First, the engine maps the input structured descriptions to corresponding entities and relationships in a knowledge graph. For example, the identified "intravenous injection" operation is mapped to the "intravenous injection" node in the knowledge graph, the used "0.9% sodium chloride injection solution" is mapped to the "drug" node, and the patient ID is associated with the specific "patient" node and its associated attribute nodes.

[0040] The first layer of validation is a time-series compliance check. The engine retrieves the reasonable execution time window specified in the current patient's electronic medical order. For example, an antibiotic order requires "every 8 hours." The engine checks whether the timestamp of the current "intravenous injection" action falls within the reasonable time window calculated based on the previous execution time. Simultaneously, it checks the order of sub-steps within complex operations. For example, the knowledge graph defines the standard subgraph order for "intravenous injection" as: "verify patient identity" → "check medication and medical order" → "disinfect skin" → "puncture vein" → "fix needle" → "adjust drip rate" → "record." The engine verifies whether the currently identified sequence of steps conforms to this order constraint and checks whether the time interval between adjacent steps is within a reasonable range.

[0041] The second layer of validation is contextual relevance checking. The engine traverses the knowledge graph to verify the match between the current operation and the patient's current state. For example, when the "subcutaneous insulin injection" operation is identified, the engine immediately queries the patient's current state node. If it finds that the patient is in a "hypoglycemic" state, and the knowledge graph defines "hypoglycemia" as a contraindication for "insulin injection," a violation flag is immediately triggered. Similarly, it checks whether the medication used matches the patient's allergy history record and whether the device model used is suitable for the current operation.

[0042] The third layer of validation is a completeness check. Based on the subgraph of operations necessary for a complete intervention as defined in the knowledge graph, the engine dynamically constructs a "set of expected steps." For example, for the intervention topic of "postoperative analgesia pump management," the set of expected steps might include "assessing the patient's pain score," "checking the analgesia pump's operational status," "verifying the remaining dosage," and "observing adverse reactions." The engine compares all identified steps in the current intervention session with the "set of expected steps," marking any missing necessary steps. This check effectively identifies steps omitted due to busyness or negligence.

[0043] After the above three layers of verification, the compliance verification engine will output clear verification results, including: compliance, or the specific type of violation identified, the severity level of the violation, the clinical entities involved, and the reasoning basis. Violation types can be further subdivided into "timing error", "object mismatch", "step omission", "contraindication violation", etc.

[0044] Finally, step S140 is executed, which is the reinforcement learning-based adaptive alarm strategy generation module. The goal of this module is to intelligently decide how to issue an alarm based on the violation information output from step S130, in order to minimize disruption to the nursing workflow while ensuring safety. See also Figure 5 This module is essentially a trained intelligent agent.

[0045] First, define the state space. A state is a tuple whose elements include at least: the currently identified violation type, the current ward context, and the historical frequency of similar violations by the nurse or in the ward. The context is obtained by integrating data from other systems, such as the current ward call frequency from the nurse call system to characterize busyness, and the patient criticality score from the electronic medical record system to characterize the severity of the patient's condition.

[0046] Next, we define the action space. Actions are alarm actions of different levels and forms, forming a discrete set. For example, Action 0: "Silent Recording" only records the violation event into the database without generating any immediate prompts; Action 1: "Minor Prompt" generates a short vibration on the smart badge worn by the nurse; Action 2: "Visual Prompt" pops up a non-modal prompt box on the screen of the mobile nursing cart currently being operated by the nurse; Action 3: "Nurse Station Alarm" highlights the ward and violation information on the central monitoring screen at the nurse station; Action 4: "Escalated Alarm" simultaneously sends SMS or application push notifications to the mobile terminals of the head nurse and supervising nurse, and may trigger local audible and visual alarms.

[0047] Next, a reward function is designed. This function is crucial for guiding the agent's learning. Its design principle is: a positive reward is given when the selected alarm action effectively prompts the nurse to correct the violation without significantly interfering with other normal work in the ward. For example, for high-risk violations, a timely strong alarm and successful prevention of the error yields a high positive reward. Conversely, a negative reward is given if a false alarm occurs (i.e., an alarm is issued for a compliant operation); a larger negative reward is given if a missed alarm occurs (i.e., an alarm of sufficient strength is not issued for a high-risk violation). Furthermore, a slight negative reward is also given for frequent use of high-strength alarms for low-risk violations, leading to "alarm fatigue" in nurses.

[0048] A deep Q-network is optimized by combining offline training with online fine-tuning. In the offline phase, historical violation handling records are used as an experience replay pool for training. In the online phase, the system continuously collects new "state-action-reward-next state" quadruplets during actual operation and periodically fine-tunes the network parameters to adapt to dynamic environments such as updates to hospital nursing standards and changes in staff behavior patterns.

[0049] In actual operation, when a violation event is output in step S130, the adaptive alarm strategy generation module immediately calls the trained deep Q-network model based on the current state to calculate the expected cumulative reward value of all possible alarm actions and selects the action with the highest expected reward to execute. For example, for a medium-risk violation such as "omission of medication verification", if the current ward is extremely busy and the nurse has a good history, the module may choose the "minor prompt" action, only vibrating the smart name tag to provide a prompt; while for an extremely high-risk violation such as "about to administer amoxicillin to a patient allergic to penicillin", regardless of the environmental context, the module will immediately choose the "escalate alarm" action, simultaneously pushing a red flashing alarm on the nurse station screen and the head nurse's mobile phone, and automatically sending a locking command to the smart medicine cabinet in the ward to prevent the medicine from being taken out, thereby building an automated safety defense line.

[0050] All alarm events, their triggering, the basis for strategic decisions, nurses' responses, and the final outcome are meticulously recorded. To further ensure the authority and credibility of the audit trail, the hash values ​​of these critical logs are uploaded in real-time to a permissioned blockchain evidence storage module. Leveraging the immutability and traceability of blockchain, a solid data evidence foundation is provided for medical quality supervision, dispute resolution, and process optimization.

[0051] Example 2 In the routine infusion room setting of a community health service center, nurse interventions are characterized by high frequency of operations, fast pace, large patient turnover, but relatively standardized individual procedures. This second embodiment adapts the system described in the first embodiment to address the specific needs of this scenario.

[0052] The deployment of the multimodal sensing network was specifically adjusted. For video acquisition, wide-angle network cameras were used for panoramic monitoring of the infusion room lobby area, while fixed-focus cameras were deployed above each infusion chair to capture close-ups of infusion tubing, regulators, and patient puncture sites. Wearable devices still use smart name tags, but their sampling strategy focuses on monitoring nurses' movement paths and dwell times to analyze work efficiency and patient rounds. Hospital information system data access focuses more on obtaining simplified electronic medical records and injection slips from outpatients.

[0053] In step S120, the behavior category library of the deep behavior recognition model is simplified and focused, mainly including several categories such as "puncture and catheter placement", "infusion adjustment", "fluid replacement", "needle removal and pressure", and "inspection". Since the lighting conditions in the community scene are relatively stable and there is less occlusion, the model can be appropriately simplified. However, for the key operation of "puncture", the standardization scoring model needs to be specially reinforced to ensure that it can accurately evaluate the standardization points such as needle insertion angle and fixation method.

[0054] In step S130, the construction of the clinical intervention knowledge graph focuses on medication safety and infusion reaction prevention for common outpatient diseases. The knowledge graph strengthens rules regarding drug incompatibilities, the relationship between infusion rate and patient age / disease, etc. The compliance verification engine particularly enhances the integrity check of "circulation observation." The system calculates the estimated completion time based on the total fluid volume and drip rate entered by the patient, and checks whether "circulation observation" behavior occurred before and after this time point. If it is missing, it is marked as a potential risk.

[0055] In step S140, the state space definition of the adaptive alarm strategy generation module adds "the number of patients currently under the nurse's care" as an environmental context indicator. The reward function design emphasizes encouraging "preventative alarms." For example, when the system analyzes excessively long nurse rounds and predicts the risk of not removing the needle promptly after infusion, even if no actual violation occurs, issuing a mild "reminder to round" alarm in advance and successfully avoiding the problem will still earn a positive reward. This makes the system more proactive in warnings in relatively relaxed but easily overlooked environments like communities. Alarm methods primarily use mobile terminal push notifications and indicator lights next to the infusion chairs to reduce interference with the overall environment.

[0056] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for intelligent monitoring of nurse intervention processes combined with deep learning, characterized in that: The method includes the following steps: Step S110: Deploy a multimodal sensing network to collect multi-source heterogeneous data streams in real time during the nurse intervention process. The multi-source heterogeneous data streams include at least the ward environment video stream, the nurse's wearable device signal, and the electronic medical order data in the hospital information system. Step S120: Construct a deep behavior recognition model based on spatiotemporal attention mechanism, extract and fuse features from the multi-source heterogeneous data stream, and generate a structured semantic description of the nurse's intervention behavior. The structured semantic description includes at least the behavior category, the execution object, the operation standardization score, and the timestamp. Step S130: Establish a compliance verification engine driven by clinical intervention knowledge graph, compare and reason with the structured semantic description in real time with the preset clinical nursing standard knowledge graph, and identify and mark potential violations or omissions. Step S140: The reinforcement learning-based adaptive alarm strategy generation module dynamically generates and pushes differentiated intervention reminders or alarm information based on the severity level, frequency of occurrence, and current ward environment context of the violation or omission step.

2. The intelligent monitoring method for nurse intervention process combined with deep learning according to claim 1, characterized in that, In step S110, deploying the multimodal sensing network specifically includes: deploying network cameras with infrared illumination and wide dynamic range functions in key areas of the ward to collect 1080P resolution video streams at a rate of 25 frames per second; equipping nurses with smart name tags integrating inertial measurement units and heart rate sensors to collect nurses' three-axis acceleration, three-axis angular velocity, and heart rate variability signals in real time at a sampling frequency of 10Hz; and acquiring electronic medical orders, nursing plans, and patient vital sign history records related to the current ward and patient in real time through the hospital information system data interface in an event-driven manner.

3. The intelligent monitoring method for nurse intervention process combined with deep learning according to claim 2, characterized in that, In step S110, the multi-source heterogeneous data stream is preprocessed and spatiotemporally aligned: a YOLOv5-based target detection algorithm is applied to the video stream to locate and track the bounding boxes of nurses, patients, and key medical equipment in real time; low-pass filtering and noise reduction are performed on the wearable device signal, and its time series is precisely aligned with the timestamp of the video stream using a dynamic time warping algorithm; the electronic medical order data is parsed into a structured triplet form and associated with the start time of events detected in the video stream.

4. The intelligent monitoring method for nurse intervention process combined with deep learning according to claim 1, characterized in that, In step S120, constructing a deep behavior recognition model based on a spatiotemporal attention mechanism includes: designing a two-stream neural network architecture, wherein the spatial stream takes a preprocessed sequence of video keyframes as input, and the temporal stream takes a sequence of continuous optical flow maps as input; introducing a multi-head self-attention module into the two-stream neural network, the calculation formula of which is: ， in, These represent the query, key, and value matrices, respectively. The dimension of the key vector is denoted as . Simultaneously, the preprocessed and aligned wearable device signal is feature-encoded through an independent one-dimensional convolutional neural network, and its encoded feature vector is concatenated with the fused features of the two-stream network along the feature dimension.

5. The intelligent monitoring method for nurse intervention process combined with deep learning according to claim 4, characterized in that, In step S120, generating a structured semantic description of nurse intervention behavior specifically involves inputting the concatenated multimodal feature vector into a fully connected layer classifier, which outputs a probability distribution corresponding to a preset nurse intervention behavior category library. Simultaneously, through a parallel regression network branch, a standardization score of the current operation relative to the standard operation template is output; the structured semantic description is finally encapsulated in JSON format, including behavior category identifier, confidence level, standardization score, associated patient ID, device ID used, and start and end timestamps accurate to milliseconds.

6. The intelligent monitoring method for nurse intervention process combined with deep learning according to claim 1, characterized in that, In step S130, establishing a compliance verification engine driven by a clinical intervention knowledge graph includes: A knowledge graph centered on clinical nursing standards is pre-constructed. The nodes of this knowledge graph include nursing procedures, medications, instruments, patient status, and contraindications. The edges represent the sequence of operation steps, the applicability of the operation to the medication / instrument, the matching relationship between the operation and the patient status, and the mutual exclusion relationship between operations. After receiving the structured semantic description output in step S120, the compliance verification engine maps it to entities and relationships in a knowledge graph and performs graph traversal and rule reasoning.

7. The intelligent monitoring method for nurse intervention process combined with deep learning according to claim 6, characterized in that, In step S130, the logic for identifying and marking potential violations or omitted steps includes: The timing compliance check verifies whether the currently identified operation is within a reasonable time window specified by the electronic medical order, and whether the time interval between multiple operation steps conforms to the sequential constraints defined in the knowledge graph. Contextual relevance check verifies whether the medication or device used in the current operation matches the patient's current medical orders and allergy history recorded in the knowledge graph; Completeness check: Based on the subgraph of operations required for a complete intervention as defined in the knowledge graph, verify that all necessary steps in the current intervention session have been identified and recorded, and mark any missing nodes.

8. The intelligent monitoring method for nurse intervention process combined with deep learning according to claim 1, characterized in that, In step S140, the adaptive alarm policy generation module based on reinforcement learning includes: Define a state space Its elements are triples <violation type, environmental context, historical frequency>. The environmental context includes the ward's level of activity and the severity of the patient's condition. Define an action space It includes different levels of alarm actions; Design a reward function This function provides a positive reward when the alarm action effectively corrects the violation and does not cause excessive interference with normal nursing work, and provides a negative reward when a false alarm or missed alarm occurs. Optimize a deep Q-network by combining offline training with online fine-tuning, enabling it to learn how to operate in a given state. Select the optimal alarm action. To maximize long-term cumulative rewards.

9. The intelligent monitoring method for nurse intervention process combined with deep learning according to claim 8, characterized in that, In step S140, the dynamic generation and push of differentiated intervention reminders or alarm information is specifically manifested as follows: For medium-risk violations, a slight vibration is generated on the smart badge worn by the nurse as a notification; For high-risk violations, a red high-brightness alarm message will be immediately pushed to the central monitoring screen at the nurse station and the head nurse's mobile terminal, and the relevant medicine cabinet will be automatically locked. All alarm events, triggered strategies, processing results, and subsequent nurse feedback are recorded in the blockchain evidence storage module.

10. An intelligent monitoring system for nurse intervention processes incorporating deep learning, characterized in that: The system includes the following components: Multimodal sensing network module: used to collect multi-source heterogeneous data streams in real time during nurse intervention, the multi-source heterogeneous data streams including at least ward environment video streams, nurse wearable device signals and electronic medical order data in the hospital information system; Deep Behavior Recognition Model Module: Used to extract and fuse features from the multi-source heterogeneous data stream to generate a structured semantic description of nurse intervention behavior. The structured semantic description includes at least behavior category, execution object, operation standardization score, and timestamp. Compliance verification engine module: used to compare and reason with the structured semantic description in real time with the preset clinical nursing standard knowledge graph, and identify and mark potential violations or omissions. Adaptive alarm strategy generation module: used to dynamically generate and push differentiated intervention reminders or alarm information based on the severity level, frequency of occurrence, and current ward environment context of the violation or omission.

Images