Intelligent nursing behavior monitoring system

Abstract

The invention discloses an intelligent nursing behavior monitoring system, and relates to the field of intelligent nursing. Comprising a multi-mode sensing terminal which collects multi-dimensional data of nursing operation through a three-view synchronous collection module and a biosensor array; the edge intelligent analysis node is used for carrying out behavior identification and risk assessment on the collected data by adopting a containerized processing architecture; the block chain evidence storage platform is used for constructing an operation evidence chain which cannot be tampered through a Hyperledger Fabric alliance chain architecture; and the feedback optimization engine adopts a double-model operation framework to be matched with a federated learning framework to realize self-optimization and scene self-adaption of the system. According to the method, the technical problem of space and time information processing separation is effectively solved by adopting a space-time diagram convolutional network technical means, so that deeper and more global understanding and modeling of nursing actions are realized, the recognition precision and robustness of complex and fine actions are greatly improved, and the false alarm rate and the missing report rate are reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent nursing, and in particular to an intelligent nursing behavior monitoring system. BACKGROUND

[0002] The analysis of human skeleton key point motion trajectory and behavior recognition technology is an important research direction in the field of computer vision, which shows a wide application prospect in intelligent monitoring, human-computer interaction, sports analysis, especially intelligent medical monitoring, etc. In the medical scene, this technology is highly expected to be used for automatic and intelligent monitoring of nursing behavior of bedridden patients to evaluate nursing quality, prevent complications such as pressure sores, and ensure that medical orders are strictly implemented.

[0003] At present, the mainstream technical solutions to realize the above-mentioned applications mainly rely on traditional computer vision methods or simple models based on two-dimensional skeletons. These existing technologies usually separate the processing of spatial information and temporal information: first, the spatial features of the key points are extracted from single-frame images using convolutional networks, and then additional recurrent networks or other models are used for time series modeling. This separate processing method is difficult to fully exploit the inherent structured spatial connection relationship between human key points and the time sequence evolution law in continuous motion, resulting in a bottleneck in the analysis ability of complex nursing actions (such as turning over and patting back). Specifically, the model's perception range is localized, it cannot understand the human posture globally, it is not sensitive to subtle but critical nursing actions, and it is difficult to effectively distinguish the standardization of actions, thereby causing the problems of insufficient recognition accuracy and high misjudgment rate. In addition, many technical solutions have high computational complexity, making it difficult to realize low-latency real-time analysis on resource-constrained edge computing devices, which cannot meet the stringent requirements of clinical monitoring for timeliness.

[0004] Therefore, the present application proposes an intelligent nursing behavior monitoring system SUMMARY

[0005] The purpose of the present application is to solve the problems existing in the prior art, and an intelligent nursing behavior monitoring system is proposed.

[0006] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] The intelligent nursing behavior monitoring system comprises:

[0008] A multi-modal perception terminal acquires multi-dimensional data of nursing operations through a three-view synchronous acquisition module and a biological sensor array;

[0009] An edge intelligent analysis node adopts a containerized processing architecture to perform behavior recognition and risk assessment on the acquired data;

[0010] A blockchain evidence storage platform that constructs an immutable chain of operational evidence using the Hyperledger Fabric consortium blockchain architecture;

[0011] The feedback optimization engine employs a dual-model operating architecture combined with a federated learning framework to achieve system self-optimization and scenario adaptation.

[0012] Preferably, the three-view synchronous acquisition module includes:

[0013] The top-view camera is installed 1.5 meters directly above the hospital bed, filming the patient's full body movements vertically downwards.

[0014] A lateral depth camera is mounted at a 45-degree angle on the side of the bed to measure the angle of limb movement.

[0015] A front-viewing RGB camera, positioned at the level of the patient's head, identifies the tools used and hand movements.

[0016] Preferably, the biosensor array includes a mattress pressure sensor for detecting body pressure distribution and an inertial measurement unit located at the human skeletal joints.

[0017] Preferably, the containerized architecture of the edge intelligent analysis node includes:

[0018] Decrypting the container: Decrypting the data stream using the AES-256-GCM algorithm;

[0019] Identify containers: Deploy a quantized version of the ST-GCNv2 model;

[0020] Alarm container: Executes a dynamic threshold decision engine.

[0021] Preferably, the implementation principle of the edge intelligent analysis node is as follows:

[0022] Spatiotemporal graph convolutional network is used to analyze the motion trajectory of skeletal key points. The network input is a tensor of dimension m×n×p×q, where m is the number of channels, n is the number of frames, p is the number of human joints with deployed sensors, and q is the number of people.

[0023] Using the graph convolution formula Extract spatial features, where F in Let D be the input feature matrix, and D be the degree matrix. The adjacency matrix before normalization The symmetric normalized adjacency matrix, where W is the weight matrix, b is the bias vector, and σ is the nonlinear activation function;

[0024] Combining temporal convolution to analyze the continuity of actions.

[0025] Preferably, the contracts of the blockchain evidence storage platform include:

[0026] Data verification contract: check the consistency of the hash value;

[0027] Alarm triggering contract: automatically associate electronic medical records.

[0028] Preferably: the working principle of the blockchain storage platform is:

[0029] When the edge node triggers an alarm, a structured storage data packet is generated, including operation type, timestamp, confidence threshold, data fingerprint element;

[0030] After consensus among medical, regulatory and technical parties is reached through the PBFT consensus mechanism, it is written into the distributed ledger.

[0031] Preferably: the feedback optimization engine adds Gaussian noise of N(0, 0.01 2 I) to the differential privacy mechanism, which aggregates new samples from multiple medical institutions every period, filters the training set through a triple active learning strategy, and updates the model parameters using an exponentially decaying learning rate (lr = lr0 x e -0.1t ).

[0032] The intelligent nursing behavior monitoring method comprises the following steps:

[0033] S1: Synchronously collect multi-angle video streams of nursing operations through a spatially distributed three-view visual sensor array, including top view, side view and front view;

[0034] S2: Implement hierarchical privacy protection processing at the data acquisition terminal, use an adaptive Gaussian blur algorithm for face regions, and implement rectangular mask coverage for non-operation sensitive regions;

[0035] S3: Deploy a quantized space-time graph convolution network at the edge computing node, input a 3x30x17x1 tensor data, extract spatial features through a graph convolution formula, and analyze action continuity in combination with a time series convolution;

[0036] S4: Dynamically update the behavior recognition threshold based on the exponential moving average algorithm, and the dynamic threshold calculation formula is: τ t is the threshold value at time t, α is the smoothing coefficient, α ∈ (0.85, 0.95), is the current period confidence mean;

[0037] S5: When the recognition confidence is lower than the dynamic threshold, a structured alarm data packet is generated and stored through the alliance chain, and the storage content includes operation type, timestamp, device fingerprint and data hash value;

[0038] S6: receiving the operation feedback data of the medical staff, triggering shadow model training when the concept drift index CDI is greater than 0.15, and aggregating multi-institutional model parameters in a federated learning framework.

[0039] Preferably: in the S2 step, the hierarchical privacy protection includes a three-level mechanism.

[0040] Terminal-level processing: a lightweight MTCNN model is used to detect the face region in real time, and adaptive Gaussian blur is implemented;

[0041] Transmission-level protection: after the video stream is compressed in real time by Zstandard, it is encrypted and transmitted by using the AES-256-GCM algorithm, and the key is changed every 30 minutes;

[0042] Processing-level desensitization: only 17 skeleton key point coordinates are retained in the behavior recognition stage, and the original image data is deleted.

[0043] The beneficial effects of the present application are:

[0044] 1. The application adopts the spatio-temporal graph convolution network technology, effectively solves the technical problem of the separation of space and time information processing, and has the following beneficial effects: the network can naturally and synchronously fuse the spatial topological relationship between nodes and the motion time sequence features between continuous frames through its unique graph structure operation, thereby realizing deeper and more global understanding and modeling of nursing actions, which not only greatly improves the recognition accuracy and robustness of complex and subtle actions, reduces the false positive and false negative rates, but also provides a highly accurate data basis for subsequent blockchain trusted evidence, and ensures the reliability of the entire monitoring system decision. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 The working method flowchart of the intelligent nursing behavior monitoring system proposed in the application. DETAILED DESCRIPTION

[0046] The technical solutions of the application will be further described in detail below in combination with specific embodiments.

[0047] In the description of the application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting", "setting" should be understood broadly, for example, it can be fixedly connected, set, or detachably connected, set, or integrally connected, set. For those skilled in the art, the specific meaning of the above terms in the application can be understood according to the specific circumstances.

[0048] Example 1:

[0049] The intelligent nursing behavior monitoring system comprises:

[0050] A multi-modal perception terminal acquires multi-dimensional data of nursing operations through a three-view synchronous acquisition module and a biosensor array;

[0051] An edge intelligent analysis node uses a containerized processing architecture to perform behavior recognition and risk assessment on the acquired data;

[0052] A blockchain storage platform constructs an operation evidence chain that is not tamperable through a Hyperledger Fabric consortium chain architecture;

[0053] A feedback optimization engine uses a dual-model running architecture to implement system self-optimization and scenario self-adaptation in cooperation with a federated learning framework.

[0054] The three-view synchronous acquisition module includes:

[0055] A top-view camera, which is installed 1.5 meters above the patient bed and vertically downward to capture the patient's whole body movement;

[0056] A side-view depth camera, which is installed at a 45-degree angle on the side of the bed to measure the limb movement angle;

[0057] A front-view camera, which is flush with the patient's head to identify the operation tool and hand movement.

[0058] The biosensor array includes a mattress pressure sensor for detecting body pressure distribution and an inertial measurement unit arranged at the human body joints.

[0059] The containerized architecture of the edge intelligent analysis node includes:

[0060] Decryption container: AES-256-GCM algorithm decrypts data stream;

[0061] Identification container: deploy quantized ST-GCNv2 model (FP16 precision);

[0062] Alarm container: execute dynamic threshold decision engine.

[0063] The implementation principle of the edge intelligent analysis node is:

[0064] A spatio-temporal graph convolutional network is used to analyze the skeletal key point motion trajectory, and the network input is a tensor with dimensions m x n x p x q, where m is the channel number, n is the frame number, p is the number of human body joints of the deployed sensor, and q is the number of people;

[0065] Spatial features are extracted through a graph convolution formula, where X is the input feature matrix, D is the degree matrix, A is the adjacency matrix before normalization, A' is the symmetric normalized adjacency matrix, W is the weight matrix, b is the bias vector, and f is the nonlinear activation function;

[0066] The action continuity is analyzed by combining the time sequence convolution.

[0067] The contract of the blockchain storage platform includes:

[0068] Data verification contract: check the consistency of the hash value;

[0069] Alarm triggering contract: automatically associate electronic medical records.

[0070] The working principle of the blockchain storage platform is:

[0071] When the edge node triggers an alarm, a structured storage data packet is generated, including operation type, timestamp, confidence threshold, and data fingerprint elements;

[0072] After reaching an agreement among the medical, regulatory, and technical parties through the PBFT consensus mechanism, it is written into the distributed ledger.

[0073] The feedback optimization engine adds Gaussian noise using the differential privacy mechanism, aggregates new samples from multiple medical institutions every cycle, filters the training set through a triple active learning strategy (uncertainty sampling, diversity sampling, and outlier sampling), and updates the model parameters using an exponentially decaying learning rate.

[0074] Embodiment 2:

[0075] The intelligent nursing behavior monitoring method includes the following steps:

[0076] S1: Synchronously collect multi-angle video streams of nursing operations through a spatially distributed three-view visual sensor array, including top view, side view, and front view, with a time synchronization error of less than 10 milliseconds for each view;

[0077] S2: Implement hierarchical privacy protection processing at the data acquisition terminal, using an adaptive Gaussian blur algorithm (blur radius dynamically adjusted to 12-18 pixels) for face regions, and implementing rectangular mask coverage for non-operation sensitive areas;

[0078] S3: Deploy a quantized space-time graph convolution network on the edge computing node, with input dimensions of 3x30x17x1 tensor data (corresponding to coordinate channel, time frame, joint node, and number of personnel, respectively), extract spatial features through graph convolution formula, and analyze action continuity by combining time sequence convolution;

[0079] S4: Dynamically update the behavior recognition threshold based on the exponential moving average algorithm, with the dynamic threshold calculation formula as: τ t is the threshold value at time t, α is the smoothing coefficient, α ∈ (0.85, 0.95), is the current period confidence mean;

[0080] S5: When the recognition confidence is lower than the dynamic threshold, a structured alarm data package is generated and stored through the alliance chain, and the storage content includes operation type, timestamp, device fingerprint, and data hash value;

[0081] S6: Receive operation feedback data of medical staff, trigger shadow model training when the concept drift index CDI> 0.15, and aggregate multi-institution model parameters using a federated learning framework.

[0082] Embodiment 3:

[0083] The intelligent nursing behavior monitoring method comprises the following steps:

[0084] S1: Synchronously collect multi-angle video streams of nursing operations through a spatially distributed three-view visual sensor array, including top view, side view and front view, and the time synchronization error of each view is less than 10 milliseconds;

[0085] S2: Implement layered privacy protection processing in the data acquisition terminal, use an adaptive Gaussian blur algorithm (blur radius dynamically adjusted to 12-18 pixels) for face area, and implement rectangular mask coverage for non-operation sensitive areas;

[0086] S3: Deploy a quantized space-time graph convolution network in an edge computing node, input dimension is a 3x30x17x1 tensor data (corresponding to coordinate channel, time frame, joint node and number of personnel respectively), extract spatial features through graph convolution formula, and analyze action continuity combined with time series convolution;

[0087] S4: Dynamically update the behavior recognition threshold based on the exponential moving average algorithm, and the dynamic threshold calculation formula is: τ t is the threshold value at time t, α is the smoothing coefficient, α∈(0.85, 0.95), is the current period confidence mean;

[0088] S5: When the recognition confidence is lower than the dynamic threshold, a structured alarm data package is generated and stored through the alliance chain, and the storage content includes operation type, timestamp, device fingerprint, and data hash value;

[0089] S6: Receive operation feedback data of medical staff, trigger shadow model training when the concept drift index CDI> 0.15, and aggregate multi-institution model parameters using a federated learning framework.

[0090] In the S2 step, the layered privacy protection includes a three-level mechanism:

[0091] Terminal level processing: Real-time face area detection using a lightweight MTCNN model, adaptive Gaussian blur (kernel size 25x25 pixels, σ value dynamically adjusted to 12-18 according to scene lighting);

[0092] Transport level protection: after Zstandard real-time compression (compression ratio 3:1) of the video stream, the AES-256-GCM algorithm is used for encryption transmission, and the key is changed every 30 minutes;

[0093] Processing level desensitization: only 17 skeleton key point coordinates are reserved in the behavior recognition stage, and the original image data is deleted.

[0094] The application effectively solves the technical problem of the split of space and time information processing by adopting the spatio-temporal graph convolution network technical means, and the significant beneficial effects brought by the application are that: the network can naturally and synchronously fuse the spatial topological relationship between the key points and the motion time sequence characteristics between the continuous frames through its unique graph structure operation, thereby realizing deeper and more global understanding and modeling of the nursing action, which not only greatly improves the recognition accuracy and robustness of complex and subtle actions, reduces the false positive and false negative rates, but also provides a highly accurate data basis for subsequent blockchain trusted evidence, and ensures the reliability of the whole monitoring system decision.

[0095] The above describes only the preferred specific embodiments of the application, but the protection scope of the application is not limited thereto, any person skilled in the art can make equivalent replacement or change according to the technical scheme and the inventive concept of the application within the technical range disclosed by the application, which should be covered within the protection scope of the application.

Claims

1. An intelligent nursing behavior monitoring system, characterized in that, include: The multimodal sensing terminal collects multidimensional data of nursing operations through a three-view synchronous acquisition module and a biosensor array; Edge intelligent analysis nodes employ a containerized processing architecture to perform behavior recognition and risk assessment on the collected data. A blockchain evidence storage platform that constructs an immutable chain of operational evidence using the Hyperledger Fabric consortium blockchain architecture; The feedback optimization engine employs a dual-model operating architecture combined with a federated learning framework to achieve system self-optimization and scenario adaptation.

2. The intelligent nursing behavior monitoring system according to claim 1, characterized in that, The three-view synchronous acquisition module includes: The top-view camera is installed 1.5 meters directly above the hospital bed, filming the patient's full body movements vertically downwards. A lateral depth camera is mounted at a 45-degree angle on the side of the bed to measure the angle of limb movement. A front-viewing RGB camera, positioned at the level of the patient's head, identifies the tools used and hand movements.

3. The intelligent nursing behavior monitoring system according to claim 1, characterized in that, The biosensor array includes a mattress pressure sensor for detecting body pressure distribution and an inertial measurement unit located at the human skeletal joints.

4. The intelligent nursing behavior monitoring system according to claim 1, characterized in that, The containerized architecture of the edge intelligent analysis node includes: Decrypting the container: Decrypting the data stream using the AES-256-GCM algorithm; Identify containers: Deploy a quantized version of the ST-GCNv2 model; Alarm container: Executes a dynamic threshold decision engine.

5. The intelligent nursing behavior monitoring system according to claim 4, characterized in that, The implementation principle of the edge intelligent analysis node is as follows: Spatiotemporal graph convolutional network is used to analyze the motion trajectory of skeletal key points. The network input is a tensor of dimension m×n×p×q, where m is the number of channels, n is the number of frames, p is the number of human joints with deployed sensors, and q is the number of people. Using the graph convolution formula Extract spatial features, where F in Let D be the input feature matrix, and D be the degree matrix. The adjacency matrix before normalization. The symmetric normalized adjacency matrix, where W is the weight matrix, b is the bias vector, and σ is the nonlinear activation function; Combining temporal convolution to analyze the continuity of actions.

6. The intelligent nursing behavior monitoring system according to claim 1, characterized in that, The contracts of the blockchain-based evidence storage platform include: Data verification contract: Verify hash value consistency; Alarm trigger contract: Automatically link to electronic medical records.

7. The intelligent nursing behavior monitoring system according to claim 6, characterized in that, The working principle of the blockchain-based evidence storage platform is as follows: When an edge node triggers an alarm, a structured evidence data packet is generated, which includes the operation type, timestamp, confidence threshold, and data fingerprint elements. After reaching a consensus among the three nodes—medical, regulatory, and technical parties—through the PBFT consensus mechanism, the agreement is written into the distributed ledger.

8. The intelligent nursing behavior monitoring system according to claim 1, characterized in that, The feedback optimization engine employs a differential privacy mechanism to add N(0, 0.01). 2 I) Gaussian noise, which aggregates new samples from multiple medical institutions each cycle, and selects the training set through a triple active learning strategy, using an exponentially decaying learning rate (lr=lr0×e). -0.1t Update model parameters.

9. A method for monitoring intelligent nursing behavior, which is the method of the intelligent nursing behavior monitoring system according to any one of claims 1-8, characterized in that, Includes the following steps: S1: Through a spatially distributed three-view vision sensor array, multi-angle video streams of nursing operations are simultaneously acquired, including top view, side view and front view. S2: Implement layered privacy protection processing in the data acquisition terminal, use an adaptive Gaussian blur algorithm for the face area, and implement rectangular mask coverage for non-operation-sensitive areas; S3: Deploy a quantized version of the spatiotemporal graph convolutional network on edge computing nodes, input tensor data with dimensions of 3×30×17×1, extract spatial features through graph convolution formula, and combine temporal convolution to analyze action continuity; S4: Dynamically update the behavior recognition threshold based on the exponential moving average algorithm. The dynamic threshold calculation formula is as follows: τ t Let be the threshold at time t, and α be the smoothing coefficient, α∈(0.85, 0.95). This represents the average confidence level for the current time period. S5: When the confidence level of identification is lower than the dynamic threshold, a structured alarm data packet is generated and stored through the consortium blockchain. The stored content includes the operation type, timestamp, device fingerprint, and data hash value. S6: Receives operational feedback data from medical staff. When the concept drift index CDI > 0.15, it triggers shadow model training and uses a federated learning framework to aggregate multi-institution model parameters.

10. The intelligent nursing behavior monitoring system according to claim 1, characterized in that, In step S2, the layered privacy protection includes a three-level mechanism: Terminal-level processing: A lightweight MTCNN model is used to detect face regions in real time and adaptive Gaussian blur is applied; Transmission-level protection: After real-time compression of the video stream using Zstandard, it is encrypted and transmitted using the AES-256-GCM algorithm, with the key rotating every 30 minutes; Processing-level desensitization: During the behavior recognition stage, only the coordinates of 17 skeletal key points are retained, and the original image data is deleted.

Images