Surgical nursing operation compliance image intelligent detection method and system

Abstract

This invention discloses an intelligent detection method and system for compliance images of surgical nursing operations, belonging to the field of image detection and processing technology. The method includes: first, acquiring operating room monitoring video streams and extracting the spatial mask of sterile areas, the outlines of nursing instruments, and key skeletal points; based on this, constructing a spatial attention matrix and an operational topology diagram of hand-instrument interaction to generate a spatiotemporal feature map to be detected; next, calling compliance benchmark reconstruction logic to reconstruct features, calculating pixel-level feature differences and performing weighted mapping to generate an initial anomaly saliency map; finally, denoising and spatially weighted aggregation are performed on the anomaly saliency map to output the detection results. This invention learns the compliance feature distribution through reconstruction logic, avoiding dependence on scarce violation samples, and enhances the feature representation of high-risk areas and interactive behaviors by utilizing spatial attention and topology, significantly improving the detection sensitivity and accuracy of images of rare violations in surgical nursing operations.

Description

Technical Field

[0001] This invention relates to the field of image detection and processing technology, and in particular to an intelligent detection method and system for images related to compliance of surgical nursing procedures. Background Technology

[0002] Surgical nursing work is characterized by its fast pace, high technicality, and high risk. As the core treatment area of ​​a hospital, the operating room's nursing procedures (such as aseptic techniques, inventory of surgical instruments, patient positioning, and specimen management) directly impact patient safety. Furthermore, with the development of computer vision, deep learning, object detection, and action recognition technologies, intelligent image detection can now perform behavioral analysis and violation identification during medical procedures.

[0003] Currently, existing technologies for intelligent image detection of surgical nursing procedures mainly rely on computer vision and deep learning to build automated monitoring processes. These processes achieve intelligent supervision of nursing behaviors through multi-stage analysis of operating room video data. Typically, this involves first acquiring real-time video streams using high-definition cameras deployed in the operating room, and then preprocessing the acquired image data with noise reduction, enhancement, normalization, and image stabilization to reduce the impact of complex environmental factors on recognition accuracy. Subsequently, a convolutional neural network model is used to extract features and detect targets in the images, identifying key targets such as medical staff, surgical instruments, and sterile areas. Semantic segmentation technology is then used to spatially divide sterile and non-sterile areas. Based on this, human pose estimation methods are used to extract key skeletal information of nursing staff, and temporal modeling algorithms are used to analyze the spatiotemporal features of continuous movements, thereby identifying specific nursing procedures. Finally, the recognition results are matched with a pre-built nursing operation rule base, and compliance is judged based on spatial constraints and action logic. When violations are detected, warning information is automatically generated and the detection results are output, achieving a closed-loop detection process from image acquisition and behavior recognition to compliance assessment.

[0004] For example, the intelligent scalp detection system and its detection method disclosed in Chinese invention patent with announcement number CN120198437B include: obtaining shooting conditions and clarity assessment through real-time environmental perception and quality detection, using deep learning segmentation and targeted enhancement to solve hair occlusion and image noise, displaying the deep structure of the scalp by means of multimodal fusion and stitching, and outputting diagnostic and optimization strategies in comprehensive analysis and adaptive closed-loop feedback.

[0005] However, in the process of implementing the inventive technical solution in the embodiments of this application, it was found that the above-mentioned technology has at least the following technical problems:

[0006] In actual operating room settings for medical care, the vast majority of nursing procedures comply with standardized procedures. Real-world violations (such as leftover gauze or contamination of sterile areas) occur very infrequently and manifest in diverse ways, leading to extreme class imbalance in the training data. This long-tailed data distribution makes it difficult for existing deep learning models to acquire sufficient negative sample (violation sample) features during training. The model's decision boundary tends to shift towards compliant samples, resulting in overfitting. Consequently, models trained in this way lack sufficient sensitivity and generalization ability for rare, high-risk violations in practical applications, easily leading to missed detections and failing to meet the stringent requirements of surgical nursing safety management for accurate detection of high-risk violation images. Summary of the Invention

[0007] To address the technical problems of existing technologies, such as data imbalance due to the scarcity of violation samples in surgical nursing operations, which makes it difficult for models to learn violation features, and low sensitivity to images of rare violations, resulting in frequent missed detections, this invention provides an intelligent detection method and system for compliance images of surgical nursing operations. The technical solution is as follows:

[0008] On the one hand, an intelligent detection method for compliance images of surgical nursing operations is provided. This method includes: acquiring real-time monitoring video streams of operating room nursing scenarios; decoding and normalizing the real-time monitoring video streams to obtain continuous image frames; extracting the spatial mask of the sterile area corresponding to the image to be detected, the contour boundary of the nursing instruments, and the skeletal key points of the medical staff from the continuous image frames; constructing a spatial attention matrix reflecting the spatial risk distribution based on the spatial mask of the sterile area; and constructing an operation topology diagram representing the hand-instrument interaction state based on the relative positional relationship between the skeletal key points of the medical staff and the contour boundary of the nursing instruments; and mapping the operation topology diagram to the feature space. Encoding is performed, and a spatiotemporal feature map to be detected is generated by combining time dimension information. The pre-built compliance benchmark reconstruction logic is called to reconstruct the spatiotemporal feature map to be detected, resulting in a reconstructed spatiotemporal feature map. The pixel-level feature differences between the spatiotemporal feature map to be detected and the reconstructed spatiotemporal feature map are calculated, and the pixel-level feature differences are weighted and mapped using a spatial attention matrix to generate an initial anomaly saliency map. The initial anomaly saliency map is processed by local connected component processing to filter out discrete noise interference, and a spatially sensitive anomaly saliency map is output. The spatially sensitive anomaly saliency map is spatially weighted and aggregated to obtain a comprehensive violation confidence score. The image detection result is output based on the relationship between the comprehensive violation confidence score and a preset threshold.

[0009] On the other hand, a surgical nursing operation compliance image intelligent detection system is provided, including: a nursing operation image acquisition module, a spatiotemporal feature map generation module, a compliance logic feature reconstruction module, and an image detection result output module; the nursing operation image acquisition module is used to acquire real-time monitoring video streams of operating room nursing scenarios, decode and normalize the real-time monitoring video streams to obtain continuous image frames, and extract the spatial mask of the sterile area corresponding to the image to be detected, the contour boundary of the nursing instrument, and the skeletal key points of the medical staff in the continuous image frames; the spatiotemporal feature map generation module is used to construct a spatial attention matrix reflecting the spatial location risk distribution based on the spatial mask of the sterile area, and construct an operation topology representing the hand-instrument interaction state based on the relative positional relationship between the skeletal key points of the medical staff and the contour boundary of the nursing instrument. The topology diagram is mapped to the feature space for encoding, and combined with time dimension information to generate a spatiotemporal feature map to be detected. The compliance logic feature reconstruction module calls the pre-built compliance benchmark reconstruction logic to reconstruct the spatiotemporal feature map to be detected, and calculates the pixel-level feature differences between the spatiotemporal feature map to be detected and the reconstructed spatiotemporal feature map. The spatial attention matrix is ​​used to weight and map the pixel-level feature differences to generate an initial anomaly saliency map. The image detection result output module performs local connected component processing on the initial anomaly saliency map to filter out discrete noise interference, outputs a spatially sensitive anomaly saliency map, performs spatial weighted aggregation calculation on the spatially sensitive anomaly saliency map to obtain the comprehensive violation confidence, and outputs the image detection result according to the relationship between the comprehensive violation confidence and the preset threshold.

[0010] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0011] This invention achieves highly sensitive, accurate, and reliable intelligent monitoring of surgical nursing operations through multi-level and multi-dimensional spatiotemporal feature analysis and anomaly detection, providing effective technical support for operating room nursing safety management and showing significant advantages compared with existing technologies.

[0012] First, this invention can extract the spatial mask of sterile areas, the outline of nursing instruments, and key skeletal points of medical personnel from consecutive image frames, achieving comprehensive perception of the surgical environment, instruments, and operational actions. Compared with existing methods that rely solely on local features or static image analysis, this invention provides more refined spatial information and operational posture data, laying the foundation for the model to learn the features of scarce violation samples and improving sensitivity to potential risk areas.

[0013] Secondly, this invention constructs a spatial attention matrix and a hand-instrument operation topology diagram, and combines this with a time dimension to generate a spatiotemporal feature map, which can reflect the dynamic behavior and spatial risk distribution in nursing operations. Unlike existing technologies that mainly rely on single-frame images or empirical rules for judgment, this invention captures complex operational behaviors through spatiotemporal feature analysis, enhancing the model's ability to identify and generalize different types of violations, and effectively solving the problem of difficulty in detecting high-risk violations.

[0014] Furthermore, by reconstructing the spatiotemporal feature map using a compliance benchmark model and calculating pixel-level feature differences to generate anomaly saliency maps, this invention achieves sensitive detection of potential violations. Compared to traditional deep learning methods, which are prone to overfitting and missing violation samples due to data class imbalance, this invention can highlight abnormal regions, enhance the detection capability of rare violations, significantly reduce the risk of missed detections, and improve detection accuracy.

[0015] Finally, by processing locally connected components and optimizing the anomaly saliency map through spatial weighted aggregation, this invention generates a comprehensive violation confidence score and outputs the final detection result, achieving a reliable judgment of abnormal behavior. Compared with existing methods that are easily affected by noise interference or false alarms, this invention improves detection stability and accuracy, enabling real-time monitoring and precise early warning of high-risk violations in actual operating room scenarios, meeting the requirements of nursing safety management for high precision and high reliability.

[0016] In summary, this invention, through multi-level and multi-dimensional spatiotemporal feature analysis and anomaly detection, not only solves the problems of insufficient model learning and serious missed detection caused by the scarcity of violation samples in the prior art, but also provides higher sensitivity, stronger generalization ability and more stable and reliable violation detection effect, providing strong technical support for the safety management of surgical nursing operations. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of the intelligent image detection method for surgical nursing operation compliance provided in this application embodiment;

[0019] Figure 2 This application provides an overall detection architecture diagram for the intelligent detection method of surgical nursing operation compliance images.

[0020] Figure 3This is a schematic diagram of the structure of the intelligent image detection system for surgical nursing operation compliance provided in this application embodiment. Detailed Implementation

[0021] Embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of the present disclosure are shown in the drawings, it should be understood that embodiments of the present disclosure may be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure.

[0022] It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure. In the description of the embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "this embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects.

[0023] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0024] In modern operating rooms, the safety of surgical nursing procedures is directly related to patient safety and surgical quality. While the vast majority of nursing procedures strictly adhere to protocols, a small number of high-risk violations still occur, such as contamination of the sterile area, leaving gauze behind, or incorrect instrument use. These violations, if they occur, can lead to postoperative infections, complications, or even serious medical accidents. However, because these violations occur very infrequently and manifest in diverse ways, the sample size of violations in actual surgical video data is extremely scarce, resulting in a typical long-tail distribution problem.

[0025] In existing technologies, the application of intelligent analysis methods such as deep learning in operating room scenarios still faces significant challenges: the model training phase struggles to acquire sufficient features from violation samples, easily leading to overfitting to compliant samples. This results in low sensitivity and high false negative rates for detecting rare violations in practical applications. Consequently, traditional methods are insufficient to meet the real-time monitoring and accurate detection needs of medical institutions for high-risk violations.

[0026] Therefore, there is an urgent need for an intelligent analysis method that can effectively address the scarcity of violation samples and improve the model's ability to detect rare, high-risk violations. This method not only needs to be able to extract key features of operational behaviors in the complex environment of the operating room, but also needs to achieve highly sensitive detection and stable identification of potential violations, thereby providing reliable technical support for surgical nursing safety management, ensuring patient safety, and reducing medical risks.

[0027] like Figure 1 The diagram shown is a flowchart of the intelligent image detection method for surgical nursing operation compliance provided in this application embodiment. (Refer to...) Figure 1 As the first step of the intelligent detection method for compliance images of surgical nursing operations, the method is as follows: acquire the real-time monitoring video stream of the operating room nursing scene, decode and normalize the real-time monitoring video stream to obtain continuous image frames, and extract the spatial mask of the sterile area corresponding to the image to be detected, the outline boundary of the nursing instrument, and the skeletal key points of the medical staff in the continuous image frames.

[0028] It should be added that high-definition network surveillance cameras deployed around the operating room's shadowless lamps and in the corners of the walls are used to collect real-time video stream data of the operating room nursing scene. The FFmpeg multimedia framework is used to decode the acquired real-time monitoring video stream, converting the compressed video stream into a continuous sequence of raw image frames. Subsequently, each raw image frame undergoes normalization preprocessing: first, the image size is uniformly scaled to a preset resolution (e.g., 640×640 pixels or 224×224 pixels) to adapt to the input requirements of the subsequent neural network; then, the image pixel values ​​are mapped from the integer range of 0-255 to the floating-point range of [0,1] or [-1,1], and mean and variance normalization is performed to eliminate the influence of light intensity changes on feature extraction, resulting in the standard image frame to be detected.

[0029] A semantic segmentation model based on a fully convolutional network (FCN) or U-Net architecture is used to identify sterile regions in normalized image frames. Specifically, standard image frames are input into a pre-trained segmentation network, which outputs a pixel-level classification probability map with the same size as the input image. The probability map is binarized by setting a threshold (e.g., 0.5), and pixel regions with probability values ​​higher than the threshold are marked as 1 (representing sterile regions), while the rest are marked as 0 (representing background regions), thus obtaining a spatial mask for the sterile regions. This mask accurately identifies the pixel-level locations of key areas such as sterile operating tables and instrument carts.

[0030] Deep learning object detection and instance segmentation models (such as Mask R-CNN or YOLOv8-seg) are used to detect and segment nursing instruments in images. After inputting standard image frames into the model, the model outputs category labels, bounding boxes, and instance masks for the nursing instruments (such as scalpels, hemostatic forceps, gauze, etc.). The Canny edge detection algorithm or contour finding algorithm is applied to the instance masks to extract the precise outer contour boundary coordinate sequence of the nursing instruments. This contour boundary not only contains the positional information of the instrument but also preserves the morphological details of the instrument, providing geometric feature support for subsequent analysis of the instrument's state (such as whether it is properly placed or being held).

[0031] Human pose estimation networks (such as OpenPose, HRNet, or MediaPipe) are used to extract skeletal key points from images of medical staff. Standard image frames are input into the pose estimation network, which locates and outputs the pixel coordinates and confidence scores of the main joints of the medical staff's body, focusing on extracting the coordinates of skeletal nodes closely related to nursing operations, such as hand key points (wrist and finger joints), elbows, and shoulders. For occluded key points, temporal information or interpolation algorithms based on symmetry constraints are used for completion and correction to ensure the continuity and integrity of the skeletal key point sequence, thereby accurately representing the hand movements and postures of medical staff.

[0032] By standardizing the preprocessing of the original video, the interference of environmental factors such as lighting and resolution on image detection is eliminated, and the quality of input data is improved. Multimodal feature extraction enables accurate segmentation and localization of three-dimensional elements such as "regions, objects and people" in the image, providing a structured visual data foundation for subsequent detection and effectively avoiding feature confusion caused by complex surgical backgrounds.

[0033] The second step of the intelligent detection method for compliance images of surgical nursing operations is as follows: a spatial attention matrix reflecting the spatial risk distribution is constructed based on the spatial mask of the sterile area, and an operation topology diagram representing the hand-instrument interaction state is constructed based on the relative positional relationship between the skeletal key points of medical staff and the contour boundary of nursing instruments. The operation topology diagram is mapped to the feature space for encoding, and the spatiotemporal feature map to be detected is generated by combining the time dimension information.

[0034] By introducing a spatial attention matrix to simulate the visual attention mechanism, image detection can adaptively focus on high-risk sterile areas, significantly improving the feature response intensity of key areas. By constructing an operational topology graph and integrating the time dimension, static image detection is extended to dynamic behavior recognition, which can accurately capture the interaction details between hands and instruments in image sequences, solving the defect of traditional static detection that cannot identify continuous violations.

[0035] In the second step, it should be noted that the specific process of constructing the spatial attention matrix is ​​as follows:

[0036] First, the sterile region spatial mask generated in the previous step is obtained. This mask is a binary image, where a pixel value of 1 represents a sterile region and a pixel value of 0 represents a non-sterile region. The Euclidean distance transform algorithm is then used to process this binary mask. Specifically, for each pixel in the image, the Euclidean distance from that pixel to the nearest non-sterile region (i.e., the region with a mask value of 0, corresponding to the boundary of the sterile region) is calculated. This generates a distance matrix with the same size as the original image. The value of each element in the matrix Represents pixels The shortest distance to the boundary of the sterile area. For pixels located on the boundary of the sterile area, the distance is 0; for pixels located inside the sterile area, the distance gradually increases as you move deeper into the center of the area.

[0037] To quantify the risk distribution at different spatial locations, a negative correlation between risk weights and distance values ​​is defined; that is, the closer to the sterile boundary, the higher the risk of violation, and the greater the attention weight. A pre-defined Gaussian decay function is used to adjust the distance matrix. The mapping is performed, and the calculation formula is as follows: ;in, For pixels Risk weights; This is the amplitude adjustment coefficient, used to control the maximum value of the weight (usually set to 1). This is a scaling parameter used to control the decay rate, i.e., the rate at which the risk decreases with distance from the boundary. Its value can be set according to the definition of "safe distance in the sterile area" in aseptic technique specifications (e.g., setting...). This calculation causes the weight of pixels located more than 5cm from the boundary to decay to a minimum. Pixels near the boundary of the sterile region will receive a high weight close to 1, while pixels far from the boundary or in the non-sterile region will have a weight close to 0 or remain low.

[0038] The calculated risk weights of all pixels Arrange them according to their corresponding coordinate positions to construct a two-dimensional matrix that is exactly the same size as the image to be detected; this is the spatial attention matrix. The matrix exhibits a numerical distribution characterized by a highly weighted centroid that is highlighted at the boundary of the sterile area, decaying outwards in a Gaussian curvature. This spatial attention matrix will serve as a priori map for subsequent feature weighting processing, guiding the model to focus on the spatial locations with the highest risk of violation.

[0039] By performing distance transformation on the spatial mask of the sterile area and constructing a spatial attention matrix, this invention can accurately reflect the spatial risk distribution of each pixel in surgical operation images. Compared with existing technologies, this method not only improves the accuracy of spatial risk perception but also provides more targeted feature inputs for deep learning models, effectively enhancing the model's sensitivity and detection capability for high-risk violations, thereby reducing the false negative rate and improving the reliability and safety of intelligent monitoring of operating room nursing operations.

[0040] As a further solution, the specific process for constructing the operational topology diagram is as follows:

[0041] First, based on the skeletal key points of medical staff extracted in the previous step, key hand points (such as wrist joints and finger key points) directly related to nursing operations are selected as the starting node group of the graph structure. Simultaneously, for the extracted nursing instrument contour boundaries, their geometric centroid coordinates are calculated as the termination nodes of the graph structure. If there are multiple nursing instruments in the image, the centroid of the contour of each instrument is calculated to form a set of termination nodes.

[0042] Traverse the set of starting nodes and the set of ending nodes to construct node pairs. And calculate the geometric features characterizing their relative positional relationships. Specifically, calculate the starting node. With the termination node Euclidean distance between The calculation formula is: Simultaneously, the relative angles in the computational space are calculated. This angle is defined as the vector connecting the start node and the end node. The angle between the horizontal direction (or the preset operating reference direction) and the horizontal direction is calculated using the following formula: The Euclidean distance value reflects the spatial proximity between the hand and the instrument, while the spatial relative angle reflects the orientation of the operation posture.

[0043] Based on the calculated Euclidean distance values ​​and spatial relative angles, a weighted adjacency matrix reflecting the connection strength between nodes is constructed. Elements in the matrix Indicates the starting node With the termination node The edge weights between them. In this embodiment, the edge weights are calculated using a combination of a Gaussian kernel function and an angle correction factor, as shown in the following formula: ;in, This is a distance scale parameter used to adjust the effect of distance on connection strength; This is an angle correction function used to assign higher connection weights to specific angles according to operational specifications. The closer the distance and the more the angle conforms to the operational logic, the greater the connection weight between nodes, representing a tighter interaction state.

[0044] To enrich the semantic information of graph nodes, visual features are embedded into the topology. Using ROI Align (Region of Interest Alignment) technology, local feature vectors corresponding to the starting node (hand region) and the ending node (device region) are extracted from the feature map extracted from the backbone network, and these feature vectors are assigned to the corresponding nodes as node attributes.

[0045] Finally, the node attribute matrix (containing embedded local feature vectors) and the weighted adjacency matrix are combined. Generate a complete operational topology diagram. ,in For a set of nodes, This is a weighted set of edges. The graph structure intuitively represents the topological relationship of the interaction between medical staff's hands and nursing instruments at the current moment, providing structured data support for subsequent analysis of the compliance evolution of operations in conjunction with the time dimension.

[0046] By constructing an operational topology diagram based on the relative positional relationship between key skeletal points of medical personnel and the contour boundaries of nursing instruments, this invention can accurately characterize the interaction state of hand-instrument and its spatiotemporal evolution characteristics. Compared with existing technologies that rely solely on static images or local feature analysis, this method can comprehensively reflect the spatial relationships and temporal evolution of hand-instrument interactions, improve the model's sensitivity to subtle violations and its recognition accuracy, thereby significantly reducing the risk of missed detections and enhancing the reliability and safety of intelligent monitoring of operating room nursing operations.

[0047] It should also be noted that, in this embodiment, the specific process of generating the spatiotemporal feature map to be detected is as follows:

[0048] Construct a graph convolutional network containing multiple graph convolutional layers. Input the operational topology graph generated in the previous step into this network, where node attributes include local feature vectors and edge attributes include weighted adjacency matrices. During graph convolution operations, each layer updates the feature representation of the current node by aggregating the feature information of neighboring nodes (i.e., hand nodes and device nodes), which can be formally expressed as: ,in It is an adjacency matrix. For degree matrix, This is a trainable weight matrix. After multi-layer graph convolution operations, the feature vectors of global nodes in the output layer are extracted, or average pooling is performed on the features of all nodes to obtain a high-dimensional structural feature vector that reflects the interaction between medical staff and nursing equipment. .

[0049] A pre-trained convolutional neural network (such as ResNet, VGG, or EfficientNet) is selected as the backbone network for feature extraction from consecutive image frames. The training process for this part is divided into two stages. First, in the pre-training stage, the backbone network (such as ResNet, VGG, or EfficientNet) is pre-trained on a large-scale general image dataset (such as ImageNet), allowing the network to learn low-level features such as edge texture, mid-level features such as local structure, and high-level semantic features sequentially from shallow to deep layers, forming initial weights with strong generalization ability. Second, in the fine-tuning stage for this task, the pre-trained weights are transferred to the surgical nursing operation compliance detection task, and the network is fine-tuned end-to-end using labeled consecutive image frame data. Initially, shallow layer weights are typically frozen, and only the parameters of deeper layers and the task output head are updated; after the network initially converges, all or some layers are unfrozen, and the overall network is finely adjusted with a small learning rate. During training, data augmentation techniques such as random cropping, flipping, and brightness perturbation are used to improve robustness, and loss functions such as cross-entropy or Focal Loss are employed to guide parameter optimization. Ultimately, the network simultaneously outputs feature maps from intermediate and deep layers, providing high-resolution detail information and strong semantic global information, respectively, for subsequent compliance assessment. Normalized consecutive image frames are input into the backbone network to extract feature maps from the intermediate and deep layers. To obtain multi-scale features, a Feature Pyramid Network (FPN) structure is used, or upsampling and fusion operations are performed on feature maps from different levels, ultimately outputting multi-channel appearance feature maps containing rich scene semantic information (such as environmental background and object texture). ,in , For the height and width of the feature map, This represents the number of channels.

[0050] Due to the high-dimensional structure feature vector It is a one-dimensional vector, while the appearance feature map Since it is a three-dimensional tensor, dimension matching is required. Specifically, for Perform a spatial reshaping operation, changing its dimensions from... Expand to That is, by copying or broadcasting, the feature vectors are spread across the entire feature map space to obtain the structural feature map. Subsequently, the structural feature map was mapped along the channel dimension. Compared with appearance feature diagram The features are stitched together to generate a fused feature map. This fused feature map contains both visual appearance information and embedded structural semantic information about hand-device interaction.

[0051] Select the fused feature maps corresponding to consecutive time steps (e.g., T consecutive frames). Stacking operations are performed along the time axis to form a four-dimensional spatiotemporal feature tensor. To enhance the expressive power of temporal features, temporal convolutional networks (TCNs) or long short-term memory networks (LSTMs) can be further introduced to temporally encode the stacked tensors, ultimately generating a spatiotemporal feature map with temporal evolution characteristics. This feature map can fully characterize the spatial interaction state and the dynamic evolution process of nursing operations.

[0052] By mapping the operational topology diagram to a feature space and combining it with the time dimension to generate a spatiotemporal feature map to be detected, this invention can simultaneously capture the structural features of hand-instrument interactions and the semantic information of the surgical scene. Compared with existing methods that rely solely on single-frame images or local motion analysis, this method significantly enhances the representation ability of complex operational behaviors, improves the sensitivity and accuracy of detecting potential violations, thereby reducing the risk of missed detections and improving the reliability and safety of intelligent monitoring of operating room nursing operations.

[0053] The third step of the intelligent detection method for compliance images of surgical nursing operations is as follows: call the pre-built compliance benchmark reconstruction logic to reconstruct the spatiotemporal feature map to be detected to obtain the reconstructed spatiotemporal feature map, calculate the pixel-level feature difference between the spatiotemporal feature map to be detected and the reconstructed spatiotemporal feature map, and use the spatial attention matrix to perform weighted mapping on the pixel-level feature difference to generate an initial abnormal saliency map.

[0054] By employing a feature reconstruction paradigm and training the model using compliant samples, it achieves high reconstruction capability for normal nursing operations but generates high reconstruction error for violations and anomalies. This enables accurate detection of unknown abnormal images even in the absence of violation samples. Pixel-level difference calculation and weighted mapping realize the transformation from abstract features to visual saliency maps, intuitively locating the specific location of violations in the image and improving the interpretability of the detection results.

[0055] In this embodiment, the construction and training process of the compliance benchmark reconstruction logic is as follows: Video clips that have been manually reviewed by experts and confirmed to fully comply with nursing operation standards are selected from the historical operating room monitoring video database. These compliant video clips undergo the same preprocessing operations (decoding, normalization, feature extraction, etc.) as the aforementioned detection stage to construct a training dataset containing only positive samples (compliant samples). This dataset does not contain any samples of violations to ensure that the model only learns the distribution of compliance features.

[0056] Construct a spatiotemporal reconstruction network comprising an encoder, a memory module, and a decoder. The encoder employs a multi-layer spatiotemporal convolutional network to extract deep spatiotemporal features from the input data and compress them into low-dimensional latent feature vectors. The memory module involves designing a learnable memory matrix. As a core component, among which For the number of memory slots, The feature dimension is defined as follows. This module stores typical prototypes of compliant operation features. The decoder employs a transposed convolutional network symmetric to the encoder to restore the feature vectors to reconstructed data with the same size as the input.

[0057] During forward propagation, the latent feature vector output by the encoder is used as the query vector input to the memory module. The memory module calculates the similarity (such as cosine similarity or Euclidean distance) between the query vector and the vectors in each memory slot of the memory matrix, retrieving one or more memory slots with the highest similarity as the most similar compliant feature prototypes. Using either hard or soft addressing strategies, the encoder's original output features are replaced or readdressed with the retrieved compliant feature prototypes, which then serve as the input to the decoder. This mechanism forces the decoder to reconstruct using only compliant prototypes from the memory, thereby stripping away potential anomalous components at the feature level.

[0058] Compliance training samples are input into the spatiotemporal reconstruction network for iterative training. The reconstruction loss function is defined as the mean squared error (MSE) or structural similarity index (SSIM) loss between the input data and the reconstructed data. The training objective is to minimize this reconstruction error. The network weight parameters of the encoder and decoder, as well as the memory matrix values ​​in the memory module, are updated synchronously using the backpropagation algorithm. As training progresses, the memory module gradually records typical characteristic patterns of compliant care operations, and the decoder learns how to perfectly reproduce compliant inputs based on these patterns.

[0059] Training stops once the training loss converges and reaches a preset accuracy threshold. The optimal network weight parameters (including encoder and decoder parameters) and the feature matrix stored in the memory module are frozen and saved. This set of parameters and the feature matrix are collectively defined as the "compliance benchmark reconstruction logic," used for subsequent feature reconstruction and anomaly detection of real-time detection data.

[0060] Preset accuracy thresholds refer to a set of target values ​​for evaluation metrics pre-set during model training. When the model's performance on the validation set consistently meets these targets, training is considered sufficiently complete, triggering a stopping condition. Specifically, the following metrics are typically considered: For classification accuracy, the model's overall accuracy in recognizing compliant and non-compliant operations should reach a preset level (e.g., above 90%); for loss, both training and validation losses should decrease to below a certain low threshold, and the gap between them should no longer widen to confirm that the model has not overfitted; for precision and recall, for the critical category of non-compliant operations, precision and recall should simultaneously reach preset lower limits to avoid missed detections due to imbalanced samples; additionally, an early stopping mechanism can be introduced as an auxiliary condition, i.e., if the core metrics on the validation set do not improve within several consecutive training epochs (e.g., 10 to 20 epochs), training is also stopped. The specific values ​​of the above thresholds need to be pre-set based on the tolerance for false positives and false negatives in the actual application scenario, and dynamically monitored during training to ensure that the final saved model weights correspond to the training state with optimal validation performance.

[0061] By constructing a spatiotemporal reconstruction network based on compliant nursing procedures, this invention can establish accurate and reliable compliance benchmark reconstruction logic. Compared with existing technologies that rely solely on traditional supervised learning or static classification methods, this method can achieve more stable and accurate detection of violations in the complex environment of the operating room, providing reliable technical support for the safety management of surgical nursing procedures and reducing potential medical risks.

[0062] As a further approach, the specific process for calculating pixel-level feature differences is as follows:

[0063] The spatiotemporal feature map to be detected is denoted as... The reconstructed spatiotemporal feature map is denoted as Both have the same dimension. (in For the length of time, and For space dimensions, (Number of feature channels).

[0064] Iterate through each spatial location (i.e., each pixel location) in the spatiotemporal feature map and obtain the feature vector corresponding to that location. and The Euclidean distance between these two feature vectors is calculated as a measure of numerical deviation at that location. The formula is as follows: ;in, This represents the spatial coordinate index. Arranging the Euclidean distance values ​​calculated for all locations into a matrix yields a pixel-level spatial difference map. This map reflects the spatial distribution deviation of the features in terms of numerical amplitude.

[0065] Similarly, for each pair of feature vectors at a spatial location, their cosine distance in the high-dimensional feature space is calculated to measure the semantic bias along the feature direction. The calculation formula is as follows: ;in, Represents the vector dot product. and These represent the magnitudes of the vectors. Cosine distance effectively captures changes in the semantic direction of features and is robust to linear numerical changes caused by factors such as illumination variations. Arranging the cosine distance values ​​at all positions into a matrix yields a semantic-level difference map. .

[0066] Since Euclidean distance and cosine distance have different numerical dimensions and physical meanings, they need to be normalized to eliminate the dimensional differences. A min-max normalization method is used to normalize the pixel-level spatial difference maps. semantic level difference graph The numerical values ​​are linearly mapped to the interval [0,1].

[0067] Finally, based on the preset weighting coefficients and (and satisfy) The normalized difference maps are then weighted and summed to obtain the final pixel-level feature differences. : ; pixel-level feature differences It comprehensively characterizes the degree of deviation between the features to be detected and the compliant reconstruction features in both spatial numerical and semantic directions.

[0068] By calculating the pixel-level feature differences between the spatiotemporal feature map to be detected and the reconstructed spatiotemporal feature map, this invention can accurately identify abnormal behavior regions during surgical procedures. The advantages of this method are twofold: firstly, it simultaneously considers spatial feature shifts and semantic feature changes, making anomaly detection more comprehensive and accurate; secondly, by quantifying subtle feature changes during surgical procedures, it can significantly improve the model's sensitivity to potential violations and its accuracy in identification, reducing the risk of missed detections. This provides reliable technical support for intelligent monitoring of operating room nursing procedures and enhances surgical safety and nursing quality.

[0069] The fourth step in the intelligent image detection method for compliance of surgical nursing procedures involves: performing local connected component processing on the initial anomaly saliency map to filter out discrete noise interference, outputting a spatially sensitive anomaly saliency map; performing spatial weighted aggregation calculation on the spatially sensitive anomaly saliency map to obtain a comprehensive violation confidence score; and outputting the image detection result based on the relationship between the comprehensive violation confidence score and a preset threshold. Local connected component processing effectively eliminates isolated pixel-level noise interference in the image, significantly reducing the false detection rate; spatial weighted aggregation calculation transforms scattered abnormal pixels into an overall confidence index, enhancing the robustness of the detection model to real violation targets and ensuring that the output detection results are more accurate in visual localization and more reliable in judgment logic.

[0070] It should be noted that the specific implementation methods for the two parts of the fourth step, namely "performing local connected component processing on the initial anomaly saliency map to filter out discrete noise interference" and "performing spatial weighted aggregation calculation on the spatially sensitive anomaly saliency map to obtain the comprehensive violation confidence level," are as follows:

[0071] First, set an abnormal response threshold. (For example ), for the initial anomaly saliency map Binarization is performed. Abnormal response values ​​greater than a threshold are considered... The pixels are set to 1 (foreground) and the rest are set to 0 (background) to obtain a binarized image. Next, a connected component analysis algorithm (such as the Two-Pass algorithm or Seed-Filling algorithm) is used to traverse the binarized image and extract all interconnected foreground pixel regions, which are then marked as independent sets of connected components. .

[0072] For each extracted connected region The geometric features are calculated, specifically including: counting the total number of pixels contained within the connected region; and calculating the width of the minimum bounding rectangle of the connected region. With height The ratio, i.e. .

[0073] Set area threshold (Used to remove minute noise) and aspect ratio range (Used to remove unreasonable linear artifacts). If a connected region satisfies or If the region is identified as discrete noise or invalid interference, it will be removed.

[0074] Morphological processing is performed on the preserved connected regions. Specifically, morphological closing operations (dilation followed by erosion) are applied to fill any voids within the regions and smooth edge contours, eliminating jagged interference. The processed image is then used as the spatially sensitive anomaly saliency map. The map only retains anomalous regions with significant geometric features and conforming to the violation pattern.

[0075] Traversing the spatially sensitive anomaly saliency graph The abnormal response values ​​of all pixels in the image are summed to obtain the global abnormal response sum, which reflects the overall violation intensity. : This indicator reflects the overall reach and intensity of the violation in the image.

[0076] The spatial attention matrix generated using the aforementioned steps As a risk weighting coefficient, the weighted anomaly response value is obtained by element-wise multiplying the spatial sensitivity anomaly saliency map and the spatial attention matrix and summing the results. : This indicator highlights abnormal responses in high-risk areas (such as the boundaries of sterile areas) and suppresses the impact of false alarms in non-critical areas.

[0077] To balance the overall severity of the violation with its risk sensitivity, a linear combination strategy is used to calculate the final comprehensive violation confidence level. : ;in, and The preset weighting coefficients are used, and they satisfy the following conditions: ; This represents the normalization function, used to map the response values ​​to the [0,1] interval. The calculated... This serves as the quantitative basis for ultimately determining whether the image frame violates regulations.

[0078] By performing local connected component processing on the initial anomaly saliency map, discrete noise interference is effectively filtered out, significantly improving the accuracy of anomaly detection. Simultaneously, through the linear combination of the global anomaly response and the weighted anomaly response value, this invention achieves accurate calculation of the comprehensive violation confidence, thereby improving the accuracy of violation judgment while maintaining high precision and low false alarms. Compared to traditional methods, this invention, through the combination of spatial weighted aggregation and connected component processing, not only improves the detection sensitivity for a small number of violations but also significantly reduces false alarms caused by noise and irrelevant regions, greatly enhancing the robustness and stability of the detection results.

[0079] As a further step, in surgical nursing procedures, timely and accurate detection of violations is crucial not only for surgical safety but also directly impacts postoperative recovery and the efficiency of medical management. While relying solely on global violation confidence assessment can identify potential risks, it cannot pinpoint the specific location and type of violation, nor provide traceable evidence for administrators. Without spatial localization, type identification, and keyframe extraction, violations may be missed or difficult to track, thus reducing the practical value of the monitoring system. Therefore, it is necessary to combine violation assessment with spatial localization, type identification, and alarm information generation to achieve precise labeling and immediate feedback on violations. Specifically, after outputting the image detection results based on the relationship between the comprehensive violation confidence and a preset threshold, the following steps are also included:

[0080] When the calculated comprehensive violation confidence level When the threshold for violation determination is exceeded (e.g., 0.85), the system determines that the current nursing operation is in violation. At this point, the spatially sensitive anomaly saliency map is traversed. For each connected region retained in the feature map, calculate the sum of the abnormal response values ​​of all pixels within that region, and select the region with the largest sum as the core violation source. Obtain the coordinates of the minimum bounding rectangle of this core violation source in the feature map coordinate system. Based on the downsampling factor during the feature extraction process (e.g., a downsampling factor of 1.5%). The feature map coordinates are mapped back to the original video frame coordinate system proportionally, where the original coordinates (set as...) ): The pixel coordinates in the original video frame obtained after mapping, i.e., the actual position of the target region or key point in the original image, in pixels. Feature map coordinates (set as...) ): The coordinates of the target region or key point on the feature map, output by the network detection head in the feature map space. Downsampling factor (set to ): The mapping factor is the spatial compression ratio experienced by the backbone network from the input image to the current feature map. For example, a downsampling factor of 32 means that the width and height of the feature map are each 1 / 32 of the original image, that is, one pixel on the feature map corresponds to a 32×32 region in the original image. This value is determined by the cumulative product of the pooling layer and the convolution stride in the network structure. The basic form of the mapping formula is: , Based on the mapped coordinates, a red-highlighted boundary box for the violation warning is drawn on the original video frame to achieve visual localization of the violation area.

[0081] To clarify the specific nature of the violation, a multi-dimensional feature fusion mapping logic is used for determination. First, the feature offset between the feature vector of the operation topology graph at the time of the violation and the standard compliant feature prototype in the memory module is calculated. This offset reflects the abnormal pattern of the interaction structure. Subsequently, combining the spatial distribution characteristics of the abnormal regions in the spatially sensitive anomaly saliency map, type identification is performed using a pre-set classification decision tree or a trained multilayer perceptron (MLP) classifier. The specific determination logic is as follows:

[0082] Sterile area crossing violation: If the spatial location of the abnormal area crosses the boundary of the sterile area spatial mask, and the topology map features show that the hand key points move from the non-sterile area to the sterile area, it is identified as a sterile area crossing violation.

[0083] Hand operation area violation: If the abnormal area is located entirely within the sterile area or outside the operation restricted area, and the topological feature offset shows abnormal hand posture (such as unnecessary contact), it is identified as a hand operation area violation.

[0084] Violation of instrument delivery posture: If the abnormal area is concentrated on the instrument delivery path, and the connection weight distribution of the hand-instrument interaction node in the topology diagram is abnormal (such as excessive distance or excessive angle deviation), it is identified as a violation of instrument delivery posture.

[0085] The system automatically extracts video clips of a preset duration (e.g., 2 seconds before and 2 seconds after) before and after the violation determination time as keyframe evidence. It encapsulates the marked image of the violation warning bounding box, the identified violation type label, the comprehensive violation confidence score, and the timestamp information to generate a standardized alarm information data packet. This alarm information is then pushed in real-time to the nurse station management terminal or mobile nursing devices via the hospital's intranet or wireless communication module, using message queues (such as MQTT) or HTTP protocols, triggering audible and visual alarms and assisting management personnel in immediate intervention.

[0086] By outputting image detection results based on the relationship between comprehensive violation confidence and a preset threshold, this invention can not only identify violations during surgery, but also provide accurate evidence of violations and detailed information on violation types, thereby effectively improving the monitoring accuracy and intelligence level of surgical operations.

[0087] like Figure 2 The diagram shown is an overall detection architecture diagram of the intelligent detection method for surgical nursing operation compliance images provided in this application embodiment, based on... Figure 2It is important to understand that this invention achieves high-precision, interpretable intelligent detection of surgical nursing operation compliance through a multi-stage collaborative processing flow. Each step cooperates with and progresses layer by layer, forming a complete and effective violation detection mechanism. Specifically, it constructs a complete intelligent closed loop of "perception-understanding-judgment-decision" through the organic connection and data interaction of four steps: low-level feature extraction, mid-level spatiotemporal modeling, high-level reconstruction anomaly detection, and final post-decision processing. Each step is interdependent and mutually supportive, jointly solving the technical bottlenecks of low detection accuracy and poor anti-interference ability in existing technologies under imbalanced sample conditions, thus achieving efficient, accurate, and intelligent monitoring of surgical nursing operation compliance.

[0088] like Figure 3 The diagram shown is a structural schematic of the intelligent image detection system for surgical nursing operation compliance provided in this application embodiment. (Refer to...) Figure 3 The system includes: a nursing operation image acquisition module, a spatiotemporal feature map generation module, a compliance logic feature reconstruction module, and an image detection result output module.

[0089] Specifically, the nursing operation image acquisition module is used to acquire real-time monitoring video streams of operating room nursing scenarios, decode and normalize the real-time monitoring video streams to obtain continuous image frames, and extract the spatial mask of the sterile area corresponding to the image to be detected, the outline boundary of nursing instruments, and the skeletal key points of medical staff in the continuous image frames.

[0090] The spatiotemporal feature map generation module is used to construct a spatial attention matrix reflecting the spatial location risk distribution based on the spatial mask of the sterile area, and to construct an operation topology map representing the hand-instrument interaction state based on the relative positional relationship between the skeletal key points of medical staff and the contour boundary of nursing instruments. The operation topology map is mapped to the feature space for encoding, and combined with the time dimension information to generate the spatiotemporal feature map to be detected.

[0091] The compliance logic feature reconstruction module is used to call the pre-built compliance benchmark reconstruction logic, reconstruct the spatiotemporal feature map to be detected to obtain the reconstructed spatiotemporal feature map, calculate the pixel-level feature difference between the spatiotemporal feature map to be detected and the reconstructed spatiotemporal feature map, and use the spatial attention matrix to perform weighted mapping on the pixel-level feature difference to generate an initial anomaly saliency map.

[0092] The image detection result output module is used to perform local connected component processing on the initial anomaly saliency map to filter out discrete noise interference, output a spatially sensitive anomaly saliency map, perform spatial weighted aggregation calculation on the spatially sensitive anomaly saliency map to obtain the comprehensive violation confidence, and output the image detection result according to the relationship between the comprehensive violation confidence and the preset threshold.

[0093] It should be noted that the system described in this invention is not a passive management platform used only for image data storage or manual quality control submission in the traditional sense, but rather an active intelligent detection system with in-depth analysis capabilities and an objective evaluation mechanism. Its analysis and objective evaluation functions are specifically reflected in the following two aspects:

[0094] At the intelligent analysis level, the system possesses proactive parsing and semantic understanding capabilities. This system utilizes computer vision and deep learning algorithms to automate the entire process of real-time operating room video stream monitoring. First, the system does not simply record video; instead, it proactively extracts spatial masks of sterile areas, the outlines of nursing instruments, and key skeletal points of medical personnel from continuous image frames, achieving precise perception of the three-dimensional elements of "environment, objects, and people." Second, the system constructs a spatial attention matrix and an operational topology diagram of hand-instrument interaction, simulating human visual attention mechanisms to capture dynamic interactive behaviors in high-risk areas. This process transforms unstructured video images into computable structured data, completing a deep intelligent analysis of nursing procedures.

[0095] At the objective evaluation level, the system establishes quantitative benchmarks and an automated judgment mechanism. It abandons the traditional review model that relies on subjective human judgment or empirical rules, and constructs an objective evaluation standard based on "compliance benchmark reconstruction logic." By calculating the pixel-level feature differences between the spatiotemporal feature map to be detected and the reconstructed spatiotemporal feature map, the system transforms the abstract compliance judgment into a specific "comprehensive violation confidence score" value (between 0 and 1), achieving precise quantification of the evaluation results. This evaluation result is automatically generated by the algorithm model based on the mathematical distance in the feature space, unaffected by subjective human factors. Simultaneously, the system can automatically output the spatial coordinates of the violation area, violation type labels (such as violations of sterile area crossing, violations of instrument delivery posture, etc.), and confidence scores, generating a traceable chain of objective evidence.

[0096] This system represents a leap from "passive image acquisition" to "active intelligent analysis," and from "human subjective review" to "objective quantitative evaluation," enabling it to provide nursing management terminals with real-time, accurate, and objectively based detection conclusions.

[0097] In summary, this invention addresses the practical challenges of scarce violation samples and highly imbalanced data distribution in operating room nursing scenarios by innovatively constructing an intelligent detection framework based on reconstruction logic and spatiotemporal feature analysis. This method extracts multimodal key features and constructs a spatial attention matrix and operational topology diagram, achieving deep characterization and risk focus of the spatiotemporal features of nursing operations. Utilizing compliance benchmark reconstruction logic, it effectively overcomes the bottleneck of traditional supervised learning's reliance on massive violation samples, significantly improving sensitivity and generalization ability for rare, high-risk violations. Simultaneously, combining local connected domain processing and spatial weighted aggregation calculation ensures the robustness and accuracy of the detection results. This invention effectively solves the problem of missed detections caused by decision boundary offsets in existing technologies, achieving accurate and intelligent monitoring of surgical nursing operation compliance. It has significant application value and practical significance for improving operating room safety management and reducing the risk of medical accidents.

[0098] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the above functions can be divided into different functional modules to complete all or part of the functions described above.

[0099] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0100] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units, located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0101] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0102] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the solution, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0103] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for intelligent image detection of compliance in surgical nursing procedures, characterized in that, Includes the following steps: The real-time monitoring video stream of the operating room nursing scene is acquired, and the real-time monitoring video stream is decoded and normalized to obtain continuous image frames. The spatial mask of the sterile area corresponding to the image to be detected, the outline boundary of the nursing instrument, and the skeletal key points of the medical staff are extracted from the continuous image frames respectively. A spatial attention matrix reflecting the spatial location risk distribution is constructed based on the spatial mask of the sterile area. An operation topology diagram representing the hand-instrument interaction state is constructed based on the relative positional relationship between the skeletal key points of medical staff and the contour boundary of nursing instruments. The operation topology diagram is mapped to the feature space for encoding and combined with the time dimension information to generate the spatiotemporal feature map to be detected. The pre-built compliance benchmark reconstruction logic is invoked to reconstruct the spatiotemporal feature map to be detected, obtain the reconstructed spatiotemporal feature map, calculate the pixel-level feature difference between the spatiotemporal feature map to be detected and the reconstructed spatiotemporal feature map, and use the spatial attention matrix to perform weighted mapping on the pixel-level feature difference to generate an initial anomaly saliency map. The initial anomaly saliency map is processed by local connected component processing to filter out discrete noise interference, and a spatially sensitive anomaly saliency map is output. The spatially sensitive anomaly saliency map is then spatially weighted and aggregated to obtain the comprehensive violation confidence. The image detection result is output based on the relationship between the comprehensive violation confidence and a preset threshold.

2. The intelligent image detection method for compliance of surgical nursing operations as described in claim 1, characterized in that, The construction of a spatial attention matrix reflecting the spatial location risk distribution based on the spatial mask of the sterile area specifically includes: The spatial mask of the sterile area is subjected to distance transformation processing, and the shortest distance value from each pixel in the image to be detected to the boundary of the sterile area is calculated. Based on the shortest distance value, the risk weight of each pixel is calculated using a preset Gaussian decay function. The risk weight increases as the distance from the pixel to the boundary of the sterile area decreases. The risk weight of each pixel is mapped to a matrix with the same size as the image to obtain the spatial attention matrix.

3. The intelligent image detection method for compliance of surgical nursing operations as described in claim 1, characterized in that, The construction of an operational topology diagram representing the hand-instrument interaction state based on the relative positional relationship between the key skeletal points of medical personnel and the contour boundaries of nursing instruments specifically includes: Using the key points of the hands in the skeletal key points of medical staff as the starting node group and the centroid of the contour boundary of nursing instruments as the ending node, the Euclidean distance and spatial relative angle between each starting node and ending node are calculated respectively. A weighted adjacency matrix reflecting the connection strength between nodes is constructed based on Euclidean distance and spatial relative angle. The local feature vectors of the starting and ending nodes are embedded into the corresponding nodes to generate an operational topology diagram to characterize the spatiotemporal evolution of the interaction between medical staff and nursing equipment.

4. The intelligent image detection method for compliance of surgical nursing operations as described in claim 1, characterized in that, The step of mapping the operation topology diagram to the feature space for encoding, and combining it with time dimension information to generate a spatiotemporal feature map to be detected, specifically includes: Graph convolutional networks are used to perform multi-layer graph convolution operations on the topological graph to extract high-dimensional structural feature vectors that reflect interaction relationships. A convolutional neural network is used to extract multi-scale features from consecutive image frames to obtain multi-channel appearance feature maps containing scene semantic information. The high-dimensional structural feature vector is spatially reshaped to match the size of the multi-channel appearance feature map, and then spliced ​​and fused with the multi-channel appearance feature map in the channel dimension and stacked in the time axis direction to generate a spatiotemporal feature map to be detected with temporal evolution characteristics.

5. The intelligent image detection method for compliance of surgical nursing operations as described in claim 1, characterized in that, The construction process of the compliance benchmark reconstruction logic includes: Historical video data containing only compliant nursing operations is obtained as training samples, and a spatiotemporal reconstruction network containing an encoder, a memory module, and a decoder is constructed. The memory module replaces the encoded features with the most similar compliant feature prototype in the memory bank through nearest neighbor retrieval and then feeds them into the decoder. The training samples are input into the spatiotemporal reconstruction network for training. The encoder extracts compliant spatiotemporal features and stores them in the memory module. The decoder reconstructs the compliant spatiotemporal features, with the optimization objective being to minimize the reconstruction error. After training, the network parameters and memory module features are stored to obtain the compliance benchmark reconstruction logic.

6. The intelligent image detection method for compliance of surgical nursing operations as described in claim 1, characterized in that, The calculation of the pixel-level feature difference between the spatiotemporal feature map to be detected and the reconstructed spatiotemporal feature map specifically includes: Calculate the Euclidean distance between the feature vectors of the spatiotemporal feature map to be detected and the reconstructed spatiotemporal feature map at corresponding positions in the spatial dimension to obtain a pixel-level spatial difference map; The cosine distance between the feature vectors at corresponding positions in the high-dimensional feature space of the spatiotemporal feature map to be detected and the reconstructed spatiotemporal feature map is calculated to obtain the semantic level difference map. The pixel-level spatial difference map and the semantic-level difference map are subjected to min-max normalization and then weighted and summed according to preset weights to obtain the pixel-level feature difference.

7. The intelligent image detection method for compliance of surgical nursing operations as described in claim 1, characterized in that, The process of performing local connected component processing on the initial anomaly saliency map to filter out discrete noise interference specifically includes: The initial anomaly saliency map is binarized to obtain a binarized image, and all connected regions in the binarized image are extracted. Calculate the area of ​​each connected region and the aspect ratio of its circumscribed rectangle. Remove connected regions whose area is less than a preset area threshold or whose aspect ratio of the circumscribed rectangle exceeds a preset ratio range. Retain and perform morphological smoothing on the remaining connected regions to obtain a spatially sensitive anomaly saliency map.

8. The intelligent image detection method for compliance of surgical nursing operations as described in claim 1, characterized in that, The process of spatially weighted aggregation of spatially sensitive anomaly saliency maps to obtain the comprehensive violation confidence score specifically includes: The global anomaly response sum is obtained by summing the anomaly response values ​​of all pixels in the spatially sensitive anomaly saliency map. The weighted anomaly response value is obtained by using the risk weight of each pixel in the spatial attention matrix to perform a weighted summation operation on the anomaly response value in the spatially sensitive anomaly saliency map. The overall violation confidence level is obtained based on the linear combination of the global anomaly response and the weighted anomaly response value.

9. The intelligent image detection method for compliance of surgical nursing operations as described in claim 1, characterized in that, After outputting the image detection result based on the relationship between the comprehensive violation confidence level and the preset threshold, the method further includes: If the current nursing operation is determined to be in violation, the coordinates of the bounding rectangle of the connected region with the largest abnormal response value in the spatially sensitive abnormality saliency map are mapped proportionally to the original video frame coordinate system, and a violation warning bounding box is drawn in the original video frame. Extract the graph feature offset of the operation topology graph at the time of violation, and combine it with the spatial location distribution of abnormal areas in the spatially sensitive anomaly salience graph. Identify the specific violation type through a preset classification mapping logic. The violation type includes at least aseptic area crossing violation, instrument transfer posture violation, and hand operation area violation. The system automatically captures key frames before and after a violation occurs, generates alarm information including screenshots of violation evidence, violation type labels, and confidence scores, and pushes them synchronously to the nursing management terminal.

10. A surgical nursing operation compliance image intelligent detection system, employing the surgical nursing operation compliance image intelligent detection method as described in any one of claims 1-9, characterized in that, include: The module includes a nursing operation image acquisition module, a spatiotemporal feature map generation module, a compliance logic feature reconstruction module, and an image detection result output module. The nursing operation image acquisition module is used to acquire real-time monitoring video streams of operating room nursing scenarios, decode and normalize the real-time monitoring video streams to obtain continuous image frames, and extract the spatial mask of the sterile area corresponding to the image to be detected, the outline boundary of nursing instruments, and the skeletal key points of medical staff in the continuous image frames respectively. The spatiotemporal feature map generation module is used to construct a spatial attention matrix reflecting the spatial location risk distribution based on the spatial mask of the sterile area, and to construct an operation topology map representing the hand-instrument interaction state based on the relative positional relationship between the skeletal key points of medical staff and the contour boundary of nursing instruments. The operation topology map is mapped to the feature space for encoding, and combined with the time dimension information to generate the spatiotemporal feature map to be detected. The compliance logic feature reconstruction module is used to call the pre-built compliance benchmark reconstruction logic, reconstruct the spatiotemporal feature map to be detected to obtain the reconstructed spatiotemporal feature map, calculate the pixel-level feature difference between the spatiotemporal feature map to be detected and the reconstructed spatiotemporal feature map, and use the spatial attention matrix to perform weighted mapping on the pixel-level feature difference to generate an initial anomaly saliency map. The image detection result output module is used to perform local connected component processing on the initial anomaly saliency map to filter out discrete noise interference, output a spatially sensitive anomaly saliency map, perform spatial weighted aggregation calculation on the spatially sensitive anomaly saliency map to obtain a comprehensive violation confidence level, and output the image detection result according to the relationship between the comprehensive violation confidence level and a preset threshold.

Images