A Deep Learning-Based Real-Time Detection Method for Abnormal Behavior in Medical Waste Temporary Storage Stations

Abstract

This invention discloses a real-time detection method for abnormal behavior in medical waste storage stations based on deep learning, belonging to the field of anomaly detection technology. This invention uses a MobileNetV3 network structure to replace the traditional YOLO V7 Backbone module, and uses an improved NAS-FPN network and CoAtNet network to replace the traditional YOLO V7 Neck module, constructing an abnormal behavior detection model based on the improved YOLO V7. The detection model is trained using an abnormal behavior dataset to obtain a deep learning model. Preprocessed images to be detected are input into the deep learning model for real-time abnormal behavior detection, and real-time warnings are issued for abnormal behavior. The improved YOLO V7 detection model in this invention is lightweight, has a fast detection speed and high detection accuracy for abnormal behavior in medical waste storage stations, and can effectively prevent the inability to identify responsibility for damage, contamination, or loss of medical waste caused by abnormal behavior.

Description

Technical Field

[0001] This invention belongs to the field of abnormal behavior detection technology, and in particular relates to a real-time detection method for abnormal behavior in medical waste temporary storage stations based on deep learning. Background Technology

[0002] When medical waste is temporarily stored in medical institution storage stations, it is difficult to monitor the compliance of staff operations in real time, posing significant safety hazards. Furthermore, the illegal sale of medical waste occurs frequently, and medical waste is prone to loss during temporary storage in medical institutions. Currently, some medical institution medical waste storage stations lack video surveillance, and even those with video surveillance often require manual review to investigate serious anomalies. If violations occur while medical waste is stored at these stations, it is difficult to respond in real time.

[0003] The Yolo series of algorithms is widely used in the field of object detection. The Yolo V7 algorithm surpasses all other Yolo series algorithms in terms of detection accuracy and generalization performance. However, the traditional Yolo V7 model is complex and consumes a lot of memory, making it difficult to apply to applications with high real-time requirements. For example, applying the traditional Yolo V7 object detection algorithm to detect abnormal behavior in video images at a medical waste storage station will result in problems such as untimely detection and low detection accuracy. Summary of the Invention

[0004] 1. The technical problem that the invention aims to solve

[0005] The purpose of this invention is to provide a real-time detection method for abnormal behavior in medical waste storage stations based on deep learning. This invention uses video images captured by cameras in medical waste storage stations and an improved YOLO V7 target detection algorithm to achieve real-time detection of abnormal behavior in medical waste storage stations. This invention improves the Backbone and Neck networks of YOLO V7, making the YOLO V7 algorithm model lightweight, with high detection accuracy and fast detection speed.

[0006] 2. Technical Solution

[0007] To achieve the above objectives, the technical solution provided by the present invention is as follows:

[0008] The present invention provides a real-time detection method for abnormal behavior of a medical waste temporary storage station based on deep learning, comprising the following steps:

[0009] S1: Collect images of abnormal behavior at medical waste temporary storage stations and establish an abnormal behavior detection dataset;

[0010] S2: Use the MobileNetV3 network structure to replace the traditional YOLO V7 Backbone module, and use the improved NAS-FPN network and CoAtNet network to replace the traditional YOLO V7 Neck module to build a rapid medical waste abnormal behavior detection model based on the improved YOLO V7;

[0011] S3: Train the detection model using the abnormal behavior dataset to obtain a deep learning model for real-time detection of abnormal behavior;

[0012] S4: Based on the video images collected in real time from the medical waste temporary storage station, preprocess them to unify the image size;

[0013] S5: Input the preprocessed image to be detected into the deep learning model for real-time detection of abnormal behavior and provide real-time warnings for abnormal behavior.

[0014] Furthermore, the replaced Backbone module includes an input layer, a convolutional layer, and eight Bottleneck units. The number of Bottleneck units in each Bottleneck unit is 1, 2, 3, 4, 3, 3, 2, 2 in sequence. The features extracted from three Bottleneck units are selected as the output features of the Backbone.

[0015] Furthermore, the features extracted from the 4th, 6th, and 8th Bottleneck units are selected as the output features of the Backbone.

[0016] Furthermore, the Bottleneck module comprises a 1×1 convolutional layer, a BN batch normalized layer, an activation function layer, a DConv depthwise separable convolutional layer, a BN batch normalized layer, a ReLU activation function, and a 1×1 convolutional layer connected in sequence.

[0017] Furthermore, an SE channel attention mechanism network is added between the ReLU activation function and the 1×1 convolution in the 3rd, 4th, 7th, and 8th Bottleneck units;

[0018] The activation functions used in the activation function layers of different Bottleneck units are different. The h-swish activation function is used in the 5th, 6th, 7th, and 8th Bottleneck units, while the ReLU6 activation function is used in the remaining Bottleneck units.

[0019] Furthermore, the SE channel attention mechanism network consists of a globally average pooling layer (GlobalPool), a fully connected layer (FC), an activation function (ReLU), another fully connected layer (FC), and an activation function (Sigmoid) connected in sequence.

[0020] Furthermore, the replaced Neck module uses an improved NAS-FPN network to fuse feature images of different sizes; it uses a CoAtNet network to further extract features from the image fused by the improved NAS-FPN network; and it uses the feature image further extracted by the CoAtNet network as the output feature layer and inputs it into the Head module of YOLO V7.

[0021] Furthermore, the replaced Neck module is convolved with the output feature images of the three Bottleneck units through 1×1 convolutions.

[0022] Furthermore, the NAS-FPN network is improved by changing the original five-layer feature fusion network structure to a three-layer feature fusion network structure. The process of fusing feature images of different sizes is as follows: Input feature images P2 and P3 are processed by the global average pooling layer GP to obtain feature map P4. Feature map P4 is processed by the CBR module to obtain feature map P5. Feature map P5 is added to P1 to obtain output feature map C1. Output feature map C1 is processed by the CBR module to obtain feature map P6. Feature map P6 is added to P5 to obtain output feature map C2. Output feature map C2 is processed by the CBR module to obtain feature map P7. Feature map P7 is processed by the global average pooling layer GP to obtain feature map P8. Feature map P8 is added to P3 to obtain output feature map C3. C1, C2, and C3 are the three feature images output by the replaced Neck module.

[0023] Furthermore, the CBR module includes a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence.

[0024] 3. Beneficial effects

[0025] Compared with existing known technologies, the technical solution provided by this invention has the following significant advantages:

[0026] (1) The abnormal behavior real-time detection method of the present invention uses video images collected by cameras in medical waste temporary storage stations and an improved YOLO V7 target detection algorithm to realize real-time detection of abnormal behavior in medical waste temporary storage stations. It can realize intelligent identification of abnormal situations such as staff smoking, staff not wearing gloves, temporary storage boxes not being covered, and strangers entering, and form immediate early warning event information, forming a closed-loop management that can be traced back throughout the process. It can effectively prevent the problem of not being able to identify responsibility for damage, pollution or loss of medical waste caused by abnormal behavior in medical waste temporary storage stations, and has high application value.

[0027] (2) To ensure the real-time performance of the detection algorithm and to ensure timely response to abnormal situations, this invention improves the traditional YOLO V7 target detection algorithm. Compared to heavyweight networks, MobileNetV3 is characterized by fewer parameters, lower computational cost, and shorter inference time. It is more suitable for scenarios with limited storage space and power consumption, such as edge computing devices like mobile embedded devices. MobileNetV3 inherits some practical achievements from V1 and V2 and introduces the SE channel attention mechanism, which gives MobileNetV3 good detection performance. NAS-FPN uses NAS technology to optimize the architecture of FPN. In target detection, features at different scales have different performances in modeling semantic and detail information, so fusing features at multiple scales is crucial to improving detection performance. FPN proposes a top-down fusion strategy, while PANet proposes a bidirectional fusion strategy that is both top-down and bottom-up. NAS-FPN automatically searches for fusion strategies to obtain a fusion strategy superior to FPN and PANet. The structure of the CoAtNet network allows for the simultaneous processing of feature information at different scales, thereby better capturing details in images or text. This makes CoAtNet perform exceptionally well in tasks such as image classification, object detection, and semantic segmentation. CoAtNet can model the global relationships within the input data, thus better understanding the relationships within the data. This enables CoAtNet to provide predictions with strong interpretability and reasoning capabilities. The improved YOLOv7 detection model is lightweight and exhibits fast detection speed and high detection accuracy for abnormal behavior detection at medical waste storage stations. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the real-time detection process for abnormal behavior in this invention;

[0029] Figure 2 This is a block diagram of the improved YOLO V7 network structure in this invention;

[0030] Figure 3 This is a block diagram of the Bottleneck network structure based on MobileNetV3 in this invention;

[0031] Figure 4 Here is a block diagram of the SE channel attention mechanism network structure;

[0032] Figure 5 Here is a diagram of the CoAtNet network structure;

[0033] Figure 6 To improve the NAS-FPN network architecture diagram. Detailed Implementation

[0034] To further understand the content of this invention, a detailed description of the invention will be provided in conjunction with the accompanying drawings and embodiments.

[0035] Example 1

[0036] Referring to the accompanying drawings, this embodiment of a method for real-time detection of abnormal behavior in a medical waste temporary storage station based on deep learning includes the following steps:

[0037] S1: Collect images of abnormal behavior at the medical waste storage station and establish an abnormal behavior detection dataset. The abnormal behavior images to be collected include: staff not wearing masks, staff not wearing gloves, unauthorized entry by strangers, uncovered medical waste storage bins, presence of smoke, presence of flames, and staff smoking. The acquired abnormal behavior images will be filtered, and the abnormal behaviors in the images will be labeled. The labeled images will be divided into a network model training dataset and a network model testing dataset.

[0038] S2: Construct a rapid medical waste abnormal behavior detection model based on improved YOLO V7, specifically:

[0039] like Figure 2 As shown, a Bottleneck unit based on MobileNetV3 is constructed to replace the traditional YOLO V7 Backbone network structure to achieve feature extraction. The replaced YOLO V7 Backbone network structure includes an input layer, a 3×3 convolutional layer, and 8 Bottleneck units. The number of modules in each Bottleneck unit is 1, 2, 3, 4, 3, 3, 2, 2, respectively.

[0040] Combination Figure 3 The Bottleneck module processes the graph in the following order: First, the input feature map goes through a 1×1 convolutional layer to a BN batch normalization layer, then to an activation function layer, then to a DConv depthwise separable convolutional layer, then through a BN batch normalization layer, then to a ReLU activation function, and finally to a 1×1 convolutional layer, and finally outputs the feature map.

[0041] Combination Figure 2 First, convolution (Conv3×3) is used to compress the input image of size 640×640×3 into an image of size 320×320×64;

[0042] An SE channel attention mechanism is added between the ReLU activation function and the 1×1 convolution in the 3rd, 4th, 7th, and 8th Bottleneck units. The network flowchart for the SE channel attention mechanism is shown below. Figure 4As shown: Global Pool is a global average pooling layer, FC is a fully connected layer, and ReLU and Sigmoid are activation functions. C is the number of channels, and r is... Figure 4 The dimensionality reduction hyperparameter of the first fully connected layer in the middle.

[0043] The activation function used in the 5th, 6th, 7th, and 8th Bottleneck units is the h-swish activation function, while the ReLU6 activation function is used in the remaining Bottleneck units.

[0044] The feature layers extracted from three Bottleneck units are selected as the output feature layers of the backbone, such as... Figure 2 The dimensions shown are (80×80×512), (40×40×1024), and (20×20×1024).

[0045] Replacing the traditional YOLO V7 Neck with improved NAS-FPN (Neural Architecture Search Feature Pyramid Network) and CoAtNet (Coordination-Attention Network) networks specifically includes the following:

[0046] like Figure 2 , Figure 5 , Figure 6 As shown, Figure 2 In the Neck network structure shown, Conv is a 1×1 convolution, CBR is a convolution, batch normalization, and activation function layer, and RConv is a RepConv convolution structure. Figure 6 P1, P2, and P3 in the diagram represent input feature images of different sizes, corresponding to... Figure 2 The middle Neck region consists of three input feature images connected by a 1×1 convolution. GP is global average pooling, P4 is the feature map after global average pooling of P2 and P3, CBR is a convolution, batch normalization, and activation function layer, Sum is the sum of different feature images, and Loss is the loss function. C1, C2, and C3 are the output feature maps, corresponding to... Figure 2 The three feature images output from the Neck section. (By...) Figure 6 It can be seen that the improved NAS-FPN network structure can perform feature fusion for features of different sizes.

[0047] Figure 5In CoAtNet, Conv1×1 and Conv3×3 are convolutions with kernels of 1×1 and 3×3, respectively. DConv3×3 is a depthwise separable convolution with a kernel size of 3×3. FFN refers to a feedforward neural network. In CoAtNet, the FFN layer is a fully connected layer used to perform linear transformations and non-linear activations on features to enhance the model's representational power. Rel-Attention refers to the relative attention mechanism. It is an attention mechanism in CoAtNet used to consider the relative relationship of positions when calculating attention weights. Global Pooling is global average pooling.

[0048] The Bottleneck feature layers of size (80×80×512), (40×40×1024), and (20×20×1024) are connected to the YOLO V7 Neck via 1×1 convolution. Simultaneously, the (20×20×1024) feature layer is upsampled and then fused with the (40×40×1024) feature layer through an improved NAS-FPN network. Further feature extraction is then performed through a CoAtNet network to obtain a (40×40×256) feature layer, which is then processed by a CBR layer and upsampled. Finally, this is fused with the (80×80×512) feature layer through an improved NAS-FPN network and further extracted through a CoAtNet network to obtain a (80×80×128) feature layer.

[0049] The feature layer with a size of (80×80×128) is passed to YoloHead via RepConv, processed by a CBR layer and downsampled, and then fused with a feature layer with a size of (40×40×256) for further feature extraction to obtain a feature layer with a size of (40×40×256). This (40×40×256) feature layer is then passed to YoloHead, processed by a CBR layer and downsampled, and then fused with a feature layer with a size of (20×20×1024) after a 1×1 convolution for feature extraction to obtain a feature layer with a size of (20×20×512), which is then passed to YoloHead via RepConv.

[0050] Traditional YOLO V7 adds a RepConv structure before YOLOHead. The basic idea is to introduce a special residual structure to assist training. This residual structure is uniquely designed. In actual prediction, the complex residual structure can be equivalent to a regular 3×3 convolution, which can reduce network complexity and ensure prediction performance.

[0051] The Yolo V7 Head module outputs three feature maps with dimensions of 80×80×[3×(4+1+i)], 40×40×[3×(4+1+i)], and 20×20×[3×(4+1+i)]. The first two numbers represent the length and width of the feature map, respectively. 3 indicates that each feature map contains 3 prior boxes. 4 represents the location information. 1 represents the confidence level, which indicates the probability that an object may exist in the current grid. i represents the probability that the grid contains an object of class i.

[0052] After predicting different types of bounding boxes for each feature point, the final prediction result that meets the requirements is obtained by using the non-maximum suppression method.

[0053] S3: Use the abnormal behavior dataset of medical waste temporary storage stations to train a fast medical waste abnormal behavior detection model based on improved YOLO V7. By setting network parameters such as the number of iterations and the initial learning rate, the model is trained and tested to obtain a deep learning model for real-time detection of abnormal behavior in medical waste temporary storage stations.

[0054] S4: Based on real-time video images collected from the medical waste temporary storage station, preprocessing is performed to standardize the image size. This includes the following:

[0055] Video frame sampling: Since videos are typically composed of consecutive frames, to improve processing efficiency and reduce redundant calculations, video frames are sampled, selecting one frame every few frames for processing. This ensures a high frame rate while reducing data volume and computational complexity.

[0056] Adjusting image size: To input the video into the model for processing, resize the video frames to 640×640. Alternatively, resize to 640×640 by cropping and padding. This helps maintain consistency in the network input, ensuring it adapts to the model's input requirements.

[0057] Image normalization: During preprocessing, the pixel values ​​of an image can be normalized, scaling their range to the [0, 1] interval. Normalization ensures a uniform numerical range for the input image, making it easier for the model to learn and process.

[0058] S5: Input the preprocessed image to be detected into the deep learning model for real-time detection of abnormal behavior in the medical waste temporary storage station, and provide real-time warnings for abnormal behavior. Specifically, this includes the following:

[0059] The system will conduct real-time monitoring of any abnormal behavior at the medical waste storage station, such as staff not wearing masks or gloves, unauthorized entry by strangers, uncovered medical waste storage bins, presence of smoke or flames, and staff smoking. It will also issue warnings upon detecting any abnormal behavior.

[0060] The present invention and its embodiments have been described above illustratively. This description is not restrictive, and the figures shown are only one embodiment of the present invention; the actual structure is not limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present invention, such designs should fall within the protection scope of the present invention.

Claims

1. A method for real-time detection of abnormal behavior in a medical waste temporary storage station based on deep learning, characterized in that, Includes the following steps: S1: Collect images of abnormal behavior at medical waste temporary storage stations and establish an abnormal behavior detection dataset; S2: Use the MobileNetV3 network structure to replace the traditional YOLO V7 Backbone module, and use the improved NAS-FPN network and CoAtNet network to replace the traditional YOLO V7 Neck module to build a rapid medical waste abnormal behavior detection model based on the improved YOLO V7; The replaced Neck module uses an improved NAS-FPN network to fuse feature images of different sizes; it uses a CoAtNet network to further extract features from the image fused by the improved NAS-FPN network; and it uses the feature image further extracted by the CoAtNet network as the output feature layer and inputs it into the Head module of YOLO V7. The replaced Neck module is convolved with the output feature images of the three Bottleneck units through a 1×1 convolution; The improved NAS-FPN network fuses feature images of different sizes as follows: Input feature images P2 and P3 are processed by a global average pooling layer (GP) to obtain feature image P4. Feature image P4 is processed by a CBR module to obtain feature image P5. Feature image P5 is added to P1 to obtain output feature image C1. Output feature image C1 is processed by a CBR module to obtain feature image P6. Feature image P6 is added to P5 to obtain output feature image C2. Output feature image C2 is processed by a CBR module to obtain feature image P7. Feature image P7 is processed by a global average pooling layer (GP) to obtain feature image P8. Feature image P8 is added to P3 to obtain output feature image C3. C1, C2, and C3 are the three feature images output by the replaced Neck module. S3: Train the detection model using the abnormal behavior dataset to obtain a deep learning model for real-time detection of abnormal behavior; S4: Based on the video images collected in real time from the medical waste temporary storage station, preprocess them to unify the image size; S5: Input the preprocessed image to be detected into the deep learning model for real-time detection of abnormal behavior and provide real-time warnings for abnormal behavior.

2. The method for real-time detection of abnormal behavior in a medical waste temporary storage station based on deep learning according to claim 1, characterized in that: The replaced Backbone module includes an input layer, a convolutional layer, and eight Bottleneck units. The number of Bottleneck units in each Bottleneck unit is 1, 2, 3, 4, 3, 3, 2, 2, respectively. The features extracted from three Bottleneck units are selected as the output features of the Backbone.

3. The method for real-time detection of abnormal behavior in a medical waste temporary storage station based on deep learning according to claim 2, characterized in that: The features extracted from the 4th, 6th, and 8th Bottleneck units are selected as the output features of the replaced Backbone module.

4. The method for real-time detection of abnormal behavior in a medical waste temporary storage station based on deep learning according to claim 3, characterized in that: The Bottleneck module consists of a 1×1 convolutional layer, a BN batch normalized layer, an activation function layer, a DConv depthwise separable convolutional layer, a BN batch normalized layer, a ReLU activation function, and a 1×1 convolutional layer connected in sequence.

5. The method for real-time detection of abnormal behavior in a medical waste temporary storage station based on deep learning according to claim 4, characterized in that: An SE channel attention mechanism network is added between the ReLU activation function and the 1×1 convolution in the 3rd, 4th, 7th, and 8th Bottleneck units; The activation function layers in the 5th, 6th, 7th, and 8th Bottleneck units use the h-swish activation function, while the activation function layers in the remaining Bottleneck units use the ReLU6 activation function.

6. The method for real-time detection of abnormal behavior in a medical waste temporary storage station based on deep learning according to claim 5, characterized in that: The SE channel attention mechanism network consists of a global average pooling layer, a fully connected layer (FC), an activation function ReLU, another fully connected layer (FC), and an activation function Sigmoid, connected in sequence.

7. The method for real-time detection of abnormal behavior in a medical waste temporary storage station based on deep learning according to claim 6, characterized in that: The CBR module includes a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence.

Images