A detection method and system based on improved yolov5-sff
By improving the YOLOv5-SFF detection method and utilizing differential hashing sampling and feature fusion modules, the problem of poor detection and recognition of worker operation behavior in complex backgrounds by YOLOv5 is solved, achieving high-precision and real-time detection results.
Patent Information
- Application Number
- CN202211392625.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-08
- Publication Date
- 2025-12-09
- Estimated Expiration
- 2042-11-08
AI Technical Summary
Existing YOLOv5 object detection algorithms have poor feature extraction capabilities for worker operation behavior in complex industrial backgrounds and ignore the temporal correlation between frames, resulting in poor detection and recognition performance.
An improved YOLOv5-SFF detection method is adopted. By filtering video frames through differential hash sampling algorithm, adding channel filtering module and spatial attention module, a YOLOv5-SFF detection model is constructed, and the correlation between multiple frames is used for target detection and behavior recognition.
It improves detection and recognition accuracy while ensuring the real-time detection speed of the algorithm, making the model training process more stable and exhibiting better robustness.
Smart Images
Figure CN115761881B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of video recognition, and particularly relates to a detection method and system based on improved yolov5-SFF. BACKGROUND
[0002] A factory monitoring camera can produce several T-level effective factory video data every day, and most of these videos are only used for monitoring workers' production. In fact, these factory video data contain a large amount of workers' operation behaviors and production operation modes contained therein, which can be further used for action recognition, abnormal event monitoring and the like. Identifying and analyzing the production operation behaviors of workers can help workers standardize operation behaviors, discover abnormal operations as soon as possible, and also avoid economic losses and dangerous accidents caused by operation errors. Therefore, real-time detection and identification of workers' production operation behaviors have great significance for industrial production.
[0003] The detection and identification of workers' operation behaviors mainly include four steps: target detection, target tracking, feature extraction and behavior recognition. The Yolov5 target detection algorithm based on deep learning can be used to detect the position and category of the target, but it has poor feature extraction capability for workers' operation behaviors in a complex industrial background, and ignores the time sequence correlation between frame pictures, resulting in poor detection and identification effect of workers' operation behaviors. SUMMARY
[0004] The purpose of the present application is to provide a detection method and system based on improved yolov5-SFF, which simultaneously performs target detection and behavior recognition, improves the detection and identification accuracy, and also ensures the real-time detection speed of the algorithm.
[0005] To achieve the above purpose, the technical scheme adopted by the present application is:
[0006] The present application provides a detection method based on improved yolov5-SFF in the first aspect, comprising:
[0007] Collecting real-time video data containing workers' operation behaviors, inputting the real-time video data into a pre-trained Yolov5-SFF detection model to obtain a detection result of the workers' operation behaviors;
[0008] The training process of the Yolov5-SFF detection model comprises:
[0009] Collecting historical video data containing workers' operation behaviors, constructing a training data set; using a difference hash sampling algorithm to sample and screen the frame pictures of each set of historical video data, and labeling the operation behaviors in the screened frame pictures;
[0010] The channel filtering module and the spatial attention module are added to the backbone network, and the backbone network, the head network, the spatio-temporal feature fusion module and the identification module are sequentially stacked to construct a Yolov5-SFF detection model.
[0011] The Yolov5-SFF detection model is trained by using the historical video data with the operation behavior annotation added in the training data set, and the training is repeated until the detection accuracy of the Yolov5-SFF detection model tends to be stable.
[0012] Preferably, the method for sampling and screening the frame pictures of each set of historical video data by using the differential hash sampling algorithm comprises:
[0013] The frame pictures of each set of historical video data are scaled to a set size, and the scaled frame pictures are grayscaled.
[0014] The left and right two pixels in each row of the frame picture are compared, if the pixel value on the left is greater than the pixel value on the right, it is recorded as 1, otherwise as 0, and the fingerprint sequence of the frame picture is obtained.
[0015] The number of difference values between the fingerprint sequences of each frame picture is counted, and the number of difference values between the two frame pictures is recorded as the Hamming distance.
[0016] The frame pictures of each set of historical video data are screened according to the Hamming distance between the frame pictures.
[0017] Preferably, the backbone network sequentially comprises a convolutional layer, a convolution-C3-AS module and a feature pyramid pooling module; a plurality of convolution-C3-AS modules are arranged between the feature pyramid pooling module and the convolutional layer.
[0018] The convolution-C3-AS module sequentially comprises a convolutional layer and a C3-AS module; the C3-AS module is formed by adding a channel filtering module and a spatial attention module to the C3 module of the Yolov5 model; the input features of the C3-AS module sequentially pass through convolution, n residual modules, a channel filtering module and a spatial attention module to form a feature C; the input features of the C3-AS module pass through convolution to form a feature D; the feature C and the feature D sequentially pass through splicing and convolution to form the output features of the C3-AS module.
[0019] Preferably, the channel filtering module comprises a 2D convolution, an average pooling module and an excitation layer; the input features of the channel filtering module pass through 2D convolution to form a feature E; the feature E passes through the average pooling module to form a feature F; the feature F passes through the excitation layer to form a feature G; a fully connected layer, a batch normalization module, a RELU activation layer, a fully connected layer and a Sigmoid activation layer are sequentially arranged in the excitation layer.
[0020] The feature G and the feature F are multiplied to form a feature H which is soft-thresholded, and the feature H is multiplied with the feature E and added to the input feature of the channel filtering module to form an output feature of the channel filtering module.
[0021] Preferably, the feature G and the feature F are multiplied to form a feature H which is soft-thresholded, and the feature H is multiplied with the feature E and added to the input feature of the channel filtering module to form an output feature of the channel filtering module.
[0022]
[0023]
[0024] RELU(x2)=max(0,x2)
[0025]
[0026] In the formula, X represents the input feature E; AvgPool(·) represents a calculation function of the average pooling module; FC(·) represents a calculation function of the full connection layer; represents multiplication of corresponding elements; ω represents a positive random number matrix with a size of 1×1×C; ω c represents the feature H.
[0027] Preferably, the spatial attention module comprises a max-pooling-average-pooling module, a concatenation module, a 2D convolution, and a Sigmoid activation layer; the input feature of the spatial attention module sequentially passes through the max-pooling-average-pooling module, the concatenation module, the 2D convolution, and the Sigmoid activation layer, and then the input feature of the spatial attention module is multiplied to form the output feature of the spatial attention module; the expression formula is:
[0028] M s (K)=Sigmoid(f 7×7 ([AvgPool(K);MaxPool(K)]))
[0029] The formula is: K represents the input feature of the spatial attention module; M s (K) represents the output feature of the spatial attention module; MaxPool(·) represents a calculation function of the max-pooling-average-pooling module; AvgPool(·) represents a calculation function of the average pooling module; f 7×7 represents a convolution kernel with a size of 7×7; Sigmoid(·) represents a calculation function of the Sigmoid activation layer.
[0030] Preferably, the spatio-temporal feature fusion module comprises a dimension adjustment module, a time domain convolution module and a GELU module; the input feature of the spatio-temporal feature fusion module is processed by the dimension adjustment module to form feature I; feature I is processed by convolution to form feature J; the feature I is sequentially processed by the time domain convolution module, the GELU module, the time domain convolution module and the GELU module, and then added to the feature J to form the output feature of the spatio-temporal feature fusion module.
[0031] Preferably, the size of the input feature of the time domain convolution module is 16xHxC 2 The time convolution module finally can completely cover the feature matrix of 16 images; the convolution kernel size of the causal convolution and the dilated convolution is 2x3.
[0032] Preferably, the function expression of the GELU module is:
[0033]
[0034] In the formula, x3 represents the input feature of the GELU module.
[0035] The second aspect of the present application provides a detection system based on improved yolov5-SFF, comprising:
[0036] The recognition detection unit is used for collecting real-time video data containing worker operation behavior, inputting the real-time video data into a pre-trained Yolov5-SFF detection model to obtain a detection result of the worker operation behavior;
[0037] The training data set construction unit is used for collecting historical video data containing worker operation behavior, and constructing a training data set; using a difference hash sampling algorithm to sample and screen the frame pictures of each group of historical video data, and labeling the operation behavior in the screened frame pictures;
[0038] The model construction unit is used for adding a channel filtering module and a spatial attention module to a backbone network, and sequentially stacking the backbone network, a head network, a spatio-temporal feature fusion module and a recognition module to construct a Yolov5-SFF detection model;
[0039] The training unit trains the Yolov5-SFF detection model using the historical video data in the training data set with operation behavior labels, and repeats the training until the detection accuracy of the Yolov5-SFF detection model tends to be stable.
[0040] The third aspect of the present application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the steps of the detection method.
[0041] Compared with the prior art, the present application has the following advantages:
[0042] The present application collects historical video data containing worker operation behaviors, constructs a training data set, uses a differential hash sampling algorithm to sample and screen the frame pictures of each group of historical video data, and labels the operation behaviors in the screened frame pictures; the frame pictures screened by the differential hash algorithm have certain differences between them, reducing repeated data, making the training process more stable, faster in convergence speed, and the trained model having better robustness, thereby ensuring the real-time detection speed of the algorithm.
[0043] The present application adds a channel filtering module and a spatial attention module to the backbone network, and sequentially stacks the backbone network, the head network, the spatio-temporal feature fusion module and the recognition module to construct a Yolov5-SFF detection model; the Yolov5-SFF detection model is trained using historical video data with operation behavior labels added in the training data set, and the training is repeated until the detection accuracy of the Yolov5-SFF detection model tends to be stable; the correlation between multiple frame pictures is used to accurately determine the current worker behavior through the information in the past frame, thereby improving the detection and recognition accuracy. BRIEF DESCRIPTION OF DRAWINGS
[0044] Figure 1 is a flowchart of the detection method based on the improved yolov5-SFF provided by the present application;
[0045] Figure 2 is a flowchart of the sampling and screening of the frame pictures of each group of historical video data using the differential hash sampling algorithm provided by the present application;
[0046] Figure 3 is a structure diagram of the yolov5-SFF detection model provided by the present application;
[0047] Figure 4 is a structure diagram of the C3-AS module provided by the present application;
[0048] Figure 5 is a structure diagram of the spatio-temporal feature fusion module provided by the present application;
[0049] Figure 6 is a detection effect comparison diagram of the Yolov5-SFF detection model provided by the present application;
[0050] Figure 7 is a result comparison diagram of the Yolov5-SFF and other algorithms provided by the present application. Detailed Implementation
[0051] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.
[0052] Example 1
[0053] like Figures 1 to 5 As shown, a detection method based on improved YOLOv5-SFF includes:
[0054] Collect real-time video data containing worker operation behavior, and input the real-time video data into a pre-trained Yolov5-SFF detection model to obtain the detection results of worker operation behavior;
[0055] The training process of the Yolov5-SFF detection model includes:
[0056] Historical video data containing worker operational behaviors was collected and divided into test and training datasets at a 2:8 ratio. The method for sampling and filtering frames from each group of historical video data using a differential hashing sampling algorithm included:
[0057] Each set of historical video data frames is scaled to a set size; the scaled frames are then converted to grayscale.
[0058] Compare the left and right pixels in each row of the frame. If the left pixel value is greater than the right pixel value, record it as 1; otherwise, record it as 0. This will obtain the fingerprint sequence of the frame.
[0059] The number of differences between the fingerprint sequences of each frame is counted, and the number of differences between two frames is denoted as the Hamming distance.
[0060] Frames in each group of historical video data are filtered based on the Hamming distance between each frame. Operational actions within the filtered frames are then labeled.
[0061] Add a channel filtering module and a spatial attention module to the backbone network, and then stack the backbone network, head network, spatiotemporal feature fusion module and recognition module in sequence to construct a Yolov5-SFF detection model.
[0062] The backbone network sequentially includes convolutional layers, convolution-C3-AS modules, and feature pyramid pooling modules; multiple convolution-C3-AS modules are provided between the feature pyramid pooling modules and the convolutional layers;
[0063] The convolution-C3-AS module sequentially comprises a convolution layer and a C3-AS module; the C3-AS module is formed by adding a channel filtering module and a spatial attention module to the C3 module of the Yolov5 model; the input feature of the C3-AS module sequentially passes through convolution, n residual modules, the channel filtering module and the spatial attention module to form feature C; the input feature of the C3-AS module passes through convolution to form feature D; the feature C and the feature D sequentially pass through splicing and convolution to form the output feature of the C3-AS module.
[0064] The channel filtering module comprises a 2D convolution, an average pooling module and an excitation layer; the input feature of the channel filtering module passes through 2D convolution to form feature E; the feature E passes through the average pooling module to form feature F; the feature F passes through the excitation layer to form feature G; the excitation layer is sequentially provided with a full connection layer, a batch normalization module, a RELU activation layer, a full connection layer and a Sigmoid activation layer;
[0065] The feature G and the feature F are multiplied to form feature H after soft thresholding processing, and the expression formula is:
[0066]
[0067]
[0068] RELU(x2)=max(0,x2)
[0069]
[0070] In the formula, X represents the input feature E; AvgPool(·) represents the calculation function of the average pooling module; FC(·) represents the calculation function of the full connection layer; represents multiplication of corresponding elements; ω represents a positive random number matrix with a size of 1×1×C; ω c represents feature H;
[0071] The feature H is multiplied by the feature E, and then added to the input feature of the channel filtering module to form the output feature of the channel filtering module.
[0072] The spatial attention module comprises a max-pooling-average-pooling module, a splicing module, a 2D convolution and a Sigmoid activation layer; the input feature of the spatial attention module sequentially passes through the max-pooling-average-pooling module, the splicing module, the 2D convolution and the Sigmoid activation layer, and then the input feature of the spatial attention module is multiplied to form the output feature of the spatial attention module; the expression formula is:
[0073] M s (K)=Sigmoid(f7×7 ([AvgPool(K);MaxPool(K)]))
[0074] The formula is: K represents the input feature of the spatial attention module; M s (K) represents the output feature of the spatial attention module; MaxPool(·) represents the calculation function of the max-pooling-average-pooling module; AvgPool(·) represents the calculation function of the average-pooling module; f 7×7 represents a convolution kernel with a size of 7x7; Sigmoid(·) represents the calculation function of the Sigmoid activation layer.
[0075] The spatio-temporal feature fusion module includes a dimension adjustment module, a time domain convolution module, and a GELU module; the input feature of the spatio-temporal feature fusion module is processed by the dimension adjustment module to form a feature I; the feature I is processed by convolution to form a feature J; the feature I is sequentially processed by the time domain convolution module, the GELU module, the time domain convolution module, and the GELU module, and then added to the feature J to form the output feature of the spatio-temporal feature fusion module.
[0076] The function expression of the GELU module is:
[0077]
[0078] In the formula, x3 represents the input feature of the GELU module.
[0079] The size of the input feature of the time domain convolution module is 16xH 2 x C, and the causal convolution, the dilated convolution with a sampling interval of 2, the dilated convolution with a sampling interval of 4, the dilated convolution with a sampling interval of 8, and the batch normalization module are sequentially arranged in the time domain convolution module. The receptive field of the time convolution module can finally completely cover the feature matrix of 16 images; the convolution kernel size of the causal convolution and the dilated convolution is both 2x3.
[0080] The Yolov5-SFF detection model is trained using the historical video data with operation behavior annotations added in the training data set, and the training is repeated until the detection accuracy of the Yolov5-SFF detection model tends to be stable.
[0081] The experimental environment of the embodiment uses Windows 10 operating system, selects pytorch framework, and uses 2060 graphics card for operation.
[0082] In the training process, the iteration number is 300 times; the detection accuracy of the trained Yolov5-SFF detection model is tested by the test data set; the detection accuracy of the improved Yolov5 model is evaluated by calculating the average precision mean value, recall rate, and accuracy.
[0083] The recall rate calculation formula is:
[0084]
[0085] The accuracy rate is:
[0086]
[0087] The average precision is:
[0088]
[0089] Where TP represents the number of samples correctly identified as true; FP represents the number of samples incorrectly identified as true, FN represents the number of samples incorrectly identified as false, n is the action category, and p(r) is the correlation between the accuracy rate and the recall rate.
[0090] As Figure 6 The improved Yolov5-SFF model is used to detect and identify the operation behavior of workers, as shown in Figure 7 The improved Yolov5-SFF algorithm is compared with other similar algorithms, and the results are shown in this embodiment. This embodiment utilizes the correlation between multiple frames of pictures, accurately judges the current worker behavior through the information in the past time pictures, and improves the detection and identification accuracy.
[0091] Embodiment two
[0092] A detection system based on an improved yolov5-SFF, the system provided in this embodiment can be applied to the method described in embodiment one, comprising:
[0093] An identification and detection unit is configured to collect real-time video data containing worker operation behavior, input the real-time video data into a pre-trained Yolov5-SFF detection model to obtain a detection result of the worker operation behavior;
[0094] A training data set construction unit is configured to collect historical video data containing worker operation behavior, and construct a training data set; utilize a differential hash sampling algorithm to sample and screen the frame pictures of each group of historical video data, and label the operation behavior in the screened frame pictures;
[0095] A model construction unit is configured to add a channel filtering module and a spatial attention module to a backbone network, and sequentially stack the backbone network, a head network, a spatio-temporal feature fusion module, and an identification module to construct a Yolov5-SFF detection model;
[0096] A training unit is configured to train the Yolov5-SFF detection model using the historical video data in the training data set with operation behavior labels, and repeat the training until the detection accuracy of the Yolov5-SFF detection model tends to be stable
[0097] Embodiment three
[0098] An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that the processor performs the steps of the detection method according to embodiment one.
[0099] Those skilled in the art will appreciate that embodiments of the present application can be supplied as methods, systems, or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.
[0100] The present application is described in reference to the flowchart illustrations and / or block diagrams according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function specified by the flowchart illustrations and / or block diagrams block or blocks.
[0101] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function specified by the flowchart illustrations and / or block diagrams block or blocks.
[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function specified by the flowchart illustrations and / or block diagrams block or blocks.
[0103] The above merely describes the preferred embodiments of the present application, and it should be pointed out that, for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.
Claims
1. An improved yolov5-SFF-based detection method, characterized in that, The method comprises: collecting real-time video data containing worker operation behaviors, inputting the real-time video data into a pre-trained Yolov5-SFF detection model to obtain a detection result of the worker operation behaviors; The training process of the Yolov5-SFF detection model comprises: collecting historical video data containing worker operation behaviors, constructing a training data set, and using a differential hash sampling algorithm to sample and screen frame pictures of each group of historical video data and label operation behaviors in the screened frame pictures; adding a channel filtering module and a spatial attention module to a backbone network, and sequentially stacking the backbone network, a head network, a spatio-temporal feature fusion module and a recognition module to construct the Yolov5-SFF detection model; the backbone network comprises, in sequence, a convolutional layer, a convolution-C3-AS module and a feature pyramid pooling module; a plurality of convolution-C3-AS modules are arranged between the feature pyramid pooling module and the convolutional layer; the convolution-C3-AS module comprises, in sequence, a convolutional layer and a C3-AS module; the C3-AS module is formed by adding a channel filtering module and a spatial attention module to a C3 module of a Yolov5 model; input features of the C3-AS module sequentially pass through convolution, n residual modules, a channel filtering module and a spatial attention module to form a feature C; the input features of the C3-AS module pass through convolution to form a feature D; the feature C and the feature D sequentially pass through splicing and convolution to form output features of the C3-AS module; the channel filtering module comprises a 2D convolution, an average pooling module and an excitation layer; input features of the channel filtering module pass through 2D convolution to form a feature E; the feature E passes through the average pooling module to form a feature F; the feature F passes through the excitation layer to form a feature G; full connection layers, a batch normalization module, a RELU activation layer, full connection layers and a Sigmoid activation layer are sequentially arranged in the excitation layer; the feature G and the feature F are multiplied to perform soft thresholding processing to form a feature H; the feature H is multiplied by the feature E and then added to the input features of the channel filtering module to form output features of the channel filtering module; the spatio-temporal feature fusion module comprises a dimension adjustment module, a time domain convolution module and a GELU module; input features of the spatio-temporal feature fusion module pass through the dimension adjustment module to form a feature I; the feature I passes through convolution to form a feature J; the feature I sequentially passes through the time domain convolution module, a GELU module, the time domain convolution module and the GELU module, and is added to the feature J to form output features of the spatio-temporal feature fusion module; The size of the input feature of the time domain convolution module is 16×S 2 The time domain convolution module sequentially comprises a causal convolution, an expansion convolution with a sampling interval of 2, an expansion convolution with a sampling interval of 4, an expansion convolution with a sampling interval of 8 and a batch normalization module, and the receptive field of the time domain convolution module can finally completely cover the feature matrix of 16 images; the size of the convolution kernel of the causal convolution and the expansion convolution is 2×3; the Yolov5-SFF detection model is trained using historical video data with operation behavior labels added in the training data set, and the training is repeated until the detection accuracy of the Yolov5-SFF detection model tends to be stable.
2. The improved yolov5-SFF-based detection method according to claim 1, characterized in that, The method for sampling and screening frame pictures of each group of historical video data using the differential hash sampling algorithm comprises: scaling frame pictures of each group of historical video data to a set size; and graying the scaled frame pictures. Comparing each row of left and right two pixels in the frame picture, if the value of the left pixel is greater than the value of the right pixel, it is recorded as 1, otherwise as 0, and the fingerprint sequence of the frame picture is obtained; The number of difference values between the fingerprint sequences of each frame picture is counted, and the number of difference values between the two frame pictures is recorded as the Hamming distance; According to the Hamming distance between each frame picture, the frame pictures of each group of historical video data are screened.
3. The improved yolov5-SFF-based detection method according to claim 1, characterized in that, The feature G and the feature F are multiplied and then soft thresholding processing is performed to form a feature H, and the expression formula is: ； ； ； ； In the formula, X represents an input feature E; AvgPool( ) represents a calculation function of an average pooling module; FC( ) represents a calculation function of a fully connected layer; represents multiplication of corresponding elements; represents a positive random number matrix with a size of 1x1xC; represents a feature H.
4. The improved yolov5-SFF-based detection method according to claim 1, characterized in that, The spatial attention module includes a max-pooling-average-pooling module, a splicing module, a 2D convolution, and a Sigmoid activation layer; the input features of the spatial attention module sequentially pass through the max-pooling-average-pooling module, the splicing module, the 2D convolution, and the Sigmoid activation layer, and then the input features of the spatial attention module are multiplied to form the output features of the spatial attention module; the expression formula is: ))； The formula is: K represents an input feature of the spatial attention module; represents an output feature of the spatial attention module; represents a calculation function of the max-pooling-average-pooling module; AvgPool( ) represents a calculation function of the average-pooling module; represents a convolution kernel with a size of 7*7; represents a calculation function of the Sigmoid activation layer.
5. An improved yolov5-SFF based detection system, characterized in that, It comprises: The recognition detection unit is used for collecting real-time video data containing worker operation behaviors, inputting the real-time video data into a pre-trained Yolov5-SFF detection model to obtain a detection result of the worker operation behaviors; The training data set construction unit is used for collecting historical video data containing worker operation behaviors and constructing a training data set; The frame pictures of each group of historical video data are sampled and screened by using a difference hash sampling algorithm, and the operation behaviors in the screened frame pictures are labeled; The model construction unit is used for adding a channel filtering module and a spatial attention module to a backbone network, and sequentially stacking a backbone network, a head network, a spatio-temporal feature fusion module, and a recognition module to construct a Yolov5-SFF detection model; The training unit trains the Yolov5-SFF detection model by using the historical video data in the training data set to which operation behaviors are labeled, and repeatedly trains until the detection accuracy of the Yolov5-SFF detection model tends to be stable; The backbone network sequentially includes a convolutional layer, a convolution-C3-AS module, and a feature pyramid pooling module; a plurality of convolution-C3-AS modules are arranged between the feature pyramid pooling module and the convolutional layer; The convolution-C3-AS module sequentially includes a convolutional layer and a C3-AS module; the C3-AS module is formed by adding a channel filtering module and a spatial attention module to a C3 module of a Yolov5 model; input features of the C3-AS module sequentially pass through convolution, n residual modules, a channel filtering module, and a spatial attention module to form a feature C; the input features of the C3-AS module pass through convolution to form a feature D; the feature C and the feature D sequentially pass through splicing and convolution to form output features of the C3-AS module; The channel filtering module includes a 2D convolution, an average pooling module, and an excitation layer; input features of the channel filtering module pass through the 2D convolution to form a feature E; the feature E passes through the average pooling module to form a feature F; the feature F passes through the excitation layer to form a feature G; a fully connected layer, a batch normalization module, a RELU activation layer, a fully connected layer, and a Sigmoid activation layer are sequentially arranged in the excitation layer; The feature G and the feature F are multiplied to form a feature H after soft threshold processing, the feature H is multiplied with the feature E, and then is added with the input feature of the channel filtering module to form an output feature of the channel filtering module; The spatio-temporal feature fusion module includes a dimension adjustment module, a time domain convolution module and a GELU module; an input feature of the spatio-temporal feature fusion module is processed by the dimension adjustment module to form a feature I; The feature I is convolved to form a feature J; The feature I is processed by the time domain convolution module, the GELU module, the time domain convolution module and the GELU module in sequence, and then is added with the feature J to form an output feature of the spatio-temporal feature fusion module; The size of the input feature of the time domain convolution module is 16×S 2 The time domain convolution module sequentially comprises a causal convolution, an expansion convolution with a sampling interval of 2, an expansion convolution with a sampling interval of 4, an expansion convolution with a sampling interval of 8 and a batch normalization module, and the receptive field of the time domain convolution module can finally completely cover the feature matrix of 16 images. The size of the convolution kernel of the causal convolution and the expansion convolution is 2×3.
6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the steps of the detection method in any one of claims 1 to 5.