A dynamic target detection method, system and medium

Abstract

The present application provides a dynamic target detection method, system and medium, the method includes 1) using an improved YOLOv7 dual-stream feature extraction Siamese network to simultaneously process standard samples and real-time monitoring samples, and by comparing the feature differences between the two, to capture key safety features; 2) integrating the CBAM attention module into the feature fusion module of YOLOv7, using the spatial and channel attention mechanism to enhance the expression of key features and suppress irrelevant information; 3) using the features output by the dual-stream feature extraction Siamese network and the multi-scale feature aggregation network, using a variable convolution and attention mechanism to optimize the detection head for target detection; 4) calculating the loss function. The present invention can effectively cope with many challenges in industrial safety monitoring, and the existing technology often has difficulty in distinguishing closely arranged or partially occluded targets, resulting in high false alarm and missed alarm rates.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power safety monitoring systems, and in particular to a dynamic target detection method, system and medium. Background Art

[0002] With the improvement of industrial automation and intelligence, the safety management of power system workplaces has gradually relied on high-tech monitoring systems. However, although existing monitoring technologies perform well in some simple scenarios, the effectiveness of these systems still faces many challenges in complex industrial environments, such as high noise, dust, changing light conditions and occlusion.

[0003] The following main defects and deficiencies can be summarized for the existing technologies of power system safety monitoring systems: (1) Low efficiency and slow speed: When processing high-resolution video streams, existing target detection systems are often slow due to high model complexity. This makes it difficult for the system to achieve true real-time monitoring and cannot provide timely feedback on emergency situations, which may miss the critical moment to prevent accidents; (2) Insufficient adaptability and flexibility: Many monitoring systems lack sufficient adaptability in their design and are difficult to cope with different environmental conditions, such as changes in lighting and weather conditions. When environmental conditions change, the detection performance of the system may drop significantly, affecting the overall monitoring effect; (3) Insufficient detection of small targets and occluded targets: In complex or crowded working environments, the accurate identification of small targets such as helmets and occluded targets remains a challenge. Existing technologies often have difficulty distinguishing closely arranged or partially occluded targets, resulting in high false alarm and missed alarm rates. Summary of the invention

[0004] The purpose of the present invention is to provide a dynamic target detection method, system and medium, and in particular to a dynamic target detection method based on an improved YOLOv7 dual-stream feature extraction Siamese network and CBAM feature fusion to solve the problems raised in the above background technology.

[0005] To achieve the above objectives, in a first aspect, an embodiment of the present application provides a dynamic target detection method, comprising the following specific steps:

[0006] 1) Use the improved YOLOv7 two-stream feature extraction Siamese network to simultaneously process standard samples and real-time monitoring samples, and capture key security features by comparing the feature differences between the two;

[0007] 2) Integrate the CBAM attention module into the feature fusion module of YOLOv7, and use the spatial and channel attention mechanisms to enhance the expression of key features and suppress irrelevant information;

[0008] 3) Using the dual-stream feature extraction Siamese network and the features output by the multi-scale feature aggregation network, a detection head with variable convolution and attention mechanism is used to optimize target detection;

[0009] 4) Calculate the loss function.

[0010] The improved YOLOv7 dual-stream feature extraction Siamese network is used to simultaneously process standard samples and real-time monitoring samples, and the key security features are captured by comparing the feature differences between the two. Specifically,

[0011] Build the YOLOv7-SiamFF model based on the Siamese-ResNet50 network framework;

[0012] In the Backbone module, a dual-channel mode is used to process the input images of standard samples and real-time monitoring samples respectively;

[0013] The first two layers of ResNet50 are used as feature extraction layers with shared weights to extract low-level features from standard samples and real-time monitoring samples respectively. The extracted low-level features are input to the feature difference module, which calculates the low-level feature difference of the two channels through formula (1): (1)

[0014] is the feature extractor of the standard sample, is a feature extractor for real-time monitoring samples. and Shared weights, is the difference feature of the output features.

[0015] The CBAM attention module is integrated into the feature fusion module of YOLOv7, and the spatial and channel attention mechanisms are used to enhance the expression of key features and suppress irrelevant information. Specifically,

[0016] The CBAM attention module is introduced into the information extracted from the Backbone feature fusion module and the attention-based difference information to enhance the expression ability of the difference. The enhanced difference information is extracted from the image through a convolutional feature extraction network designed based on the last three layers of ResNet50 to enhance the expression ability of the difference feature.

[0017] The channel attention module of the CBAM attention module compresses the spatial dimension of the input feature map through average pooling and maximum pooling, which is expressed by formula (2):

[0018] (2)

[0019] Where F is the input feature map; AvgPool represents the average pooling operation; MaxPool represents the maximum pooling operation; MLP represents the fully connected layer; is the Sigmoid activation function;

[0020] The spatial attention module compresses the channels of the input feature map through average pooling and maximum pooling, which can be expressed as formula (3):

[0021] (3)

[0022] in represents an operation with a convolution kernel size of 7×7,

[0023] The final output of the CBAM attention module is expressed as formula (4):

[0024] (4)

[0025] Input represents the difference feature map obtained from the subtraction operation, which is input into the CBAM attention module to obtain its feature representation based on the attention mechanism. Then, a 1×1 convolution operation is applied to adjust its channel dimension. At the same time, the difference feature map is also input into a convolution module to obtain its feature representation based on the convolution layer. Finally, the feature maps of the two branches are added to obtain an enhanced feature fusion map.

[0026] The method of using dual-stream features to extract features output by Siamese network and multi-scale feature aggregation network, and using a detection head with variable convolution and attention mechanism to optimize target detection is as follows:

[0027] The traditional yolov7 head is improved by equipping it with an attention mechanism to obtain a dynamic head;

[0028] The dynamic head uses the attention mechanism between feature layers, spatial positions and output channels to achieve scale perception, space perception and task perception. In order to make full use of the perception ability of the dynamic head, the feature map fused by the CBAM attention module is sent to the dynamic head to perform scale perception, space perception and task perception operations.

[0029] The feature map fused by the CBAM attention module is sent to the dynamic head to perform scale perception, space perception and task perception operations. Specifically, after the fused feature map is input to the dynamic head, the feature maps are first average pooled, then a 1×1 convolution operation is performed, and the Hard Sigmoid function is used for activation. Finally, the generated tensor is multiplied with the input feature map to complete the perceptual interaction between the feature maps and realize scale perception.

[0030] The scale-aware feature map is input into the spatial perception module, where a 3×3 convolution operation is used to calculate the offset value and weight term of the feature map offset. After activating these values ​​through the Sigmoid function, a deformable convolution operation is performed to further realize spatial perception. This comprehensive attention processing strategy effectively improves the performance and accuracy of the model in complex environments.

[0031] The loss function is specifically:

[0032] ,

[0033] is a tag set, are the remaining labels in the label space, is the model’s output score for the i-th class, is the output score of the model for the jth class, and the constant value of b is set to 0.005.

[0034] In a second aspect, an embodiment of the present application provides a dynamic target detection system, including a memory and a processor, wherein the memory includes a program of a dynamic target detection method, and when the program of the dynamic target detection method is executed by the processor, the following steps are implemented: using an improved YOLOv7 dual-stream feature extraction Siamese network to simultaneously process standard samples and real-time monitoring samples, and capturing key safety features by comparing the feature differences between the two; integrating the CBAM attention module into the feature fusion module of YOLOv7, and using spatial and channel attention mechanisms to enhance the expression of key features and suppress irrelevant information; using the features output by the dual-stream feature extraction Siamese network and the multi-scale feature aggregation network, and using a detection head with variable convolution and attention mechanism to perform target detection; and calculating the loss function.

[0035] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the dynamic target detection method as described above are implemented.

[0036] Technical effects and advantages of the present invention:

[0037] The present invention proposes an improved YOLOv7 dual-stream feature extraction Siamese network and CBAM feature fusion network technology that can effectively cope with many challenges in industrial safety monitoring. The dual-stream Siamese network can process real-time images from security surveillance cameras and preset standard images of safety equipment wearing, respectively, and judge whether there are safety violations by comparing the feature differences between the two images. At the same time, the CBAM feature fusion network technology ensures that the model can still effectively identify safety helmets and work clothes even when the worker is far away from the camera or partially blocked by other objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the method of the present invention;

[0039] Figure 2 This is the Siamese-ResNet50 network framework diagram of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The present invention provides Figure 1 A dynamic target detection method shown in FIG. 1 includes the following specific steps:

[0042] 1) Use the improved YOLOv7 two-stream feature extraction Siamese network to simultaneously process standard samples and real-time monitoring samples, and capture key security features by comparing the feature differences between the two;

[0043] 2) Integrate the CBAM attention module into the feature fusion module of YOLOv7, and use the spatial and channel attention mechanisms to enhance the expression of key features and suppress irrelevant information;

[0044] 3) Using the dual-stream feature extraction Siamese network and the features output by the multi-scale feature aggregation network, a detection head with variable convolution and attention mechanism is used to optimize target detection;

[0045] 4) Calculate the loss function.

[0046] Step 1: Use the two-stream feature extraction Siamese network (SiamFF) to process standard samples and real-time monitoring samples simultaneously

[0047] First, the YOLOv7-SiamFF model is based on the Siamese-ResNet50 network framework. Compared with the traditional yolov7 framework, we use a dual-channel mode in the Backbone module to process the input images of standard samples and real-time monitoring samples respectively. The first two layers of ResNet50 are used as feature extraction layers with shared weights to extract low-level features from standard samples and real-time monitoring samples respectively. These features are then input to the feature difference module, which calculates the low-level feature difference of the two channels through formula (1): (1)

[0048] is the feature extractor of the standard sample, is a feature extractor for real-time monitoring samples. and Shared weights, is the difference feature of the output features.

[0049] In the Siamese-ResNet50 network framework, as follows Figure 2 As shown in the figure, the standard and real-time monitoring sample images with a size of 640×640×3 are fed into their respective input channels. The two feature maps are simultaneously passed through Snet for low-level feature extraction. Snet consists of the first two feature extraction layers of ResNet50 and a deep information feature fusion module, which effectively captures the low-level feature information from each image. The parameter weights of the two Snets are fully shared, and their respective feature extraction results are input into the feature difference module to capture the difference between the two samples.

[0050] Step 2: The CBAM attention module is integrated into the feature fusion module of YOLOv7;

[0051] The feature differences extracted in the previous step are enhanced through an attention mechanism feature fusion module. The present invention introduces the CBAM (Convolutional Block Attention Module) attention module into the information extracted in the Backbone feature fusion module and the attention-based difference information to enhance the expressiveness of the difference. The enhanced difference information is extracted from the image through a convolutional feature extraction network designed based on the last three layers of ResNet50 to enhance the expressiveness of the difference features. CBAM is a plug-and-play feature extraction module that combines spatial attention and channel attention mechanisms. It learns the importance of each feature in space and channel, and enhances the expressiveness of important features while suppressing irrelevant information.

[0052] The channel attention module compresses the spatial dimension of the input feature map through average pooling and maximum pooling, which can be expressed by formula (2):

[0053] (2)

[0054] Where F is the input feature map; AvgPool represents the average pooling operation; MaxPool represents the maximum pooling operation; MLP represents the multi-layer perceptron (fully connected layer); is the Sigmoid activation function.

[0055] The spatial attention module compresses the channels of the input feature map through average pooling and maximum pooling, which can be expressed by formula (3):

[0056] (3)

[0057] in Represents a convolution operation with a kernel size of 7×7.

[0058] The final output of the CBAM attention module is expressed as formula (4):

[0059] (4)

[0060] The input represents the difference feature map obtained from the subtraction operation, which is input into the CBAM module to obtain its feature representation based on the attention mechanism. Then, a 1×1 convolution operation is applied to adjust its channel dimension. At the same time, the difference feature map is also input into a convolution module to obtain its feature representation based on the convolution layer. Finally, the feature maps of the two branches are added to obtain an enhanced feature fusion map.

[0061] Step 3: Use a detection head with variable convolution and attention mechanism to perform efficient object detection;

[0062] In complex target detection scenarios, the target detection head needs to be able to detect objects of different scales and have the ability to detect from multiple perspectives. We have made significant improvements to the traditional YOLOv7 Head part. The Dynamic Head equipped with an attention mechanism achieves scale perception, spatial perception, and task perception by utilizing the attention mechanism between feature layers, spatial positions, and output channels. In order to fully utilize the perception ability of the dynamic head, the feature map fused by the CBAM attention module is sent to the dynamic head to perform scale perception, spatial perception, and task perception operations.

[0063] After the fused feature maps are input into the dynamic head, the feature maps are first average pooled (AvgPool). Then, a 1×1 convolution operation is performed and activated by the Hard Sigmoid function. Finally, the generated tensor is multiplied by the input feature map to complete the perceptual interaction between the feature maps, which mainly involves scale perception.

[0064] The scale-aware feature map is then fed into the spatial perception module. In this module, a 3×3 convolution operation is used to calculate the offset value and weight term of the feature map offset. After activating these values ​​through the Sigmoid function, a deformable convolution operation is performed to further achieve spatial perception. This comprehensive attention processing strategy effectively improves the performance and accuracy of the model in complex environments.

[0065] Step 4: Calculate the loss function

[0066] When dealing with multi-label classification tasks, they are often treated as a series of independent binary classification tasks. In order to achieve multi-label classification, each category is activated using the Sigmoid function, and then the binary cross entropy loss (BCE Loss) is used to calculate the classification loss of each binary category. This method indirectly achieves the goal of multi-label classification. However, when the positive and negative categories are extremely unbalanced, this method may perform poorly. In contrast, the ZLPR (zero-bounded logarithmic accumulation and paired ranking) loss uses the "Softmax+cross entropy" strategy, which can effectively solve this problem. The formula of ZLPR is as follows:

[0067]

[0068] in, is a tag set, are the remaining labels in the label space, is the model's output score for the i-th class. When predicting, Indicates that it may be the target category, No, this is what "zero bound" means.

[0069] After the loss function is trained to a small value, continuing training may cause the model to overfit, and the loss on the training set continues to decrease, while the loss on the test set gradually increases. In order to deal with this phenomenon, the present invention proposes a technology called "flooding", the formula of which is as follows:

[0070]

[0071] Where L is the original learning target, is the modified learning objective, and b>0 is the user-specified threshold. Once , the original gradient descent will become gradient ascent.

[0072] Based on the above equation, the present invention proposes a new multi-label classification loss function, which is expressed as follows:

[0073]

[0074] The loss function of the present invention not only effectively solves the problem that the loss function cannot effectively converge due to the imbalance of label categories, but also can further optimize the model parameters when the classification loss iterates to a smaller value. In our experiments, this loss function is used to calculate the loss for both classification tasks and detection tasks. According to experience, the constant value of b is set to 0.005.

[0075] An embodiment of the present application provides a dynamic target detection system, including a memory and a processor, wherein the memory includes a program of a dynamic target detection method, and when the program of the dynamic target detection method is executed by the processor, the following steps are implemented: using an improved YOLOv7 dual-stream feature extraction Siamese network to simultaneously process standard samples and real-time monitoring samples, and capturing key safety features by comparing the feature differences between the two; integrating a CBAM attention module into a feature fusion module of YOLOv7, and using spatial and channel attention mechanisms to enhance the expression of key features and suppress irrelevant information; using the features output by the dual-stream feature extraction Siamese network and the multi-scale feature aggregation network, and using a detection head with variable convolution and attention mechanisms to perform target detection; and calculating a loss function.

[0076] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the dynamic target detection method as described above are implemented.

[0077] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0078] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0079] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0081] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0082] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0083] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0084] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A dynamic target detection method, characterized in that: The specific steps include: 1) Use the improved YOLOv7 two-stream feature extraction Siamese network to simultaneously process standard samples and real-time monitoring samples, and capture key security features by comparing the feature differences between the two; 2) Integrate the CBAM attention module into the feature fusion module of YOLOv7, and use the spatial and channel attention mechanisms to enhance the expression of key features and suppress irrelevant information; 3) Using the dual-stream feature extraction Siamese network and the features output by the multi-scale feature aggregation network, a detection head with variable convolution and attention mechanism is used to optimize target detection; 4) Calculate the loss function.

2. A dynamic target detection method according to claim 1, characterized in that: The improved YOLOv7 dual-stream feature extraction Siamese network is used to simultaneously process standard samples and real-time monitoring samples, and the key security features are captured by comparing the feature differences between the two. Specifically, Build the YOLOv7-SiamFF model based on the Siamese-ResNet50 network framework; In the Backbone module, a dual-channel mode is used to process the input images of standard samples and real-time monitoring samples respectively; The first two layers of ResNet50 are used as feature extraction layers with shared weights to extract low-level features from standard samples and real-time monitoring samples respectively. The extracted low-level features are input to the feature difference module, which calculates the low-level feature difference of the two channels through formula (1): (1) is the feature extractor of the standard sample, is a feature extractor for real-time monitoring samples. and Shared weights, is the difference feature of the output features.

3. A dynamic target detection method according to claim 2, characterized in that: The CBAM attention module is integrated into the feature fusion module of YOLOv7, and the spatial and channel attention mechanisms are used to enhance the expression of key features and suppress irrelevant information. Specifically, The CBAM attention module is introduced into the information extracted from the Backbone feature fusion module and the attention-based difference information to enhance the expression ability of the difference. The enhanced difference information is extracted from the image through a convolutional feature extraction network designed based on the last three layers of ResNet50 to enhance the expression ability of the difference feature.

4. A dynamic target detection method according to claim 3, characterized in that: The channel attention module of the CBAM attention module compresses the spatial dimension of the input feature map through average pooling and maximum pooling, which is expressed by formula (2): （2） Where F is the input feature map; AvgPool represents the average pooling operation; MaxPool represents the maximum pooling operation; MLP represents the fully connected layer; is the Sigmoid activation function; The spatial attention module compresses the channels of the input feature map through average pooling and maximum pooling, which can be expressed as formula (3): （3） in represents an operation with a convolution kernel size of 7×7, The final output of the CBAM attention module is expressed as formula (4): （4） Input represents the difference feature map obtained from the subtraction operation, which is input into the CBAM attention module to obtain its feature representation based on the attention mechanism. Then, a 1×1 convolution operation is applied to adjust its channel dimension. At the same time, the difference feature map is also input into a convolution module to obtain its feature representation based on the convolution layer. Finally, the feature maps of the two branches are added to obtain an enhanced feature fusion map.

5. A dynamic target detection method according to claim 1, characterized in that: The method of using dual-stream features to extract features output by Siamese network and multi-scale feature aggregation network, and using a detection head with variable convolution and attention mechanism to optimize target detection is as follows: The traditional yolov7 head is improved by equipping it with an attention mechanism to obtain a dynamic head; The dynamic head uses the attention mechanism between feature layers, spatial positions and output channels to achieve scale perception, space perception and task perception. In order to make full use of the perception ability of the dynamic head, the feature map fused by the CBAM attention module is sent to the dynamic head to perform scale perception, space perception and task perception operations.

6. A dynamic target detection method according to claim 5, characterized in that: The feature map fused by the CBAM attention module is sent to the dynamic head to perform scale perception, space perception and task perception operations. Specifically, after the fused feature map is input to the dynamic head, the feature maps are first average pooled, then a 1×1 convolution operation is performed, and the Hard Sigmoid function is used for activation. Finally, the generated tensor is multiplied with the input feature map to complete the perceptual interaction between the feature maps and realize scale perception. The scale-aware feature map is input into the spatial perception module, where a 3×3 convolution operation is used to calculate the offset value and weight term of the feature map offset. After activating these values ​​through the Sigmoid function, a deformable convolution operation is performed to further realize spatial perception. This comprehensive attention processing strategy effectively improves the performance and accuracy of the model in complex environments.

7. A dynamic target detection method according to claim 1, characterized in that: The loss function is specifically: ， is a tag set, are the remaining labels in the label space, is the model’s output score for the i-th class, is the output score of the model for the jth class, and the constant value of b is set to 0.

005.

8. A dynamic target detection system, characterized in that: The invention comprises a memory and a processor, wherein the memory comprises a program of a dynamic target detection method, and when the program of the dynamic target detection method is executed by the processor, the following steps are implemented: using an improved YOLOv7 dual-stream feature extraction Siamese network to simultaneously process standard samples and real-time monitoring samples, and by comparing the feature differences between the two, to capture key safety features; The CBAM attention module is integrated into the feature fusion module of YOLOv7, and the spatial and channel attention mechanisms are used to enhance the expression of key features and suppress irrelevant information; the features output by the Siamese network and the multi-scale feature aggregation network are extracted using two-stream features, and a detection head with variable convolution and attention mechanism is used to optimize target detection; the loss function is calculated.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the dynamic target detection method according to any one of claims 1 to 7 are implemented.

Images