A method for detecting a target of a power operation site personnel in the presence of an occlusion

Abstract

The present application belongs to the technical field of pattern recognition and machine learning, and particularly relates to a power operation site personnel target detection method with occlusion, steps of which comprise: enhancing the visible area features of personnel in the power personnel operation image through a feature extraction module comprising a channel attention module and a spatial attention module; generating a target personnel candidate frame to obtain a third feature map containing an occluded personnel candidate frame; inputting the third feature map into a double-branch classification and regression module to obtain a fourth feature map with a unified size, then obtaining classification scores and regression scores to realize accurate classification judgment and fine-tuning of the candidate area; constructing a joint loss function according to the classification scores and the regression scores for back propagation to realize optimization and training of the overall network, and realizing accurate detection of personnel under the occlusion condition through the overall network. The present application can effectively alleviate the problems of missed detection and false detection of pedestrians under the condition that personnel are occluded in the power operation site, and realize an efficient personnel target detection task.

Description

Technical Field

[0001] This invention belongs to the field of pattern recognition and machine learning technology, specifically relating to a method for detecting personnel targets at power operation sites with occlusion. Background Technology

[0002] In recent years, deep learning-based pedestrian detection algorithms have achieved many significant breakthroughs, demonstrating excellent detection performance in standard testing environments. However, in real-world power operation scenarios, pedestrians are often obstructed by power equipment and other personnel, making it difficult for existing pedestrian detection algorithms to accurately detect occluded pedestrians, thus limiting their application in practical scenarios.

[0003] The difficulties in handling pedestrian occlusion mainly include two aspects: 1) the visibility of the target pedestrian is reduced due to occlusion by objects or other people, making it difficult to extract effective sample features of the occluded person; 2) the detection boxes overlap severely due to mutual occlusion between people, making it difficult to achieve accurate identification and localization of occluded people.

[0004] Currently, pedestrian target detection methods for the occlusion challenge are mainly divided into two categories: 1) single-stage detection methods; 2) two-stage detection methods.

[0005] Single-stage pedestrian detection methods: Single-stage detection methods aim to detect and classify pedestrians in images through a single network inference. These methods improve the recognition of occluded pedestrians in complex scenes due to their fast feature extraction capabilities, reducing the difficulty of extracting effective features from occluded pedestrians. However, they lack a clear region proposal step, making it difficult to accurately distinguish and locate each individual pedestrian when occlusion exists.

[0006] Two-stage pedestrian detection methods aim to generate a series of candidate regions (region proposals) that may contain pedestrians, and then perform fine-grained classification and bounding box regression on each proposed region to achieve more accurate pedestrian detection. This type of method can more accurately identify and locate each pedestrian even in occluded situations. However, it is susceptible to insufficient effective features of occluded persons when generating candidate regions, leading to ineffective detection in subsequent classification and regression steps.

[0007] In summary, although existing pedestrian detection algorithms have made efforts and achieved some results in addressing the pedestrian occlusion problem, the challenges of accurate localization in single-stage detection methods and the difficulties in extracting features of occluded pedestrians in two-stage detection methods limit their practicality and reliability. Therefore, researching and developing pedestrian detection algorithms that can better adapt to the complex environment of power operation sites and effectively address the occlusion problem remains an urgent issue to be addressed. Summary of the Invention

[0008] In view of the shortcomings of the prior art, the present invention provides a method for detecting personnel targets at power operation sites with obstructions, which can effectively alleviate the problem of missed and false detection of pedestrians when personnel are obstructed at power operation sites, and achieve efficient personnel target detection tasks.

[0009] To achieve the above objectives, the present invention provides a method for detecting personnel targets at power work sites with obstructions, comprising the following steps:

[0010] S1. Enhance the extraction of visible region features of power workers in operation images by using a feature extraction module that includes a channel attention module and a spatial attention module:

[0011] S11. Channel guidance is applied to the output features of each residual block through the channel attention module to obtain feature information of the visible body parts of the target person and to obtain first feature maps at different scales.

[0012] S12. Introduce the first feature map into the spatial attention module to obtain the second feature map of the visible body parts of the target person at different scales.

[0013] S2. Generate candidate bounding boxes for target personnel. Input the last scale features of the obtained first feature map and all scale features of the second feature map into the region candidate module for coarse calculation to obtain a third feature map containing candidate bounding boxes for occluded personnel.

[0014] S3. Perform precise positioning and detailed division of the target candidate box. Input the third feature map into the dual-branch classification and regression module. First, obtain the fourth feature map of uniform size through ROIPooling operation. Then, input the fourth feature map into the dual-branch classification and regression module to obtain the classification score and regression score, so as to achieve accurate classification judgment and fine-tuning of the candidate region.

[0015] S4. Construct a joint loss function based on the classification score and regression score for backpropagation to optimize and train the overall network built in S1-S3, and achieve accurate detection of people under occlusion conditions through the overall network.

[0016] In S1, the feature extraction module uses ResNet50 and includes four residual blocks.

[0017] The channel attention module is used to explore the feature maps of the body parts of the person being observed in different residual blocks; the spatial attention module is used to explore the visible features of the person's body parts at different scales observed through the channel. Together, they achieve effective extraction of the visible area features of occluded persons.

[0018] In S11, the channel attention module is defined as:

[0019] H cn =sigmoid(f ml (f m (F 0n ))+f ml (f a (F 0n ))), n=1, 2, 3, 4 (1);

[0020] f ml For a shared multilayer perceptron, f m For the max pooling layer, F 0n These are feature maps at different scales extracted from different residual blocks, where n represents different scales and f is the number of feature maps. a For average pooling layers, H cn This is a channel attention map; where f ml Composed of two fully connected layers f F With a ReLU activation function f RL composition;

[0021] The steps to obtain first feature maps at different scales are as follows:

[0022] Feature maps F extracted from different residual blocks at different scales 0n After passing through f respectively m with f a After processing, then through f ml Processing; then adding them together, and generating a channel attention map H using the sigmoid function. cn Finally, through H cn ×F 0n Obtain the first feature map F with channel weighting 1n .

[0023] In S12, the spatial attention module is defined as:

[0024] H sn =sigmoid(f 7×7 (f c (f m (F 1n ), f a (F 1n )))), n=1, 2, 3, 4 (2);

[0025] In the formula, f 7×7 It is a 7×7 convolutional layer, f c This indicates the cat concatenation operation, H sn Spatial attention map;

[0026] The steps to obtain the second feature map are as follows:

[0027] S11 obtained F1n After f m with f a After processing, it first goes through f c They are pieced together according to the channel dimension, and then processed by f. 7×7 The sigmoid function generates the final spatial attention map H. sn Finally, through H sn ×F 1n The spatially weighted second feature map F is obtained. 2n .

[0028] In S2, the region candidate module is defined as:

[0029]

[0030]

[0031] F 3n =f P (M 1n M 2n ), n=1, 2, 3, 4, 5 (5);

[0032] In the formula, f 1×1 It is a 1×1 convolutional layer, f 3×3 It is a 3×3 convolutional layer. It is a 1×1 convolutional layer with a channel dimension of 2k. For a 1×1 convolutional layer with a channel dimension of 4k, f re For the Reshape layer, f S For the Softmax function, F 14 F when n=4 1n This refers to the final scale feature map in the first feature map. In the region candidate module, its corresponding n is denoted as 5, meaning n = 5 represents F. 14 f P For the Proposal layer, F 3n This is the third feature map;

[0033] M 1n and M 2n For use in calculating F 3n The intermediate result can be understood as: M 1n M represents the intermediate result calculated from the coarse classification branch of the region candidate module. 2n This represents the intermediate results calculated using the coarse regression branch of the region candidate module.

[0034] The steps to obtain the third feature map are as follows:

[0035] The final scale feature map F in the first feature map 14 All scale feature maps F obtained from the second feature map2n via f 1×1 and f 3×3 It then splits into two branches, one of which passes through... f re and f S M was obtained later 1n Another branch road passes through M was obtained later 2n M 1n With M 2n via f P Obtain the third feature map F 3n .

[0036] In S3, the bi-branch classification and regression module is defined as follows:

[0037] F 4n =f RP (F 3n ), n=1, 2, 3, 4, 5 (6);

[0038]

[0039] In the formula, f RP For ROI pooling layer, N C For classification scoring, N R For the regression score, F 4n This is the fourth feature map, where D is the number of features;

[0040] The steps to obtain the fourth feature map and input it into the two-branch classification and regression module are as follows:

[0041] The third feature map F obtained by the region candidate module 3n After f RP Then, a fourth feature map F of uniform size is obtained. 4n Then obtain F 4n The input includes the classification branch and the regression branch, where the classification branch passes through two f... F The classification score N was then obtained. C The return branch passes through two f 1×1 The regression score N was then obtained. R Through N C and N R This allows for accurate classification and fine-tuning of candidate regions.

[0042] In S4, the joint loss function includes a model classification loss function and a model regression loss function. The model classification loss function adopts the cross-entropy loss function, and the model regression loss function adopts the Smooth L1 function. Backpropagation is performed using the gradient descent method.

[0043] The algorithm involved in this invention can be executed by an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the software to implement the algorithm described above.

[0044] The beneficial effects of this invention are:

[0045] This invention addresses the problem of missed and false detections of pedestrians at power work sites due to occlusion. First, it employs a feature extraction module incorporating both channel attention and spatial attention modules. Channel attention is introduced after different residual blocks in the feature extraction module, while spatial attention is introduced into features at different scales that are subject to channel attention, enabling effective extraction of features from the visible area of ​​occluded personnel. Second, in the fine-grained classification and regression process of candidate regions, a dual-branch classification and regression module is proposed. Adding a convolutional branch improves the lack of spatial awareness in the regression branch caused by fully connected layers. Finally, through the combined optimization of these two improvements, the problem of missed and false detections of pedestrians at power work sites with occluded personnel is alleviated, achieving efficient personnel target detection. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the principle of this invention;

[0047] Figure 2 This is a schematic diagram of the overall network principle in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the channel attention module.

[0049] Figure 4 This is a schematic diagram of the spatial attention module.

[0050] Figure 5 This is a structural diagram of the region candidate module;

[0051] Figure 6 This is a schematic diagram of the structure of the bi-branch classification and regression module. Detailed Implementation

[0052] The embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0053] like Figure 1 As shown, a method for detecting personnel targets at an electrical work site with obstructions includes the following steps:

[0054] S1. A feature extraction module containing channel attention and spatial attention modules, using ResNet50 and including four residual blocks, is used to enhance the extraction of visible region features of power workers in images.

[0055] S11. Channel guidance is applied to the output features of each residual block using the channel attention module to obtain feature information of the visible body parts of the target person, resulting in first feature maps F at different scales. 11 -F 14 ;

[0056] S12. Introduce the first feature map into the spatial attention module to obtain the second feature map F of the visible body parts of the target person at different scales. 21 -F 24 ;

[0057] S2. Generate candidate bounding boxes for target personnel. Input the final scale features of the obtained first feature map and all scale features of the second feature map into the region candidate module for coarse calculation to obtain a third feature map F containing the candidate bounding boxes for occluded personnel. 31 -F 35 ;

[0058] S3. Perform precise localization and detailed segmentation of the target candidate boxes. Input the third feature map into the dual-branch classification and regression module, and first obtain a fourth feature map F of uniform size through ROI Pooling. 41 -F 45 Then, the fourth feature map is input into the two-branch classification and regression module to obtain the classification score N. C and regression score N R This enables accurate classification and fine-tuning of candidate regions;

[0059] S4. Construct a joint loss function based on the classification score and regression score for backpropagation to optimize and train the overall network built in S1-S3, and achieve accurate detection of people under occlusion conditions through the overall network.

[0060] The schematic diagram of the overall network built from S1 to S3 is as follows: Figure 2 As shown. Figure 2 In the diagram, P1 represents the input image of power workers at work; in S1, channel attention represents the channel attention module, and spatial attention represents the spatial attention module; in S3, the classification branch network is used to obtain the classification score N. C The branch network, the regression branch network, is used to obtain the regression score N. R Branch network.

[0061] This invention employs a joint channel and spatial attention mechanism in S1 to enhance the feature extraction capability of pedestrian targets under occlusion conditions. First, a channel attention mechanism is introduced to improve the model's focus on the part feature information of the pedestrian target to obtain a first feature map. Then, a spatial attention mechanism is used to recalibrate the feature maps of different scales containing the part feature information of the pedestrian in the spatial dimension to obtain a second feature map focusing on the visible part feature information of the pedestrian, thereby making up for the problem that the feature extraction network cannot extract effective feature information of occluded persons.

[0062] During the generation of candidate bounding boxes for target personnel by S2, after convolution operations, the input feature maps F at different scales are converted into bounding boxes. 14 With F 21 -F 24 Further processing is performed through two parallel branches. One branch is responsible for classifying the content in the candidate box; the other branch is responsible for the preliminary prediction of the candidate box coordinates to obtain the third feature map. S3 performs precise localization and detailed segmentation of the target candidate box.

[0063] In S11, such as Figure 3 As shown, the channel attention module is defined as follows:

[0064] H cn =sigoid(f ml (f m (F 0n ))+f ml (f a (F 0n ))), n=1, 2, 3, 4 (1);

[0065] f ml For a shared multilayer perceptron, f m For the max pooling layer, F 0n These are feature maps at different scales extracted from different residual blocks, where n represents different scales and f is the number of feature maps. a For average pooling layers, H cn This is a channel attention map; where f ml Composed of two fully connected layers f F With a ReLU activation function f RL composition;

[0066] The steps to obtain first feature maps at different scales are as follows:

[0067] Feature maps F extracted from different residual blocks at different scales 0n After passing through f respectively m with f a After processing, then through f ml Processing; then adding them together, and generating a channel attention map H using the sigmoid function. cn Finally, through H cn×F 0n Obtain the first feature map F with channel weighting 11 -F 14 .

[0068] In S12, such as Figure 4 As shown, the spatial attention module is defined as follows:

[0069] H sn =sigmoid(f 7×7 (f c (f m (F 1n ), f a (F 1n )))), n=1, 2, 3, 4 (2);

[0070] In the formula, f 7×7 It is a 7×7 convolutional layer, f c This indicates the cat concatenation operation, H sn Spatial attention map;

[0071] The steps to obtain the second feature map are as follows:

[0072] S11 obtained F 11 -F 14 After f m with f a After processing, it first goes through f c They are pieced together according to the channel dimension, and then processed by f. 7×7 The sigmoid function generates the final spatial attention map H. sn Finally, through H sn ×F 1n The spatially weighted second feature map F is obtained. 21 -F 24 .

[0073] In S2, such as Figure 5 As shown, the region candidate module is defined as follows:

[0074]

[0075]

[0076] F 3n =f P (M 1n M 2n ), n=1, 2, 3, 4, 5 (5);

[0077] In the formula, f 1×1 It is a 1×1 convolutional layer, f 3×3 It is a 3×3 convolutional layer. It is a 1×1 convolutional layer with a channel dimension of 2k. For a 1×1 convolutional layer with a channel dimension of 4k, f re For the Reshape layer, f S For the Softmax function, F 14 F when n=4 1n This refers to the final scale feature map in the first feature map. In the region candidate module, its corresponding n is denoted as 5, meaning n = 5 represents F. 14 f P For the Proposal layer, F 3n This is the third feature map;

[0078] The steps to obtain the third feature map are as follows:

[0079] The final scale feature map F in the first feature map 14 All scale feature maps F obtained from the second feature map 21 -F 24 via f 1×1 and f 3×3 It then splits into two branches, one of which passes through... f re and f S M was obtained later 1n Another branch road passes through M was obtained later 2n M 1n With M 2n via f P Obtain the third feature map F 31 -F 35 .

[0080] In this embodiment, k = 9.

[0081] In S3, such as Figure 6 As shown, the bibranch classification and regression module is defined as follows:

[0082] F 4n =f RP (F 3n ), n=1, 2, 3, 4, 5 (6);

[0083]

[0084] In the formula, f RP For ROI pooling layer, N C For classification scoring, N R For the regression score, F 4n This is the fourth feature map, where D is the number of features;

[0085] The steps to obtain the fourth feature map and input it into the two-branch classification and regression module are as follows:

[0086] The third feature map F obtained by the region candidate module 31 -F 35 After f RP Then, a fourth feature map F of uniform size is obtained. 41 -F 45 Then obtain F 41 -F 45 The input includes the classification branch and the regression branch, where the classification branch passes through two f... F The classification score N was then obtained. C The return branch passes through two f 1×1 The regression score N was then obtained. R Through N C and N R This allows for accurate classification and fine-tuning of candidate regions.

[0087] In S4, the joint loss function includes the model classification loss function and the model regression loss function. The model classification loss function adopts the cross-entropy loss function, and the model regression loss function adopts the Smooth L1 function. Backpropagation is performed through gradient descent.

[0088] S4 can optimize and train the entire network, and finally report the generalization accuracy through the detection results of the test set.

Claims

1. A method for detecting personnel targets at a power work site with obstructions, characterized in that... Includes the following steps: S1. Enhance the extraction of visible region features of power workers in operation images by using a feature extraction module that includes a channel attention module and a spatial attention module: S11. Channel guidance is applied to the output features of each residual block through the channel attention module to obtain feature information of the visible body parts of the target person and to obtain first feature maps at different scales. S12. Introduce the first feature map into the spatial attention module to obtain the second feature map of the visible body parts of the target person at different scales. S2. Generate candidate bounding boxes for target personnel. Input the last scale features of the obtained first feature map and all scale features of the second feature map into the region candidate module for coarse calculation to obtain a third feature map containing candidate bounding boxes for occluded personnel. S3. Perform precise positioning and detailed division of the target candidate box. Input the third feature map into the dual-branch classification and regression module. First, obtain a fourth feature map of uniform size through ROI Pooling operation. Then, input the fourth feature map into the dual-branch classification and regression module to obtain the classification score and regression score, so as to achieve accurate classification judgment and fine-tuning of the candidate region. S4. Construct a joint loss function based on the classification score and regression score for backpropagation to optimize and train the overall network built in S1-S3, and achieve accurate detection of people under occlusion conditions through the overall network. In S1, the feature extraction module uses ResNet50 and includes 4 residual blocks; In S2, the region candidate module is defined as: （3）； （4）； （5）； In the formula, It is a 1×1 convolutional layer. It is a 3×3 convolutional layer. It is a 1×1 convolutional layer with a channel dimension of 2k. It is a 1×1 convolutional layer with a channel dimension of 4k. for layer, for function, When n=4 This refers to the final scale feature map in the first feature map. In the region candidate module, its corresponding n is denoted as 5, meaning n=5 represents... , for layer, This is the third feature map; This is the second feature map; The steps to obtain the third feature map are as follows: The final scale feature map in the first feature map All scale feature maps obtained with the second feature map through and It then splits into two branches, one of which passes through... , and Later obtained Another branch road passes through Later obtained , and through Obtain the third feature map ; In S3, the two-branch classification and regression module is defined as follows: （6）； （7）； In the formula, For ROI pooling layer, To score by category, For the regression score, This is the fourth feature map, where D is the number of features. It is a fully connected layer; The steps to obtain the fourth feature map and input it into the two-branch classification and regression module are as follows: The third feature map obtained by the region candidate module ,through Then, a fourth feature map of uniform size is obtained. Then obtain The input includes the classification branch and the regression branch, where the classification branch passes through two... After obtaining the classification score The return route passes through two branches. Regression score was then obtained. ,pass and This allows for accurate classification and fine-tuning of candidate regions.

2. The method for detecting personnel targets at an obstructed power work site according to claim 1, characterized in that: In S11, the channel attention module is defined as: （1）； For a shared multilayer perceptron, For max pooling layer, These are feature maps at different scales extracted from different residual blocks, where n represents different scales. For average pooling layer, This is a channel attention map; where, Composed of two fully connected layers With one Activation function composition; The steps to obtain first feature maps at different scales are as follows: Feature maps of different scales extracted from different residual blocks After passing through and After processing, then Process; then add them together, after... Function generates channel attention map Finally passed Obtain the first feature map of channel weighting .

3. The method for detecting personnel targets at an obstructed power work site according to claim 2, characterized in that: In S12, the spatial attention module is defined as: （2）； In the formula, It is a 7×7 convolutional layer. This indicates the cat concatenation operation. Spatial attention map; The steps to obtain the second feature map are as follows: S11 obtained go through and After processing, it first goes through They are pieced together according to the channel dimension, and then... and The function generates the final spatial attention map. Finally passed Obtain the spatially weighted second feature map .

4. The method for detecting personnel targets at an obstructed power work site according to claim 3, characterized in that: In S4, the joint loss function includes a model classification loss function and a model regression loss function. The model classification loss function adopts the cross-entropy loss function, and the model regression loss function adopts the Smooth L1 function. Backpropagation is performed using the gradient descent method.

Images