Pixel-level mask assisted camouflage target detection method based on edge prior guidance

Abstract

The invention relates to a pixel-level mask assisted camouflage target detection method based on edge prior guidance, and belongs to the technical field of computer vision. Firstly, Resnet is used as a backbone network to extract five layers of feature maps with different sizes, and the second layer of feature maps and the fifth layer of feature maps are used as input of an edge extraction module to obtain target edge priori with global semantic information. And then, target edge priori guides to extract target features of different scales from the second layer to the fifth layer of feature maps. Target features of different scales are fused in sequence from a high level to a low level through three context information aggregation modules, and a pixel-level mask is obtained through a Sigmoid function of the fused feature map with the same scale as the second-layer feature map. And finally, taking the feature map output by the second layer feature extraction module and the pixel-level mask with the same level scale as the input of a detection head, obtaining the category of the camouflage target and the coordinate of a detection frame, and assisting the detection of the camouflage target to obtain a finer detection result.

Description

Technical Field

[0001] The invention relates to a method for detecting camouflaged targets using pixel-level masks assisted by edge prior guidance, and belongs to the technical field of computer vision. Background Art

[0002] Camouflaged targets refer to objects that are difficult to detect or distinguish in their surroundings by using various means and techniques. This camouflage can involve many aspects, including color, shape, texture, etc., so that the target blends into its environment and is difficult to be detected by the naked eye or sensor system. The design and implementation of camouflaged targets may be for the purpose of concealment, deception, security, etc. Specifically: In the military field, camouflaged targets can be military equipment, vehicles, aircraft, etc., which can be made more difficult to be detected or monitored by the enemy in a battlefield environment by using special coatings, materials or structural designs, thereby increasing the chances of survival and mission execution; in the field of biology, some animals also use camouflage to escape predators or hunt prey more effectively. For example, some animals use the characteristics of body color and texture that can be similar to the surrounding environment to make them difficult to detect in vegetation and avoid being discovered by natural enemies.

[0003] Disguised target detection and segmentation is an important research topic in the field of computer vision and image processing. It is an emerging visual detection task. In recent years, disguised target detection based on deep learning is a research direction that has made significant progress in the field of computer vision. Deep learning methods can learn complex feature representations from large-scale data by using deep neural networks, especially convolutional neural networks (CNNs), so as to better solve the challenges faced by disguised target detection. The task of target detection based on deep learning methods is to locate and identify objects in images or videos, usually using bounding boxes to describe the location of the target. The task of semantic segmentation based on deep learning methods is to assign each pixel in the image to a predefined semantic category, so as to achieve pixel-level understanding of the image. With the powerful learning ability of deep learning, the research on disguised target detection and segmentation based on deep learning has also received increasing attention in the field of computer vision.

[0004] However, the relevant technologies for camouflaged target detection are still in a rapid development stage, and the relevant technologies are not mature. In order to solve the problem that the camouflaged target has similar color, texture or shape to the surrounding environment, resulting in the inability of traditional target detection methods based on manually designed features and traditional target detection methods to effectively detect the camouflaged target, many researchers have provided many solutions to this problem from methods such as bionics and large models. However, there is still a lack of practice in the field of camouflaged target detection based on target detection and segmentation multi-task collaboration. The present invention mainly focuses on how to complement the advantages of multiple tasks in camouflaged target detection, and uses segmentation information to assist the main task of target detection to mine more sufficient camouflaged target information to carry out work. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method for detecting camouflaged targets based on edge prior-guided pixel-level masks to assist detection. When the target has a high visual similarity with the background or the target is occluded, the traditional target detection network has the problem of uncertain and insufficient feature extraction of camouflaged or occluded targets.

[0006] The technical solution of the present invention is: a method for assisting camouflage target detection based on edge prior-guided pixel-level mask, the specific steps of which are:

[0007] Step 1: Fuse the 2nd and 5th layers of the 5 layers of feature maps of different scales extracted by the Resnet backbone network, and use the edge extraction module to extract the target edge features with global semantic information;

[0008] Step 2: The extracted target edge features are used as the input of the feature extraction module to guide the extraction of target key features in the feature maps of different sizes and scales from the 2nd to the 5th layers;

[0009] Step 3: Three context information aggregation modules are used to fuse target features of three different sizes and scales from high-level to low-level in the target key features, and three pixel-level masks of three different sizes and scales are obtained at the same time;

[0010] Step 4: Use a pixel-level mask with the same size and scale as the feature map of the second layer to enhance the target features of this layer, output the target category and the coordinates of the detection box, and optimize the model training to achieve camouflaged target detection.

[0011] The Step 1 is specifically as follows:

[0012] The Resnet backbone network is used to extract 5 layers of feature maps of different scales F. i(i=1, 2, 3, 4, 5), the 5th layer feature map F5 is subjected to 1×1 convolution and upsampling operations to obtain a feature map of the same size as the F2 feature map and concatenated with F2. The concatenated feature map is then subjected to 3×3 convolution, 1×1 convolution and Sigmoid function in sequence to obtain the target edge feature F with global semantic information. e .

[0013] The Step 2 is specifically as follows:

[0014] Step 2.1: Extract the target edge feature F e After downsampling and Sigmoid function, it is multiplied element-by-element with the F5 feature map and then added, and the primary features of the target are extracted using a 3×3 convolution operation;

[0015] Step 2.2: Use the local channel attention mechanism to interact across channels and mine the target key features between channels in the target primary features F5 a ;

[0016] Step 2.3: Target key feature map F5 a After upsampling, it is multiplied element-by-element with the feature map of the fourth layer and then added, and then undergoes a 3×3 convolution operation and a local channel attention mechanism to obtain F4. a Repeat this process to obtain the target key feature F corresponding to the i-th layer feature map i a (i=2, 3, 4, 5).

[0017] The Step 2.2 is specifically as follows:

[0018] Step 2.2.1: The target primary features are sequentially subjected to global average pooling, one-dimensional convolution and sigmoid function to obtain local attention weights;

[0019] Step 2.2.2: The local attention weight is multiplied by the target primary feature element by element and then subjected to a 3×3 convolution operation to obtain the target key feature F. i a (i=2, 3, 4, 5).

[0020] The context information aggregation module in Step 3 is specifically:

[0021] S1: Concatenate and fuse the two input feature maps, obtain the feature map W through 1×1 convolution, and divide it into four feature maps N according to the channel. j (j=1, 2, 3, 4).

[0022] S2: Add N1 and N2 element by element and then perform dilated convolution with a dilation rate of 1 to obtain feature map N1 d ;

[0023] S3: N2, N1 d After adding element-by-element to N3, the feature map N2 is obtained by dilation convolution with a dilation rate of 2. d ;

[0024] S4: N3, N2 d After adding element-by-element to N4, the feature map N3 is obtained by dilation convolution with a dilation rate of 3. d ;

[0025] S5: Connect N4 and N3 d After element-by-element addition, a dilated convolution with a dilation rate of 4 is performed to obtain feature map N4. d ;

[0026] S6: N1 d 、N2 d 、N3 d 、N4 d After concatenation and fusion and 1×1 convolution, the feature map W is added element by element and then convolved to obtain the feature map F. i c (i=2, 3, 4).

[0027] The Step 3 is specifically as follows:

[0028] Step 3.1: Press F5 a With F4 a F4 is obtained through the context information aggregation module c Feature map, F4 c With F3 a F3 is obtained through the context information aggregation module c Feature map, F3 c With F2 a F2 is obtained through the context information aggregation module c Feature map;

[0029] Step 3.2: F i c The (i=2, 3, 4) feature maps are processed by the Sigmoid function to obtain pixel-level masks of three different sizes and scales.

[0030] In Step 4, the pixel-level mask with the same size and scale as the feature map of the second layer is used to enhance the target features of this layer:

[0031] F2 c The pixel-level mask corresponding to the feature map and F2 a After element-by-element multiplication and addition of the feature map, the detection head composed of 3×3 convolution, pooling layer, and fully connected layer outputs the target category and the coordinates of the detection box.

[0032] The optimization model training in Step 4 is specifically as follows:

[0033] The target edge feature in Step 1 is passed through the Sigmoid function and then the loss L is calculated with the edge label. e , the pixel-level masks of three different sizes and scales in Step 3 are combined with the mask labels to calculate the loss L m , calculate the loss L by combining the target category and detection box coordinates in Step 4 with the target position label det , the total loss L is:

[0034] L=λL e +L m +L det

[0035] Among them, L e , L m , L det They represent edge, mask and target detection losses respectively, and λ is a weight parameter. Specifically, binary cross entropy loss and smoothL1 loss are used as classification loss and bounding box regression loss for target detection, weighted binary cross entropy loss and intersection-over-union loss are used as semantic segmentation loss for pixel-level masks of three different sizes and scales, and Dice coefficient loss is used to calculate edge loss. The three loss functions are as follows:

[0036] L det =L BCE (P c ,G c )+L smoothL1 (P b ,G b )

[0037]

[0038] L e =L dice (F e ,G e )

[0039] Among them, P c and G c Represent the predicted category and the true category respectively, P b and G b Represent the predicted bounding box and the target true bounding box respectively, Represents the semantic segmentation results of pixel-level masks of three different sizes and scales, G c represents the target true semantic segmentation label, F e and G e Represent the target predicted edge segmentation result and the target true edge segmentation label respectively. In addition, L BCE , L smoothL1, L IOU and L dice They represent binary cross entropy loss, smoothL1 loss, intersection-over-union loss, and Dice coefficient loss respectively.

[0040] Traditional target detection networks have difficulty in effectively separating camouflaged targets from the background when extracting them, which results in the model failing to learn effective visual features of camouflaged targets, resulting in low detection rate of the model, missed detection, false detection, etc. Starting from target features, the present invention uses target edge information to guide the extraction of effective target information, strengthens the model's perception of target edges, and uses segmentation mask attention with global semantic information to obtain effective features for target detection.

[0041] The beneficial effects of the present invention are as follows: compared with the prior art, the present invention mainly solves the problem of uncertain and insufficient target feature extraction in traditional target detection methods when the target and the background have a high visual similarity or the target is obscured, and can provide higher detection results for identifying camouflaged soldiers in the military field and identifying wild animals and plants in the biological field, and provides an efficient detection method for urban security and ecological protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the steps of the present invention;

[0043] Figure 2 It is a framework diagram of the feature extraction module in the present invention;

[0044] Figure 3 It is a framework diagram of the context information aggregation module in the present invention;

[0045] Figure 4 It is the visualization result of the present invention for detecting the camouflaged target. DETAILED DESCRIPTION

[0046] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation methods.

[0047] Example 1: Figure 1 As shown in FIG. 1 , a method for detecting camouflaged targets using pixel-level masks assisted by edge prior guidance is described. The specific steps are as follows:

[0048] Step 1: Fuse the 2nd and 5th layers of the 5 layers of feature maps of different scales extracted by the Resnet backbone network, and use the edge extraction module to extract the target edge features with global semantic information;

[0049] Step 2: The extracted target edge features are used as the input of the feature extraction module to guide the extraction of target key features in the feature maps of different sizes and scales from the 2nd to the 5th layers;

[0050] Step 3: Three context information aggregation modules are used to fuse target features of three different sizes and scales from high-level to low-level in the target key features, and three pixel-level masks of three different sizes and scales are obtained at the same time;

[0051] Step 4: Use a pixel-level mask with the same size and scale as the feature map of the second layer to enhance the target features of this layer, output the target category and the coordinates of the detection box, and optimize the model training to achieve camouflaged target detection.

[0052] The Step 1 is specifically as follows:

[0053] The Resnet backbone network is used to extract 5 layers of feature maps of different scales F. i (i=1, 2, 3, 4, 5), the 5th layer feature map F5 is subjected to 1×1 convolution and upsampling operations to obtain a feature map of the same size as the F2 feature map and concatenated with F2. The concatenated feature map is then subjected to 3×3 convolution, 1×1 convolution and Sigmoid function in sequence to obtain the target edge feature F with global semantic information. e .

[0054] The Step 2 is specifically as follows:

[0055] Step 2.1: Extract the target edge feature F e After downsampling and Sigmoid function, it is multiplied element-by-element with the F5 feature map and then added, and the primary features of the target are extracted using a 3×3 convolution operation;

[0056] Step 2.2: Use the local channel attention mechanism to interact across channels and mine the target key features between channels in the target primary features F5 a ;

[0057] Step 2.3: Target key feature map F5 a After upsampling, it is multiplied element-by-element with the feature map of the fourth layer and then added, and then undergoes a 3×3 convolution operation and a local channel attention mechanism to obtain F4. a Repeat this process to obtain the target key feature F corresponding to the i-th layer feature map i a (i=2, 3, 4, 5).

[0058] The Step 2.2 is specifically as follows:

[0059] Step 2.2.1: The target primary features are sequentially subjected to global average pooling, one-dimensional convolution and sigmoid function to obtain local attention weights;

[0060] Step 2.2.2: The local attention weight is multiplied by the target primary feature element by element and then subjected to a 3×3 convolution operation to obtain the target key feature F. i a (i=2, 3, 4, 5).

[0061] The context information aggregation module in Step 3 is specifically:

[0062] S1: Concatenate and fuse the two input feature maps, obtain the feature map W through 1×1 convolution, and divide it into four feature maps N according to the channel. j (j=1, 2, 3, 4).

[0063] S2: Add N1 and N2 element by element and then perform dilated convolution with a dilation rate of 1 to obtain feature map N1 d ;

[0064] S3: N2, N1 d After adding element-by-element to N3, the feature map N2 is obtained by dilation convolution with a dilation rate of 2. d ;

[0065] S4: N3, N2 d After adding element-by-element to N4, the feature map N3 is obtained by dilation convolution with a dilation rate of 3. d ;

[0066] S5: Connect N4 and N3 d After element-by-element addition, a dilated convolution with a dilation rate of 4 is performed to obtain feature map N4. d ;

[0067] S6: N1 d 、N2 d 、N3 d 、N4 d After concatenation and fusion and 1×1 convolution, the feature map W is added element by element and then convolved to obtain the feature map F. i c (i=2, 3, 4).

[0068] The Step 3 is specifically as follows:

[0069] Step 3.1: Press F5 a With F4 a F4 is obtained through the context information aggregation module c Feature map, F4 c With F3 a F3 is obtained through the context information aggregation module c Feature map, F3 c With F2 a F2 is obtained through the context information aggregation module c Feature map;

[0070] Step 3.2: F i c The (i=2, 3, 4) feature maps are subjected to the Sigmoid function to obtain three pixel-level masks of different sizes and scales. In Step 4, the pixel-level mask with the same size and scale as the feature map of the second layer is used to enhance the target features of this layer:

[0071] F2 c The pixel-level mask corresponding to the feature map and F2 a After element-by-element multiplication and addition of the feature map, the detection head composed of 3×3 convolution, pooling layer, and fully connected layer outputs the target category and the coordinates of the detection box.

[0072] The optimization model training in Step 4 is specifically as follows:

[0073] The target edge feature in Step 1 is passed through the Sigmoid function and then the loss L is calculated with the edge label. e , the pixel-level masks of three different sizes and scales in Step 3 are combined with the mask labels to calculate the loss L m , calculate the loss L by combining the target category and detection box coordinates in Step 4 with the target position label det , the total loss L is:

[0074] L=λL e +L m +L det

[0075] Among them, L e , L m , L det They represent edge, mask and target detection losses respectively, and the λ weight parameter is set to 5. Specifically, binary cross entropy loss and smoothL1 loss are used as classification loss and bounding box regression loss for target detection, weighted binary cross entropy loss and intersection-over-union loss are used as semantic segmentation loss for pixel-level masks of three different sizes and scales, and Dice coefficient loss is used to calculate edge loss. The three loss functions are as follows:

[0076] L det =L BCE (P c ,G c )+L smoothL1 (P b ,G b )

[0077]

[0078] L e =L dice (Fe ,G e )

[0079] Among them, P c and G c Represent the predicted category and the true category respectively, P b and G b Represent the predicted bounding box and the target true bounding box respectively, Represents the semantic segmentation results of pixel-level masks of three different sizes and scales, G c represents the target true semantic segmentation label, F e and G e Represent the target predicted edge segmentation result and the target true edge segmentation label respectively. In addition, L BCE , L smoothL1 , L IOU and L dice They represent binary cross entropy loss, smoothL1 loss, intersection-over-union loss, and Dice coefficient loss respectively.

[0080] The effectiveness of the technical solution of the present invention is illustrated below by way of experiments based on the specific implementation records:

[0081] 1. Experimental data

[0082] Grayscale-polarization camouflaged target detection dataset: The camouflaged objects in this dataset are soldiers wearing camouflage uniforms and camouflaged into the background. The grayscale-polarization images can highlight the edge information of the target while reducing the color and texture differences between the target and the background, increasing the difficulty of target detection. The dataset includes 2,313 training images and 580 test images. The images cover targets of different scales and various postures, such as standing, prone, kneeling, etc. In addition, the image background has different geographical environments, lighting conditions, and occlusion conditions.

[0083] 2. Experimental content

[0084] The training data of all grayscale-polarization camouflaged target detection datasets are used for model training, the model performance is verified on the test set, and the target detection results are evaluated using the evaluation indicators in the following formula. Table 1 shows the precision, recall, and F score of the method of the present invention on the test set of grayscale-polarization camouflaged target detection.

[0085] precision=TP / (TP+FP)

[0086] recall=TP / (TP+FN)

[0087]

[0088] Among them, TP is the number of samples correctly identified as target instances by the model, FP is the number of samples incorrectly predicted as target instances by the model, and FN is the number of samples of target instances missed by the model.

[0089] Table 1

[0090]

[0091] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A method for detecting camouflaged targets using pixel-level masks assisted by edge prior guidance, characterized in that: Step 1: Fuse the 2nd and 5th layers of the 5 layers of feature maps of different scales extracted by the Resnet backbone network, and use the edge extraction module to extract the target edge features with global semantic information; Step 2: The extracted target edge features are used as the input of the feature extraction module to guide the extraction of target key features in the feature maps of different sizes and scales from the 2nd to the 5th layers; Step 3: Three context information aggregation modules are used to fuse target features of three different sizes and scales from high-level to low-level in the target key features, and three pixel-level masks of three different sizes and scales are obtained at the same time; Step 4: Use a pixel-level mask with the same size and scale as the feature map of the second layer to enhance the target features of this layer, output the target category and the coordinates of the detection box, and optimize the model training to achieve camouflaged target detection.

2. The method for detecting camouflaged targets based on edge prior guided pixel-level masks according to claim 1 is characterized in that: The Step 1 is specifically as follows: The Resnet backbone network is used to extract 5 layers of feature maps of different scales F. i (i=1, 2, 3, 4, 5), the 5th layer feature map F5 is subjected to 1×1 convolution and upsampling operations to obtain a feature map of the same size as the F2 feature map and concatenated with F2. The concatenated feature map is then subjected to 3×3 convolution, 1×1 convolution and Sigmoid function in sequence to obtain the target edge feature F with global semantic information. e .

3. The method for detecting camouflaged targets based on edge prior guided pixel-level masks according to claim 1, characterized in that: The Step 2 is specifically as follows: Step 2.1: Extract the target edge feature F e After downsampling and Sigmoid function, it is multiplied element-by-element with the F5 feature map and then added, and the primary features of the target are extracted using a 3×3 convolution operation; Step 2.2: Use the local channel attention mechanism to interact across channels and mine the target key features between channels in the target primary features F5 a ; Step 2.3: Set the target key feature F5 a After upsampling, it is multiplied element-by-element with the feature map of the fourth layer and then added, and then undergoes a 3×3 convolution operation and a local channel attention mechanism to obtain F4. a Repeat this process to obtain the target key feature F corresponding to the i-th layer feature map i a (i=2, 3, 4, 5).

4. The method for detecting camouflaged targets based on edge prior guided pixel-level masks according to claim 3 is characterized in that: The Step 2.2 is specifically as follows: Step 2.2.1: The target primary features are sequentially subjected to global average pooling, one-dimensional convolution and sigmoid function to obtain local attention weights; Step 2.2.2: The local attention weight is multiplied by the target primary feature element by element and then subjected to a 3×3 convolution operation to obtain the target key feature F. i a (i=2, 3, 4, 5).

5. The method for detecting camouflaged targets based on edge prior guided pixel-level masks according to claim 1, characterized in that: The context information aggregation module is specifically: S1: Concatenate and fuse the two input feature maps, obtain the feature map W through 1×1 convolution, and divide it into four feature maps N according to the channel. j (j=1, 2, 3, 4); S2: Add N1 and N2 element by element and then perform dilated convolution with a dilation rate of 1 to obtain feature map N1 d ; S3: N2, N1 d After adding element-by-element to N3, the feature map N2 is obtained by dilation convolution with a dilation rate of 2. d ; S4: N3, N2 d After adding element-by-element to N4, the feature map N3 is obtained by dilation convolution with a dilation rate of 3. d ; S5: Connect N4 and N3 d After element-by-element addition, a dilated convolution with a dilation rate of 4 is performed to obtain feature map N4. d ; S6: N1 d 、N2 d 、N3 d 、N4 d After concatenation and fusion and 1×1 convolution, the feature map W is added element by element and then convolved to obtain the feature map F. i c (i=2, 3, 4).

6. The method for edge prior-guided pixel-level mask-assisted camouflage target detection according to claim 1, characterized in that: The Step 3 is specifically as follows: Step 3.1: Press F5 a With F4 a F4 is obtained through the context information aggregation module c Feature map, F4 c With F3 a F3 is obtained through the context information aggregation module c Feature map, F3 c With F2 a F2 is obtained through the context information aggregation module c Feature map; Step 3.2: F i c The (i=2, 3, 4) feature maps are processed by the Sigmoid function to obtain pixel-level masks of three different sizes and scales.

7. The method for detecting camouflaged targets based on edge prior guided pixel-level masks according to claim 1, characterized in that: The method of enhancing the target features of the layer with a pixel-level mask of the same size and scale as the feature map of the second layer is specifically as follows: F2 c The pixel-level mask corresponding to the feature map and F2 a After element-by-element multiplication and addition of the feature map, the detection head composed of 3×3 convolution, pooling layer, and fully connected layer outputs the target category and the coordinates of the detection box.

8. The method for edge prior-guided pixel-level mask-assisted camouflage target detection according to claim 1, characterized in that: The optimization model training is specifically as follows: The target edge feature in Step 1 is passed through the Sigmoid function and then the loss L is calculated with the edge label. e , the pixel-level masks of three different sizes and scales in Step 3 are combined with the mask labels to calculate the loss L m , calculate the loss L by combining the target category and detection box coordinates in Step 4 with the target position label det , the total loss L is: L=λL e +L m +L det Among them, L e , L m , L det They represent edge, mask and target detection losses respectively, and λ is a weight parameter. Specifically, binary cross entropy loss and smoothL1 loss are used as classification loss and bounding box regression loss for target detection, weighted binary cross entropy loss and intersection-over-union loss are used as semantic segmentation loss for pixel-level masks of three different sizes and scales, and Dice coefficient loss is used to calculate edge loss. The three loss functions are as follows: L det =L BCE (P c ,G c )+L smoothL1 (P b ,G b ) L e =L dice (F e ,G e ) Among them, P c and G c Represent the predicted category and the true category respectively, P b and G b Represent the predicted bounding box and the target true bounding box respectively, Represents the semantic segmentation results of pixel-level masks of three different sizes and scales, G c represents the target true semantic segmentation label, F e and G e Represent the target predicted edge segmentation result and the target true edge segmentation label respectively. In addition, L BCE , L smoothL1 , L IOU and L dice They represent binary cross entropy loss, smoothL1 loss, intersection-over-union loss, and Dice coefficient loss respectively.

Images