Camouflage target detection method based on multi-dimensional polarization information

Abstract

The invention discloses a camouflage target detection method based on multi-dimensional polarization information. The method comprises the following steps: 1, acquiring a polarization image data set with pixel-level labels; 2, processing the four original polarization angle images, learning the exclusive weight of the feature of each image through a polarization dynamic attention module, extracting polarization information and structure details at the same time, and fusing the polarization information and the structure details with the original images; and 3, constructing a deep convolutional neural network based on multi-dimensional polarization information, inputting the fused image, and training the deep convolutional network to obtain an accurate camouflage target detection result. Under a reasonable data model condition, scene representation with rich textures and rich edge details is generated by utilizing correlation and difference of group polarization images, so that the problem that the detection effect of a camouflage target is influenced due to the fact that DoLP may be very weak or does not exist under certain illumination conditions can be effectively solved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, image processing and analysis, and specifically is a camouflaged target detection method based on multi-dimensional polarization information. Background Art

[0002] Camouflage is a very common biological phenomenon in nature. Animals will hide themselves in the surrounding environment by changing their colors and shapes to avoid being discovered by predators. In addition to the natural camouflage of animals, there are also artificial camouflage phenomena in real life, such as soldiers wearing camouflage uniforms in military activities and body painting art. Chameleons that change their colors according to their habitats, seahorses with the same color and shape as the background environment, soldiers who achieve camouflage through clothing and coloring, and body art paintings that blend into the background; Obviously, the salient feature of this type of image is that the visual features of the target area are highly similar to the background area, while the current mainstream computer vision detection algorithms, such as the salient target detection algorithm and the general target detection algorithm, have a large majority of detection objects that are significantly different from the surrounding environment. Therefore, when facing images with camouflage features, the detection results often have problems such as missed detection and false detection of targets, which is not conducive to the further development of related target detection and recognition systems. In this context, camouflaged target detection algorithms came into being, dedicated to detecting objects that are "perfectly" integrated into the surrounding environment. They have high visual feature recognition capabilities and can effectively improve the overall performance of the target detection system. It is a very challenging and indispensable visual task.

[0003] With the rapid development of computer vision task algorithms, more and more researchers have begun to pay attention to the research direction of camouflaged target detection, especially the emerging camouflaged target detection algorithm based on deep learning, which has achieved quite rich research results in recent years. The practical application value of camouflaged target detection is also increasingly prominent. In the medical field, most lesion areas are highly similar to the surrounding normal tissue areas; polyp segmentation is a typical example, and the boundary between polyps and their surrounding mucosa is usually unclear. Therefore, polyps at the lesion can also be regarded as a camouflaged object, and camouflaged target detection can be used to detect and segment polyps. In the military field, the enemy usually camouflages itself and its equipment according to the characteristics of the environment to achieve the purpose of hiding troops, and camouflaged target detection can detect camouflaged enemies and military equipment from the background, thereby ensuring the smooth completion of military reconnaissance missions. In the field of agronomy, camouflaged target detection can be used to detect camouflaged pests from similar backgrounds, assisting intelligent pest monitoring systems to improve the level and effect of pest monitoring and early warning. In addition, by utilizing the powerful visual feature resolution ability of the camouflaged target detection model, it can also be used to detect and protect wild animals, search and rescue people in need of help in the wild, and some scenes that have yet to be explored.

[0004] At present, the camouflaged target detection methods can be simply divided into traditional methods based on manually designed features and learning-based methods. Camouflaged target detection based on traditional methods mainly focuses on the underlying features of the image, such as color, texture, gradient and other prior information, and performs camouflaged target detection by manually extracting features. Usually, this type of manual extraction method requires a lot of prior knowledge, is often time-consuming and labor-intensive, and is difficult to apply to large-scale data. It was the mainstream method for camouflaged target detection before the emergence of deep learning models. Camouflaged target detection technology based on traditional methods is generally effective, but detection algorithms based on a single feature are difficult to apply to all scenarios, and detection algorithms based on multiple features are difficult to implement. When the texture is similar to the background, color features play a role in identifying camouflage. When the camouflage is due to the similarity of color, texture features are effective. When the motion state of the camouflaged object is good, the motion state can be used for detection.

[0005] With the continuous development of deep neural network technology, researchers are no longer limited to the traditional method of extracting manual features to identify camouflaged targets. Due to the powerful learning ability of deep neural networks, they can adaptively extract potential connections and semantic expressions from massive samples, reducing the complex process of manual feature extraction. The ability to detect camouflaged targets has therefore reached a new level, and its accuracy and real-time performance have been significantly improved. Researchers have constructed different excellent deep neural network models from different perspectives such as designing novel feature fusion methods, improved network structures, attention mechanisms, data set expansion, and joint learning tasks. It can be seen that the key to camouflaged target detection technology lies in: sufficient training samples, excellent feature extraction networks, efficient feature fusion technology, and interference noise removal methods, which are necessary for excellent camouflaged target detection models. However, the data targeted by current learning-based methods are almost all traditional intensity images, which can only use external feature information in the scene. In challenging scenes where the external feature differences between the target and the background are small, these methods are difficult to accurately segment the target. Traditional methods often highlight the camouflaged area by mining the intrinsic feature differences between the target and the environment, but the detection efficiency and accuracy need to be improved. Summary of the invention

[0006] In order to solve the deficiencies in the prior art, the present invention provides a camouflaged target detection method based on multi-dimensional polarization information, so as to effectively detect camouflaged targets in complex scenes, thereby improving the precision and accuracy of camouflaged target detection in complex and changeable environments, and effectively overcoming the problem that under certain lighting conditions, the polarization angle is weak or non-existent, which affects the detection effect of the camouflaged target.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] The camouflaged target detection method based on multi-dimensional polarization information of the present invention is characterized in that it is performed according to the following steps:

[0009] Step 1: Data collection and processing;

[0010] Step 1.1: Use a polarization camera to capture the relative polarization angle in the nth scene They are , , , A set of original polarization images , thus obtaining N groups of original polarization images under N scenes; among them, Indicates the relative polarization angle in the nth scene The original polarization image under ;

[0011] Step 1.2: For each set of original polarization images under N scenes, the polarization angle is The original polarization image is annotated to obtain a pixel-level annotated image, which is used as a real camouflage image;

[0012] Step 2: Construct a deep convolutional neural network for camouflage target detection based on multi-dimensional polarization information, including: encoding module and decoding module;

[0013] Step 2.1, the encoder After processing, the structural feature map of the nth scene and the polarization fusion feature of the nth scene are obtained;

[0014] Step 2.2: After the decoder processes the structural feature map and polarization fusion features of the nth scene, the nth scene is obtained. The predicted camouflage map F of the scene n ;

[0015] Step 3: training a camouflaged target detection model based on multi-dimensional polarization information;

[0016] Step 3.1: Based on The real camouflage image and the predicted camouflage image F of the scene n Construct binary cross entropy loss;

[0017] Step 3.2: Based on The real camouflage image and the predicted camouflage image F of the scene n The intersection and union construction of loss;

[0018] Step 3.3: Binary cross entropy loss and After the losses are weighted, the total loss is obtained;

[0019] Step 3.4: Use the gradient descent method to train the deep convolutional neural network for camouflaged target detection, and calculate the total loss to update the network parameters until the total loss converges, thereby obtaining a camouflaged target detection model based on multi-dimensional polarization information, which is used to realize camouflaged target detection for any multi-dimensional polarization image to be predicted.

[0020] The camouflaged target detection method based on multi-dimensional polarization information of the present invention is also characterized in that: the encoder in step 2.1 includes: a polarization dynamic attention module, a Res2Net backbone network, and H receptive field modules;

[0021] Step 2.1.1, polarized attention module After processing, the nth polarization-enhanced dimensional information image D is obtained. n ;

[0022] Step 2.1.2, the Res2Net backbone network is composed of H-level initial convolutional layers, H-level downsampling convolutional layers and H-level residual blocks;

[0023] Among them, the h-th level initial convolution layer includes: a convolution layer, a BN layer and a ReLU activation function layer in sequence;

[0024] when When D n After being processed by the h-th level initial convolution layer, the h-th level downsampling convolution layer and the h-th level residual block, the h-th level structural feature map is output. ;

[0025] when When Level structure feature map Input to the hth level initial convolutional layer for processing, and then pass through the hth level initial convolutional layer in sequence. The downsampling convolutional layer and the After processing the level-h residual block, the h-th level structural feature map is obtained. , so that the H-th level residual block outputs the H-th level structural feature map ;

[0026] Step 2.1.3, construct each receptive field module including K branches, a standard convolutional layer and a ReLU layer;

[0027] when When the h-th level structural feature graph Input into the hth receptive field module and processed by the 2D convolution of the kth branch, the texture feature of the kth branch output by the hth receptive field module is obtained. ;

[0028] when When the h-th level structural feature graph After inputting several 2D convolutional layers of the kth branch in the hth receptive field module for processing, the texture features of the kth branch output by the hth receptive field module are obtained. ;

[0029] The texture features of the K branches output by the h-th receptive field module After cascading, the hth detail feature is output through the processing of the standard convolution layer ;

[0030] The second branch feature output by the hth receptive field module and After addition, the data is input into the ReLU layer for processing, and finally the polarization fusion feature of the nth scene is output by the hth receptive field module. ; Thus, the H receptive field modules output the H polarization fusion features of a scene .

[0031] Furthermore, the polarized attention module in step 2.1.1 includes: several 3D convolutional layers, an adaptive average pooling layer and a sigmoid function;

[0032] 3D convolutional layer pairs After 3D convolution processing, the difference features between the polarization angle images in the nth scene are obtained ;

[0033] After being processed by the adaptive average pooling layer and the sigmoid function, the weighted features of the group polarization image in the nth scene are obtained. ;

[0034] right and After performing element-by-element multiplication, we get Relative polarization angle in each scene Angle characteristics below , and then After channel cascade operation, the nth polarization-enhanced dimensional information image D is obtained. n .

[0035] Furthermore, the decoder in step 2.2 includes: a feature edge fusion module, a high and low gating selection module;

[0036] Step 2.2.1, the feature edge fusion module is composed of H fusion blocks, each of which includes several convolutional layers and an average pooling layer; and As input, it is sent to the feature edge fusion module for processing to obtain the edge fusion feature ;

[0037] when When With The first scene Level polarization fusion feature After channel cascading, The input is sent to the hth fusion block for processing to obtain the hth edge feature map ;

[0038] when When Edge feature map and As an input, it is sent to the hth fusion block for processing to obtain the hth edge feature map , so that the Hth fusion block outputs the Hth edge feature map ;

[0039] Step 2.2.2 The high and low gated selection module contains a gated selection block and a residual attention block;

[0040] No. The first-level polarization fusion features of the scene and The input gate selection block is processed to obtain the Cross-layer features ;

[0041] Will After being processed in the residual attention block, the output The predicted camouflage map F of the scene n .

[0042] Furthermore, the gate selection block in step 2.2.2 first selects After the standard convolution operation, batch normalization and ReLU function processing are performed in sequence, the first Refinement Features ;

[0043] right After the transposed convolution operation, the upsampling operation is performed to obtain the first Perceptual features ;

[0044] right and After cascading on the channel, we get Gated Map ;

[0045] right After the transposed convolution operation, After cascading in the channel dimension, we get Cross-layer features .

[0046] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the camouflaged target detection method, and the processor is configured to execute the program stored in the memory.

[0047] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the camouflaged target detection method when the computer program is executed by a processor.

[0048] Compared with the prior art, the beneficial effects of the present invention are:

[0049] 1. The present invention constructs a deep neural network for camouflaged target detection based on multi-dimensional polarization information, and uses labeled data to supervise the deep neural network for learning, so as to obtain a robust polarization camouflaged target detection feature model. It solves the problem of using clues such as color, depth, and background priors in statistical models to design models while ignoring the intrinsic feature information of the scene, and the problem of low detection accuracy in scenes with small external differences.

[0050] 2. Due to the relative differences in the refraction and reflection characteristics of light by different materials, surface roughness and structures, the polarization state of light will change. The camouflaged target detection model based on multi-dimensional polarization information constructed by the present invention distinguishes the reflection source in the scene according to the change of polarization state, and by introducing the intrinsic polarization information characteristics in the scene, it can provide richer information for scene understanding and provide more basis for distinguishing targets, so that the camouflaged target and the background can be well separated.

[0051] 3. The deep neural network based on multi-dimensional polarization information constructed by the present invention takes the multi-dimensional polarization image as the network input, uses the polarization dynamic attention module to extract the polarization information and structural details, and makes full use of the polarization information in the scene; thereby solving the problem that most camouflage target methods based on deep learning are difficult to accurately segment the target in scenes where the external features of the target and the background are slightly different.

[0052] 4. The present invention uses multi-level fusion and different aggregation strategies to effectively explore low-level and high-level functions. The feature edge fusion module fuses boundary information with high-level features, allowing the network to focus on global and contextual information at a high level to accurately locate the camouflaged object. In addition, by designing a high-low gate selection module, high-level features and low-level features are effectively fused, while filtering out redundant information, reducing the network burden, and merging useful information, thereby obtaining a more accurate camouflaged target prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Flowchart for camouflaged target detection based on camouflaged target model with multi-dimensional polarization information;

[0054] Figure 2 Schematic diagram of the structure of the deep neural network for polarization images with multi-dimensional polarization information;

[0055] Figure 3 Schematic diagram of a single receptive field module;

[0056] Figure 4 This is a diagram of camouflaged target prediction results on a polarization data set using the method of the present invention and other camouflaged target detection methods. DETAILED DESCRIPTION

[0057] In this embodiment, a camouflaged target detection method based on multi-dimensional polarization information is designed to locate an "indistinguishable target" with a highly similar background. Polarization information is introduced to expand the difference between the object and its surrounding environment in the camouflaged target detection task, and a CNN-based deep neural network is constructed to process polarized images. By constructing a more sophisticated camouflaged target detection model based on multi-dimensional polarization, the intrinsic feature differences between the target and the background can be fully utilized to better complete the camouflaged target detection task. Specifically, Figure 1 As shown, the method is performed in the following steps:

[0058] Step 1: Data collection and processing;

[0059] Step 1.1: Use a polarization camera to capture the relative polarization angle in the nth scene They are , , , A set of original polarization images , thus obtaining N groups of original polarization images under N scenes; among them, Indicates the relative polarization angle in the nth scene The original polarization image under ;

[0060] In this embodiment, a Lucid Triton focal plane polarization camera is used to shoot a polarization camouflage target detection dataset, which contains The original resolution of each image is 1024×1024.

[0061] Step 1.2: For each set of original polarization images under N scenes, the polarization angle is The original polarization image is annotated to obtain a pixel-level annotated image, which is used as a real camouflage image;

[0062] In this embodiment, the labeling is performed by using Labelme software. The labeled image is to assign a category label V to each pixel in the polarization image. , respectively, represent the color of the pixel: black, white; black represents the background of the pixel, and white represents the target. The polarization camouflage target detection dataset is divided into training and testing, where the training set contains 511 scenes and the test set contains 128 scenes.

[0063] Step 2: Construct a deep convolutional neural network for camouflage target detection based on multi-dimensional polarization information, including: encoding module and decoding module;

[0064] Step 2.1, build the encoder, including: polarized dynamic attention module, Res2Net backbone network, H receptive field modules;

[0065] Step 2.1.1, the polarized attention module includes: several 3D convolutional layers, a 3D global average pooling layer and a sigmoid function;

[0066] After being processed by the polarization attention module, the nth polarization-enhanced dimensional information image D is obtained. n ;

[0067] In this embodiment, the 3D convolution layer After 3D convolution processing, the difference features between the polarization angle images in the nth scene are obtained ;

[0068] After being processed by the adaptive average pooling layer and the sigmoid function, the weighted features of the group polarization image in the nth scene are obtained. ;

[0069] right and After performing element-by-element multiplication, the relative polarization angle in the nth scene is obtained. Angle characteristics below , and then After channel cascade operation, the nth polarization-enhanced dimensional information image D is obtained. n .

[0070] In the specific implementation, in order to balance the training efficiency and accuracy of the model, The size of the four original polarization images is downsampled to 416×416,

[0071] In this embodiment, the number of samples in the batch processing is set to 12, the number of channels of each sample is 3, and the height and width are both 416, that is, each batch of training input is 48 multi-dimensional polarization images of size 416×416, which is equivalent to 4 groups of original polarization images of size 352×352. Finally, the size of the multi-dimensional polarization image input to the polarization dynamic attention module for each batch of training is 12×3×416×416, and the feature size output by the polarization dynamic attention module is 12×3×352×352.

[0072] Step 2.1.2, the Res2Net backbone network is composed of H-level initial convolutional layers, H-level downsampling convolutional layers and H-level residual blocks;

[0073] Among them, the h-th level initial convolution layer includes: a convolution layer, a BN layer and a ReLU activation function layer in sequence;

[0074] when When D n After being processed by the h-th level initial convolution layer, the h-th level downsampling convolution layer and the h-th level residual block, the h-th level structural feature map is output. ;

[0075] In this embodiment, the first level structural feature map The width, height, and number of channels are 352, 352, and 256.

[0076] when When Level structure feature map Input to the hth level initial convolutional layer for processing, and then pass through the hth level initial convolutional layer in sequence. The downsampling convolutional layer and the After processing the level-h residual block, the h-th level structural feature map is obtained. ; Thus, the H-th level residual block outputs the H-th level structural feature map ;

[0077] In this embodiment, the second level structural feature map The width, height, and number of channels are 176, 176, and 512; the third-level structural feature map The width, height, and number of channels are 88, 88, and 1024; the fourth-level structural feature map The width, height, and number of channels are 44, 44, and 2048.

[0078] Step 2.1.3, construct each receptive field module including K branches, a standard convolutional layer and a ReLU layer;

[0079] when When the h-th level structural feature graph Input into the hth receptive field module and processed by the 2D convolution of the kth branch, the texture feature of the kth branch output by the hth receptive field module is obtained. ;

[0080] when When the h-th level structural feature graph After inputting several 2D convolutional layers of the kth branch in the hth receptive field module for processing, the texture features of the kth branch output by the hth receptive field module are obtained. ;

[0081] The texture features of the K branches output by the h-th receptive field module After cascading and processing through the standard convolutional layer, the hth detail feature is output ;

[0082] The second branch feature output by the hth receptive field module and After addition, the data is input into the ReLU layer for processing, and finally the polarization fusion feature of the nth scene is output by the hth receptive field module. ; Thus, the H receptive field modules output the H polarization fusion features of a scene ;

[0083] In this example, K=5. The input of the h-th receptive field module is the h-th level structure feature map .

[0084] like Figure 3 As shown, when k=1,2, the h-th level structural feature map Input a 1×1 standard 2D convolution (consisting of a 2D convolution layer and a BN layer) of the kth branch in the hth receptive field module for processing. The receptive field module The branches and The texture feature of each branch is and ;

[0085] when When the h-th level structural feature graph Input the hth receptive field module The branches are processed by several standard convolutional layers to obtain the The receptive field module Texture features output by each branch . No. The branch consists of a convolution kernel of size A standard convolutional layer with a dilation rate of of The standard convolutional layer is used. Convolutional layers and Combination of convolutional layers instead Convolutional layer to reduce computational overhead. The output texture features are recorded as ;

[0086] Will After cascading, a size of The standard convolution layer outputs the hth detail feature ; Texture features of the output of the second branch of the receptive field module and After adding, it is input into a ReLU layer for processing, and finally The receptive field module outputs A polarization fusion feature for each scene

[0087] The input of the first receptive field module is the dimensional information image D n And the structural feature map output by the first layer of the backbone network After 2 times downsampling, the two are cascaded to obtain the polarization fusion feature with a final output size of 352×352×32. ; The input of the second receptive field module is the structural feature map of the second layer of the backbone network The final output size is 176×176×32 polarization fusion features ; The input of the third receptive field module is the structural feature map of the third layer of the backbone network The final output size is 88×88×32 polarization fusion features ; The input of the fourth receptive field module is the structural feature map of the fourth layer of the backbone network The final output size is 44×44×32 polarization fusion features .

[0088] Step 2.2, construct a decoder, including: feature edge fusion module, high and low gating selection module;

[0089] Step 2.2.1, the feature edge fusion module is composed of H fusion blocks, each of which includes several convolutional layers and an average pooling layer; and As input, it is sent to the feature edge fusion module to obtain the edge fusion feature ;

[0090] when When With The first scene Level polarization fusion feature After channel cascading, The input is sent to the hth fusion block for processing to obtain the hth edge feature map ;

[0091] when When Edge feature map and As an input, it is sent to the hth fusion block for processing to obtain the hth edge feature map ; Thus, the Hth fusion block outputs the Hth edge feature map ;

[0092] Step 2.2.2 The high and low gated selection module contains a gated selection block and a residual attention block;

[0093] No. The first-level polarization fusion features of the scene and The input gate selection block is processed to obtain the Cross-layer features ; In this embodiment, the first level polarization fusion feature The number of channels is 32, and the edge fusion feature The number of channels is 32;

[0094] In the specific implementation, the gate selection block first After the standard convolution operation, batch normalization and ReLU function processing are performed in sequence, the first Refinement Features ; In this embodiment, the refinement feature The dimensions are 44×44×192;

[0095] right After the transposed convolution operation, the upsampling operation is performed to obtain the first Perceptual features In this embodiment, the perception feature The dimensions are 88×88×1;

[0096] right and After channel cascading, we get Gated Map In this embodiment, the gate mapping G n

[0097] The dimensions are 88×88×32;

[0098] right Perform a transposed convolution operation and then After cascading in the channel dimension, we get Cross-layer features In this embodiment, the features are upsampled to the original size of 352×352;

[0099] Will After being processed in the residual attention block, the output The predicted camouflage map F of the scene n ;

[0100] Step 3: training a camouflaged target detection model based on multi-dimensional polarization information;

[0101] Step 3.1: Based on The binary cross entropy loss is constructed by comparing the real camouflage image and the predicted camouflage image of each scene;

[0102] Step 3.2: Based on The intersection and union of the real camouflage map and the predicted camouflage map of each scene are constructed loss;

[0103] Step 3.3: Binary cross entropy loss and After the losses are weighted, the total loss is obtained;

[0104] Step 3.4: Use the gradient descent method to train the deep convolutional neural network for camouflaged target detection, and calculate the total loss to update the network parameters until the total loss converges, thereby obtaining a camouflaged target detection model based on multi-dimensional polarization information, which is used to realize camouflaged target detection for any multi-dimensional polarization image to be predicted.

[0105] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0106] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

[0107] In this embodiment, the multi-dimensional polarization images of 2044 scenes after the polarization data set data enhancement and their corresponding real camouflage images are used for training, and the F output by the output module is n Weighted binary cross entropy and weighted The loss calculation results in 5 training losses, which are added together to get a total loss. This total loss is combined with the gradient descent algorithm to guide the network training, and a multi-dimensional network camouflage target detection feature model using polarization images is obtained. The multi-dimensional polarization scene images of 128 test scenes in the polarization camouflage target detection dataset are used as input. The camouflage target monitoring model based on multi-dimensional polarization information calculates the predicted camouflage target image, which is then compared with the real camouflage target image of the corresponding scene to calculate the detection accuracy.

[0108] Table 1 shows the comparison results of the camouflage target detection method based on multi-dimensional polarization information of the present invention with other current camouflage target detection methods using the test set of the polarization camouflage target detection data set, using "S-measure", "E-measure", "MAE" and "F-measure" as evaluation indicators respectively. "S-measure" can be used to reflect structural characteristics and can be used to evaluate the structural similarity between the prediction result and the real camouflage target. The closer its value is to 1, the better the camouflage target detection effect is. "E-measure" evaluates the overall and local accuracy and quality of the prediction results in the binary segmentation task by combining pixel-level matching with image-level statistics. The closer its value is to 1, the better the camouflage target detection effect is. "F-measure" is a weighted harmonic mean based on precision and recall. The closer its value is to 1, the better the camouflage target detection effect is. "MAE" is an evaluation indicator widely used in salient target detection. It is used for the pixel-by-pixel difference between the binary prediction result and the true value image. The closer its value is to 0, the better the camouflage target detection effect is. "BASNet" stands for Boundary-Aware Segmentation Network; "PraNet" stands for Parallel Reverse Attention Network; "MFFN" stands for Multi-view Feature Fusion Network; SINet-V2 stands for the second version of Search Identification Network; "ZoomNet" comes from the paper Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object Detection; "C 2"FNet-V2" comes from the paper CamouflagedObject Detection via Context-aware Cross-level Fusion; "BGNet" stands for Boundary-Guided Network. According to the quantitative analysis in Table 1, it can be seen that the method proposed in this invention has achieved good results in all evaluation indicators.

[0109] Table 1

[0110]

[0111] Figure 4The results of the multi-dimensional camouflaged target detection method designed for this paper and other current camouflaged target detection methods. Among them, "POL4Net" stands for Polarization four-dimensional network, which is a polarization image camouflage target detection method based on a multi-dimensional network of the present invention; "PGSNet" stands for Polarization Glass Segmentation Network, which proposes a new learning-based glass segmentation network, which utilizes the three-color (RGB) intensity and three-color linear polarization cues of a single photo, uses a novel global guidance and multi-scale self-attention module to dynamically fuse and weight the three-color and polarization cues, and uses global cross-domain context information to achieve robust segmentation; "CMX" stands for Cross-Modal Fusion for RGB-X, which is a unified fusion framework that uses the features of one modality to correct the features of another modality to calibrate the bimodal features, deploys a feature fusion module by correcting the feature pairs, and performs a full exchange of remote context before mixing; "PopNet" stands for Popping Network, which uses the prior knowledge of objects in 3D to pop-out and uses the deep reasoning model for object segmentation. This prior knowledge can assist in reasoning about objects in 3D space, and uses 3D information for positioning by adjusting the reasoned depth map; "DGNet" stands for Deep GradientNetwork, a new deep framework for camouflaged target detection using target gradient supervision - Deep Gradient Network, decouples the task into two connected branches, namely context and texture encoders; "FAPNet" stands for Feature Aggregation and Propagation Network, which is a new feature aggregation and propagation network that explicitly models boundary features through boundary guidance modules to provide boundary enhancement features, characterizes the multi-scale information of each layer through multi-scale feature aggregation modules, and obtains aggregated feature representations, and integrates the features of adjacent layers using cross-layer fusion and propagation modules; "MFFN" stands for Multi-view Feature Fusion Network, which is a multi-view feature fusion network that imitates the behavior of humans looking for unclear objects in images, that is, observing from multiple angles, distances, and angles, and capturing key boundary and semantic information by comparing and fusing the extracted multi-view features; "PraNet" stands for Parallel Reverse Attention Network, which is a parallel reverse attention network that first aggregates features in high layers using parallel partial decoders, and then mines boundary clues through reverse attention modules to establish the relationship between regions and boundary clues.

Claims

1. A camouflaged target detection method based on multi-dimensional polarization information, characterized in that: The steps are as follows: Step 1: Data collection and processing; Step 1.1: Use a polarization camera to capture the relative polarization angle in the nth scene They are , , , A set of original polarization images , thus obtaining N groups of original polarization images under N scenes; among them, Indicates the relative polarization angle in the nth scene The original polarization image under ; Step 1.2: For each set of original polarization images under N scenes, the polarization angle is The original polarization image is annotated to obtain a pixel-level annotated image, which is used as a real camouflage image; Step 2: Construct a deep convolutional neural network for camouflage target detection based on multi-dimensional polarization information, including: encoding module and decoding module; Step 2.1, the encoder After processing, the structural feature map of the nth scene and the polarization fusion feature of the nth scene are obtained; Step 2.2: After the decoder processes the structural feature map and polarization fusion features of the nth scene, the nth scene is obtained. The predicted camouflage map F of the scene n ; Step 3: training a camouflaged target detection model based on multi-dimensional polarization information; Step 3.1: Based on The real camouflage image and the predicted camouflage image F of the scene n Construct binary cross entropy loss; Step 3.2: Based on The real camouflage image and the predicted camouflage image F of the scene n The intersection and union construction of loss; Step 3.3: Binary cross entropy loss and After the losses are weighted, the total loss is obtained; Step 3.4: Use the gradient descent method to train the deep convolutional neural network for camouflaged target detection, and calculate the total loss to update the network parameters until the total loss converges, thereby obtaining a camouflaged target detection model based on multi-dimensional polarization information, which is used to realize camouflaged target detection for any multi-dimensional polarization image to be predicted.

2. A camouflaged target detection method based on multi-dimensional polarization information according to claim 1, characterized in that: The encoder in step 2.1 includes: a polarized dynamic attention module, a Res2Net backbone network, and H receptive field modules; Step 2.1.1, polarized attention module After processing, the nth polarization-enhanced dimensional information image D is obtained. n ; Step 2.1.2, the Res2Net backbone network is composed of H-level initial convolutional layers, H-level downsampling convolutional layers and H-level residual blocks; Among them, the h-th level initial convolution layer includes: a convolution layer, a BN layer and a ReLU activation function layer in sequence; when When D n After being processed by the h-th level initial convolution layer, the h-th level downsampling convolution layer and the h-th level residual block, the h-th level structural feature map is output. ; when When Level structure feature map Input to the hth level initial convolutional layer for processing, and then pass through the hth level initial convolutional layer in sequence. The downsampling convolutional layer and the After processing the level-h residual block, the h-th level structural feature map is obtained. , so that the H-th level residual block outputs the H-th level structural feature map ; Step 2.1.3, construct each receptive field module including K branches, a standard convolutional layer and a ReLU layer; when When the h-th level structural feature graph Input into the hth receptive field module and processed by the 2D convolution of the kth branch, the texture feature of the kth branch output by the hth receptive field module is obtained. ; when When the h-th level structural feature graph After inputting several 2D convolutional layers of the kth branch in the hth receptive field module for processing, the texture features of the kth branch output by the hth receptive field module are obtained. ; The texture features of the K branches output by the h-th receptive field module After cascading, the hth detail feature is output through the processing of the standard convolution layer ; The second branch feature output by the hth receptive field module and After addition, the data is input into the ReLU layer for processing, and finally the polarization fusion feature of the nth scene is output by the hth receptive field module. ; Thus, the H receptive field modules output the H polarization fusion features of a scene .

3. A camouflaged target detection method based on multi-dimensional polarization information according to claim 2, characterized in that: The polarized attention module in step 2.1.1 includes: several 3D convolutional layers, an adaptive average pooling layer and a sigmoid function; 3D convolutional layer pairs After 3D convolution processing, the difference features between the polarization angle images in the nth scene are obtained ; After being processed by the adaptive average pooling layer and the sigmoid function, the weighted features of the group polarization image in the nth scene are obtained. ; right and After performing element-by-element multiplication, we get Relative polarization angle in each scene Angle characteristics below , and then After channel cascade operation, the nth polarization-enhanced dimensional information image D is obtained. n .

4. A camouflaged target detection method based on multi-dimensional polarization information according to claim 3, characterized in that: The decoder in step 2.2 includes: a feature edge fusion module and a high and low gating selection module; Step 2.2.1, the feature edge fusion module is composed of H fusion blocks, each of which includes several convolutional layers and an average pooling layer; and As input, it is sent to the feature edge fusion module for processing to obtain the edge fusion feature ; when When With The first scene Level polarization fusion feature After channel cascading, The input is sent to the h-th fusion block for processing to obtain the h-th edge feature map ; when When Edge feature map and As an input, it is sent to the hth fusion block for processing to obtain the hth edge feature map , so that the Hth fusion block outputs the Hth edge feature map ; Step 2.2.2 The high and low gated selection module contains a gated selection block and a residual attention block; No. The first-level polarization fusion features of the scene and The input gate selection block is processed to obtain the Cross-layer features ; Will After being processed in the residual attention block, the output The predicted camouflage map F of the scene n .

5. A camouflaged target detection method based on multi-dimensional polarization information according to claim 4, characterized in that: The gate selection block in step 2.2.2 is first After the standard convolution operation, batch normalization and ReLU function processing are performed in sequence, the first Refinement Features ; right After the transposed convolution operation, the upsampling operation is performed to obtain the first Perceptual features ; right and After cascading on the channel, we get Gated Map ; right After the transposed convolution operation, After cascading in the channel dimension, we get Cross-layer features .

6. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the camouflaged target detection method according to any one of claims 1 to 5, and the processor is configured to execute the program stored in the memory.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the disguised target detection method according to any one of claims 1 to 5 are executed.

Images