A Camouflaged Target Detection Method Based on Split-Focal-Plane Polarization Imaging

Abstract

The present invention discloses a method for detecting camouflaged targets based on split focal plane polarization imaging, including: 1. Data acquisition and processing to obtain split focal plane images for subsequent operations; 2. Constructing a deep convolutional neural network using polarization images as input, taking the split focal plane images as input, and using the deep convolutional neural network to extract and generalize the feature distribution of camouflaged targets, thereby obtaining a camouflaged target detection model; 3. Using the trained model to detect camouflaged targets in the camouflaged scene to be detected. Under reasonable data model conditions, the present invention can effectively improve the accuracy of detecting camouflaged targets in complex and changeable environments by introducing scene polarization information.

Description

Technical Field

[0001] The present invention belongs to the fields of computer vision, image processing and analysis, and specifically relates to a method for detecting camouflaged targets based on split focal plane polarization imaging. Background Art

[0002] Camouflage, as an important example of natural selection evolution, is widely adopted by prey in the wild to reduce the possibility of being discovered by predators. Camouflaged target detection is a technique for segmenting the entire range of a camouflaged target. It has a wide range of applications in fields such as military (e.g., military camouflage pattern design), agriculture (e.g., pest identification and protection), medicine (e.g., polyp segmentation), and ecological protection (e.g., wildlife protection). Due to its scientific value and practical application value, the task of camouflaged target detection has attracted increasing attention from researchers. Since the foreground target usually has a very similar appearance to the surrounding objects, visual cues for target detection such as texture, contrast, edges, color, and target size are vulnerable to basic camouflage strategies such as background matching and disruptive coloring. Compared with general visual tasks, the task of camouflaged target detection is more challenging and researchable.

[0003] Currently, camouflaged target detection methods can be simply divided into traditional methods based on manually designed features and learning-based methods. Traditional camouflaged target detection methods are mainly oriented towards military scenario applications, mainly focusing on the spectral characteristics differences between the target and the ambient light under different materials, and then performing spectral analysis through hyperspectral detection technology. Moreover, traditional methods based on manually designed features highly rely on visual features (such as edges, colors, brightness, gradients, textures, etc.), and their generalization ability is limited. When the environment undergoes major changes resulting in changes in some visual features, it is necessary to redesign the visual features, relabel, and design the model. Therefore, traditional camouflaged target detection methods cannot be applied to overly complex and changeable environments. Learning-based camouflaged target detection methods mainly construct large-scale and standardized datasets, use neural networks to fit the features exhibited by the camouflaged targets in the datasets, and finally make predictions on the input. Learning-based methods mainly focus on using deep neural networks to extract high-level semantic features to distinguish hidden objects from complex scenes. Currently, deep learning-based camouflaged target detection methods can balance detection accuracy and detection efficiency in most scenarios, broaden the application scenarios of camouflaged target detection, and improve their generalization ability.

[0004] However, the data targeted by current learning-based methods are almost all traditional intensity images, which can only utilize the external feature information in the scene. In challenging scenarios where the external feature differences between the target and the background are small, these methods are difficult to accurately segment the target. Traditional methods often highlight the camouflaged area by mining the internal feature differences between the target and the environment, but both the detection efficiency and accuracy need to be improved. Summary of the Invention

[0005] The present invention addresses the deficiencies in the prior art by providing a method for detecting camouflaged targets based on split focal plane polarization imaging, with the aim of obtaining a generalization model that can effectively detect camouflaged targets in complex scenarios, thereby improving the accuracy and precision of detecting camouflaged targets in complex and variable environments.

[0006] To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] A method for detecting camouflaged targets based on split focal plane polarization imaging according to the present invention is characterized by being carried out in the following steps:

[0008] Step 1: Data acquisition and processing;

[0009] Step 1.1: Use a polarization camera to capture a set of original polarization images with relative polarization angles θ of 0°, 45°, 90°, and 135° in the nth scene Thereby obtaining N sets of original polarization images in N scenes; where represents the original polarization image at the relative polarization angle θ in the nth scene, n ∈ [1, N];

[0010] Step 1.2: For the N sets of original polarization images, label the original polarization image with a polarization angle of 0° in each set of original polarization images to obtain a pixel-level labeled image, which is used as the true camouflage image;

[0011] Step 1.3: Recombine a set of original polarization images of the nth scene using the micro-polarization array distribution method of the split focal plane polarization camera sensor to obtain the split focal plane image DoFP of the nth scene n ;

[0012] Using the split focal plane image DoFP of the nth scene n and its corresponding true camouflage image to form a set of polarization image data;

[0013] Step 2: Construct a deep convolutional neural network for detecting camouflaged targets based on split focal plane polarization imaging, including: an encoding module and a decoding module;

[0014] Step 2.1: The encoder is used to process the split focal plane image DoFP of the nth scene n and obtain H enhanced features of the nth scene;

[0015] Step 2.2: The decoding module processes the H enhanced features and outputs the camouflage prediction map r of the nth scene n ;

[0016] Step 3: Train a camouflaged target detection model based on split focal plane polarization imaging;

[0017] Based on the defocused plane images corresponding to N scenarios and their corresponding true camouflage images, the polarization image camouflage target detection model is trained using an adaptive motion amount random optimization method, and the weighted binary cross-entropy loss and the weighted IoU loss are jointly used as the loss function of the model to calculate the loss between the camouflage prediction map and the true camouflage map, so as to update the model parameters until the loss function of the model converges, thereby obtaining an optimal dual-input network camouflage target detection model using polarization images, which is used to detect camouflage targets for any intensity image and polarization image to be predicted.

[0018] The characteristics of a camouflage target detection method based on defocused plane polarization imaging according to the present invention also lie in that the encoder in step 2.1 includes: a polarization information extraction module, a Res2Net50 backbone network, and H receptive field modules, and the processing is carried out according to the following steps:

[0019] Step 2.1.1: The polarization information extraction module includes: a number of convolutional layers, and a BN layer and a ReLU activation layer are connected after each convolutional layer;

[0020] The defocused plane image DoFP of the nth scenario n After being processed by the polarization information extraction module, the shallow feature map of the nth scenario is obtained x h , y h , c h respectively represent the height, width, and number of channels of the shallow feature map;

[0021] Step 2.1.2: The Res2Net backbone network is composed of H levels of residual convolutional blocks. Among them, the hth level of residual convolutional block is composed of the hth level of X two-dimensional convolutional layers Dconv2d h connected in series;

[0022] Among them, the hth level of the xth layer of two-dimensional convolutional layer Dconv2d h，x successively includes: a convolutional layer, a BN layer, and a ReLU activation function layer;

[0023] When h = 1, the input xth layer convolutional layer Dconv2d of the hth level of residual convolutional module h is processed, and the fth level of image feature map is output

[0024] When f = 2, 3,..., H, the f - 1th level of image feature map is input into the hth level of residual convolutional module for processing, and the fth level of image feature map is obtained Thus, the Hth level of image feature map is output by the Hth level of residual convolutional module

[0025] Step 2.1.3: Construct H receptive field modules, where each receptive field module includes K branches, a standard convolutional layer, and a ReLU layer;

[0026] Take the H-level image feature maps output by the H-level residual convolutional blocks as the inputs of the H receptive field modules respectively;

[0027] When k ≤ 2, the h-level image feature map is processed by a standard 2D convolution in the k-th branch of the h-th receptive field module to obtain the feature output by the k-th branch of the h-th receptive field module

[0028] When k > 2, the h-level image feature map is processed by several standard convolutional layers in the k-th branch of the h-th receptive field module to obtain the feature output by the h-th branch of the h-th receptive field module

[0029] Cascade the features output by the K branches of the h-th receptive field module and after processing by a standard convolutional layer, output the features Add the feature output by the second branch of the h-th receptive field module to and then input it into a ReLU layer for processing, finally obtaining an enhanced feature of the h-th receptive field module for the n-th scene Thus, H enhanced features of the n-th scene are output by the H receptive field modules, where h = 1, 2, …, H.

[0030] The decoding module in Step 2.2 includes: H decoding branches; input the H enhanced features into the decoding module and process them according to the following process:

[0031] When h = 1, 2, the h-th decoding branch directly outputs the h-th enhanced feature denoted as the h-th feature

[0032] When h = 3, the h-th decoding branch directly outputs the h enhanced features denoted as the h-th feature Meanwhile, the h-th decoding branch performs upsampling on the h-th enhanced feature and then performs a standard convolution operation to obtain the h-th feature

[0033] When h = 4, 5, …, H, the h-th decoding branch performs an upsampling operation by a factor of 2a on the h-th enhanced feature and then performs a standard convolution operation to obtain the h-th feature Meanwhile, the h-th decoding branch performs an upsampling operation by a factor of a on the h-th enhanced feature and then performs a standard convolution operation to obtain the h-th feature And the h-th decoding branch performs an upsampling operation by a factor of a on the h-th enhanced feature and then performs a standard convolution operation to obtain the h-th feature Thus, the H-th decoding branch performs an upsampling operation by a factor of 2a on the H-th enhanced feature and then performs a standard convolution operation to obtain the H-th feature Meanwhile, the H-th decoding branch performs an upsampling operation by a factor of a on the H-th enhanced feature and then performs a standard convolution operation to obtain the H-th feature And the h-th decoding branch performs an upsampling operation by a factor of a on the h-th enhanced feature and then performs a standard convolution operation to obtain the H-th feature

[0034] After multiplying the features a new feature is obtained

[0035] After performing a multiplication operation on the features a new feature is obtained

[0036] After cascading the features and and first performing a standard convolution operation, then performing an upsampling operation by a factor of a and a standard convolution operation in sequence, a cascaded feature

[0037] After cascading the features with and the feature and performing a cascading operation and then performing several convolution operations, the output of the decoder module is the camouflage prediction map of the n-th scene

[0038] An electronic device according to the present invention includes a memory and a processor, characterized in that the memory is used to store a program for supporting the processor to execute any one of the camouflage target detection methods, and the processor is configured to execute the program stored in the memory.

[0039] A computer-readable storage medium of the present invention, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the camouflage target detection method are executed.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] 1. By constructing a camouflage target detection deep neural network based on split focal plane polarization imaging and using label data to supervise the deep neural network for learning, a robust polarization camouflage target detection feature model is obtained, solving the problem that in the statistical model, clues such as color, depth, and background prior are used for model design, ignoring the internal feature information of the scene and having low detection accuracy in scenes with small external differences.

[0042] 2. Due to the relative differences in the refraction and reflection characteristics of light by different materials, surface roughness, and structures, the polarization state of light will change. Therefore, the reflection sources in the scene can be distinguished by the change in the polarization state. Thus, the polarization characteristics can provide richer information for scene understanding and more basis for distinguishing targets. The camouflage target detection model based on split focal plane polarization imaging constructed by the present invention can well segment the camouflage target and the background by introducing the internal polarization information characteristics in the scene and utilizing this characteristic.

[0043] 3. The deep neural network based on split focal plane polarization imaging constructed by the present invention takes the DoFP image as the network input and uses the polarization information extraction module to extract features from the DoFP image, making full use of the polarization information and intensity information in the scene; thus solving the problem that most deep learning-based camouflage target methods are difficult to accurately segment the target in scenes where the external feature differences between the target and the background are small.

[0044] 4. By means of the idea of weight recombination and weight sharing within macro-pixels, the present invention solves the problem of weight initialization of the polarization information extraction module, improves the pertinence of the polarization information extraction module for scene feature extraction, and thus effectively improves the robustness of camouflage target detection in low-contrast and complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flowchart for camouflage target detection of the camouflage target model based on split focal plane polarization imaging;

[0046] Figure 2 It is a schematic diagram of the polarization image deep neural network structure of split focal plane polarization imaging;

[0047] Figure 3 It is a schematic diagram of a single receptive field module;

[0048] Figure 4 This is the camouflage target prediction result graph of this design method and other camouflage target detection methods on the polarization dataset. Specific implementation manner

[0049] In this embodiment, a camouflage target detection method based on split focal plane polarization imaging aims to locate "indistinguishable targets" with highly similar backgrounds. By introducing polarization information, the difference between an object and its surrounding environment in the camouflage target detection task is enlarged, and a deep neural network based on CNN is constructed to process polarization images. By constructing a camouflage target detection model based on split focal plane polarization imaging, the inherent feature differences between the target and the background can be fully utilized to better complete the camouflage target detection task. Specifically, as Figure 1 shown, this method is carried out according to the following steps:

[0050] Step 1: Data acquisition and processing;

[0051] Step 1.1: Use a polarization camera to capture a set of original polarization images with relative polarization angles θ of 0°, 45°, 90°, and 135° in the nth scene to obtain N sets of original polarization images in N scenes; where, represents the original polarization image at the relative polarization angle θ in the nth scene, n ∈ [1, N];

[0052] In this embodiment, a LucidTriton split focal plane polarization camera is used to capture the polarization camouflage target detection dataset, which contains a total of N = 639 scenes, and the original resolution of each image is 1224×1024.

[0053] Step 1.2: For the N sets of original polarization images, take the original polarization image with a polarization angle of 0° in each set of original polarization images for annotation to obtain a pixel-level annotation image, which is used as the true camouflage map;

[0054] In this embodiment, annotation is carried out through the Labelme software. The annotation image assigns a class label V, V ∈ (0,1), to each pixel point in the polarization image, which respectively represent the color of the pixel point as: black, white; where black represents the background of the pixel point, and white represents the target. And the polarization camouflage target detection dataset is divided into training and testing sets, where the training set contains 511 scenes and the testing set contains 128 scenes.

[0055] Step 1.3: Recombine a set of original polarization images in the nth scene by using the micro-polarization array distribution method of the split focal plane polarization camera sensor to obtain the split focal plane image DoFP of the nth scene n ;

[0056] Using the defocused plane image DoFP of the nth scene n and its corresponding true camouflage image to form a set of polarization image data;

[0057] In this embodiment, in order to balance the training efficiency and accuracy of the model, the sizes of the four original polarization images at 0°, 45°, 90°, and 135° are downsampled to 352×352, and then synthesized into a 704×704 DoFP image, which is used as the network input.

[0058] Step 2: Construct a deep convolutional neural network for camouflage target detection based on defocused plane polarization imaging, including: an encoding module and a decoding module;

[0059] Step 2.1: The encoder is constructed by a polarization information extraction module, a Res2Net50 backbone network, and a receptive field module.

[0060] Step 2.1.1: The polarization information extraction module includes: a number of convolutional layers, and each convolutional layer is followed by a BN layer and a ReLU activation layer;

[0061] The defocused plane image DoFP of the nth scene n After being processed by the polarization information extraction module, the shallow feature map of the nth scene is obtained x h , y h , c h respectively represent the height, width, and number of channels of the shallow feature map.

[0062] In this embodiment, the number of samples in the batch processing is set to 25, the number of channels of each sample is 3, and the height and width are both 704. That is, each batch of training inputs is 25 defocused plane images with a size of 704×704, which is equivalent to 25 groups of original polarization images with a size of 352×352. Finally, the size of the DoFP image input to the polarization information extraction module in each batch of training is 25×3×708×708, and the feature size output by the polarization information extraction module is 25×64×88×88.

[0063] The polarization information extraction module consists of a 6×6 convolutional layer and two 3×3 convolutional layers in sequence. After each layer of convolution, there are corresponding BN layers and ReLU activation layers to accelerate convergence and increase the nonlinearity of the network. At the same time, the pre-trained weights of the first layer of Res2Net are copied four times and extended to a size of 6×6, and the information in different polarization directions is processed with the same weight parameters.

[0064] Step 2.1.2: The Res2Net backbone network is composed of H-level residual convolutional blocks. Among them, the h-level residual convolutional block is composed of the h-level X-layer two-dimensional convolutional layer Dconv2d h connected in series;

[0065] Among them, the two-dimensional convolutional layer Dconv2d of the x-th layer of the h-th level h,x successively includes: a convolutional layer, a BN layer, and a ReLU activation function layer;

[0066] In this embodiment, H = 4, and the residual convolutional blocks are respectively denoted as Rblock1,..., Rblock h ,..., Rblock4; among them, Rblock h represents the h-th level residual convolutional block, h = 1, 2,...., 4, Figure 2 Layer1, Layer, Layer3, and Layer4 in

[0067] correspond to 4 residual convolutional modules Rblock1, Rblock2, Rblock3, and Rblock4 respectively. h,x The two-dimensional convolutional layer Dconv2d of the x-th layer of the h-th level x successively includes: a convolutional layer, a BN layer, and a ReLU activation function layer, where the convolutional kernel size of the convolutional layer of the x-th layer is k

[0068] In this embodiment, X = 3, k x are respectively equal to 1, 3, 1; when h = 1, Rblock1 contains 9 Dconv2d1s, when h = 2, Rblock2 contains 12 Dconv2d2s, when h = 3, Rblock3 contains 18 Dconv2d3s, and when h = 4, Rblock4 contains 9 Dconv2d4s.

[0069] When h = 1, input the X-layer convolutional layer Dconv2d of the h-th level residual convolutional module h for processing, and output the h-th level image feature map

[0070] In this embodiment, the width, height, and number of channels of the first-level image feature map are 88, 88, and 256.

[0071] When h = 2, 3,..., H, input the (h - 1)-th level image feature map into the h-th level residual convolutional module for processing, and obtain the h-th level image feature map Thus, the H-th level image feature map is output by the H-th level residual convolutional module

[0072] In this embodiment, the output features of the second-level multi-dimensional convolutional layer The width, height, and number of channels are 44, 44, and 512; the output features of the third-level multi-layer two-dimensional convolutional layer The width, height, and number of channels are 22, 22, and 1024; the output features of the fourth-level multi-layer two-dimensional convolutional layer The width, height, and number of channels are 11, 11, and 2048.

[0073] Step 2.1.3, construct H receptive field modules, where a receptive field module includes K branches, a standard convolutional layer, and a ReLU layer;

[0074] Take the H-level image feature maps output by the H-level residual convolutional blocks As the inputs of the H receptive field modules respectively;

[0075] When k ≤ 2, the h-level image feature map Is input into a standard 2D convolution in the k-th branch of the h-th receptive field module for processing, and the features output by the k-th branch of the h-th receptive field module are obtained

[0076] When k > 2, the h-level image feature map Is input into several standard convolutional layers in the k-th branch of the h-th receptive field module for processing, and the features output by the k-th branch of the h-th receptive field module are obtained

[0077] The features output by the K branches of the h-th receptive field module After concatenation and processing through a standard convolutional layer, the output features The features output by the second branch of the h-th receptive field module And After addition, and then input into a ReLU layer for processing, finally an enhanced feature of the n-th scene output by the h-th receptive field module is obtained Thus, H enhanced features of the n-th scene are output by the H receptive field modules, h = 1, 2,..., H.

[0078] In this example, K = 5. The input of the h-th receptive field module is the h-level image feature map

[0079] As Figure 3 Shown, when k = 1, 2, the h-level image feature map Is input into a standard 2D convolution with a size of 1×1 (composed of a 2D convolutional layer and a BN layer) in the k-th branch of the h-th receptive field module for processing. Denote the output features of the first branch and the second branch of the h-th receptive field module as And

[0080] When k = 3, 4, 5, the h-th level image feature map Input several standard convolutional layers in the k-th branch of the h-th receptive field module for processing, and obtain the features output by the k-th branch of the h-th receptive field module The k-th branch consists of a standard convolutional layer with a convolutional kernel size of (2k - 1)×(2k - 1) and a standard convolutional layer with a dilation rate of (2k - 1) for 3×3. And, a combination of 1×(2k - 1) convolutional layer and (2k - 1)×1 convolutional layer is used to replace the (2k - 1)×(2k - 1) convolutional layer to reduce the computational cost. Denote the output features as

[0081] After cascading, pass through a standard convolutional layer with a size of 3×3, and output features Take the output of the second branch of the h-th receptive field module and add them together, then input them into a ReLU layer for processing, and finally the h-th receptive field module outputs an enhanced feature of the n-th scene h = 1, 2, …, 4;

[0082] The input of the first receptive field module is the features output by the polarization information extraction module and the features output by the first layer of the backbone network After 2-fold downsampling, the two are cascaded to obtain an enhanced feature with a final output size of 88×88×32 The input of the second receptive field module is the output features of the second layer of the backbone network Finally, an enhanced feature with an output size of 44×44×32 is obtained The input of the third receptive field module is the output features of the third layer of the backbone network Finally, an enhanced feature with an output size of 22×22×32 is obtained The input of the fourth receptive field module is the output features of the fourth layer of the backbone network Finally, an enhanced feature with an output size of 11×11×32 is obtained

[0083] Step 2.2, construct a decoding module including H decoding branches; input the H enhanced features into the decoding module;

[0084] When h = 1, 2, the h-th decoding branch directly outputs the h-th enhanced feature and denote it as the h-th feature

[0085] When h = 3, the h-th decoding branch directly outputs the h enhanced features which are denoted as the h-th feature Meanwhile, the h-th decoding branch performs an upsampling operation on the h-th enhanced feature and then performs a standard convolution operation to obtain the h-th feature

[0086] When h = 4, 5, …, H, the h-th decoding branch performs a 2a-fold upsampling operation on the h-th enhanced feature and then performs a standard convolution operation to obtain the h-th feature Meanwhile, the h-th decoding branch performs an a-fold upsampling operation on the h-th enhanced feature and then performs a standard convolution operation to obtain the h-th feature Moreover, the h-th decoding branch performs an a-fold upsampling operation on the h-th enhanced feature and then performs a standard convolution operation to obtain the h-th feature Thus, the H-th decoding branch performs a 2a-fold upsampling operation on the H-th enhanced feature and then performs a standard convolution operation to obtain the H-th feature Meanwhile, the H-th decoding branch performs an a-fold upsampling operation on the H-th enhanced feature and then performs a standard convolution operation to obtain the H-th feature Moreover, the h-th decoding branch performs an a-fold upsampling operation on the h-th enhanced feature and then performs a standard convolution operation to obtain the H-th feature

[0087] After multiplying the features a new feature is obtained

[0088] After performing a multiplication operation on the features a new feature is obtained

[0089] After cascading the features and performing a standard convolution operation first, and then performing an a-fold upsampling and a standard convolution operation in sequence, a cascaded feature is obtained

[0090] After cascading the feature with and the feature performing a cascading operation and then performing several convolution operations, the output of the decoder module for the n-th scene's camouflage prediction map is obtained

[0091] In this embodiment, when h = 1 or 2, the h-th decoding branch directly outputs the h-th enhanced feature, denoted as the h-th feature.

[0092] When h = 3, the h-th decoding branch directly outputs the h enhanced features denoted as the h-th feature. At the same time, the h-th decoding branch performs a 2x upsampling process on the h-th enhanced feature and then performs a 3x3 convolution operation to obtain the h-th feature.

[0093] When h = 4, the h-th decoding branch performs a 4x upsampling operation on the h-th enhanced feature and then performs a 3x3 convolution operation to obtain the h-th feature. At the same time, the h-th decoding branch performs a 2x upsampling operation on the h-th enhanced feature and then performs a 3x3 convolution operation to obtain the h-th feature. Moreover, the h-th decoding branch performs a 2x upsampling operation on the h-th enhanced feature and then performs a 3x3 convolution operation to obtain the h-th feature.

[0094] Multiply the features to obtain a new feature denoted as

[0095] Multiply the features to obtain a new feature denoted as

[0096] Concatenate the features and the feature and first perform a 3x3 convolution operation, then successively perform a 2x upsampling and a 3x3 convolution operation to obtain the concatenated feature.

[0097] Concatenate the feature with and the feature and perform two 3x3 standard convolution operations and then a 1x1 convolution operation after the concatenation operation to obtain the camouflage prediction map of the n-th scene output by the decoder module.

[0098] To improve the prediction effect, the prediction result r obtained by the decoder module n is input into an optimization module. The optimization module has a total of 9 operation blocks block. l, where \(l = 1, 2, \ldots, 9\). Each block performs basically the same operation on the input, and the only difference lies in the last operation before the output of each operation block, which are max - pooling downsampling, upsampling, and convolution operations respectively. The refined final prediction result output is output by the optimization module. n .

[0099] Denote the input as \(x\). First, perform a \(3\times3\) convolution operation on \(x\), and denote the output at this time as \(h_x\). When \(l = 1, 2, \ldots, 5\), take \(h_x\) as the input and enter an operation block composed of a \(3\times3\) convolutional layer, a BN layer, and a ReLU layer in sequence to obtain the output feature \(h_x\). l , and at the same time, each time entering the operation block, \(h_x\) is updated once. That is, when \(l = 1, \ldots, 4\), perform a max - pooling downsampling operation on \(h_x\). l When \(l = 5\), perform a 2 - fold upsampling operation on \(h_x\). And the output of the previous block will be used as the input of the next block. l

[0100] When \(l = 6\), concatenate \(h_{x5}\) and \(h_{x4}\) as the input of block6, and perform a 2 - fold upsampling operation after the output. Denote the output feature at this time as \(d4\). When \(l = 7\), concatenate \(d4\) and \(h_{x3}\) as the input of block7, and perform a 2 - fold upsampling operation after the output. Denote the output feature at this time as \(d3\). When \(l = 8\), concatenate \(d3\) and \(h_{x2}\) as the input of block8, and perform a 2 - fold upsampling operation after the output. Denote the output feature at this time as \(d2\). When \(l = 9\), concatenate \(d2\) and \(h_{x1}\) as the input of block9. Denote the output feature at this time as \(d1\). Finally, input \(d1\) into a \(3\times3\) convolution module to restore the number of channels when the initial input is \(x\), and add it to the initial input \(x\) to obtain the refined final prediction result output. n . The refined final prediction result output n has a size of \(352\times352\times1\).

[0101] Step 3: Train a camouflage target detection model based on defocused - plane polarization imaging;

[0102] Step 3.1: For the DoFP images of \(N\) scenes and their corresponding real camouflage maps, use the adaptive motion - amount random optimization method to train the polarization - image camouflage target detection model, and use the weighted binary cross - entropy loss and the weighted IoU loss together as the loss function to calculate the loss between the camouflage prediction map and the real camouflage map to update the network parameters until the loss function converges, so as to obtain the optimal camouflage target detection model based on defocused - plane polarization imaging;

[0103] Step 3.2: Use the optimal polarization image camouflage target detection model based on sub-focal plane imaging to perform camouflage target detection on any intensity image and polarization degree image to be predicted.

[0104] In this embodiment, the DoFP images of 128 test scenarios in the polarization camouflage target detection dataset are used as inputs. After being calculated by the camouflage target monitoring model based on sub-focal plane polarization imaging, the predicted camouflage target map is obtained, and then it is compared with the true camouflage target map of the corresponding scenario to calculate the detection accuracy.

[0105] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0106] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is run by a processor, it executes the steps of the above method.

[0107] Table 1 shows the comparison results of the camouflage target detection method based on sub-focal plane polarization imaging of the present invention with other current camouflage target detection methods, using "S-measure", "E-measure", "MAE", and "F-measure" as evaluation indicators and the test set of the polarization camouflage target detection dataset. "S-measure" can be used to reflect the structural characteristics and can be used to evaluate the structural similarity between the predicted result and the true camouflage target. The closer its value is to 1, the better the camouflage target detection effect. "E-measure" evaluates the overall and local accuracy and quality of the predicted result in the binary segmentation task by combining pixel-level matching and image-level statistics. The closer its value is to 1, the better the camouflage target detection effect. "F-measure" is the weighted harmonic mean based on precision and recall. The closer its value is to 1, the better the camouflage target detection effect. "MAE" is an evaluation indicator widely used in salient object detection and is used for the pixel-by-pixel difference between the binary prediction result and the ground truth map. The closer its value is to 0, the better the camouflage target detection effect. According to the quantitative analysis in Table 1, it can be seen that the method proposed in this design has achieved good results in each evaluation indicator.

[0108] Table 1

[0109] Methods S-measure E-measure F-measure MAE BASNet 0.830 0.868 0.722 0.020 PraNet 0.855 0.908 0.756 0.019 LSR 0.863 0.910 0.793 0.014 SINet-V1 0.789 0.811 0.042 0.646 PFNet 0.849 0.910 0.769 0.017 <![CDATA[C 2 FNet]]> 0.860 0.913 0.774 0.018 SINet-V2 0.865 0.925 0.784 0.015 Ours 0.864 0.928 0.793 0.014

[0110] Figure 4The results of the camouflage target detection method based on split focal plane polarization imaging in this design and other current camouflage target detection methods. Among them, Ours represents the polarization image camouflage target detection method based on a dual-input network of the present invention; BASNet represents stacking two U-shaped networks in sequence to generate a saliency map in a prediction-refinement manner, and also proposes a hybrid training loss to supervise the training process; SINet-V1 represents designing and utilizing a search module and an identification module to locate and identify camouflage objects based on imitating the behavior process of hunters; LSR represents constructing a multi-task learning framework in the camouflage target prediction process, introducing auxiliary tasks such as classification, and proposing a model that simultaneously locates, segments, and ranks camouflage objects, where the Rank module can rank the difficulty levels of camouflage target detection; RraNet represents proposing a parallel reverse attention network, first using a parallel region decoder to aggregate high-level network features, generating a global map based on the aggregated features and using it as the initial guiding region for subsequent components, and in addition using a reverse attention module to mine boundary clues; PFNet represents proposing a new distraction mining strategy based on context feature learning and developing a framework for accurate camouflage target detection, which locates potential targets by exploring long-term correlations and refines the segmentation effect by distraction discovery and removal; C 2 Net also represents designing a dual-branch global context module (DGCM) based on context features to mine rich context information. In addition, an attention-induced cross-layer fusion module (ACFM) is introduced to aggregate multi-level features; SINet-V2 represents that its detection mechanism is similar to SINet-V1, also divided into two steps of search and identification. At the same time, in order to better achieve inter-layer information fusion and prevent information loss or feature redundancy, a reverse guidance method is adopted to obtain the final camouflage prediction map.

Claims

1. A method for detecting camouflaged targets based on split focal plane polarization imaging, characterized in that It is carried out according to the following steps: Step 1, data acquisition and processing; Step 1.

1. Use a polarization camera to capture a set of original polarization images at the relative polarization angles in the th scene, respectively, so as to obtain sets of original polarization images in the th scene; where represents the original polarization image at the relative polarization angle in the th scene, ; and ; Step 1.

2. For each group of original polarization images, select the original polarization image with a relative polarization angle of in each group of original polarization images for annotation to obtain a pixel-level annotated image, which is used as the true camouflage image; Step 1.

3. Recombine a set of original polarization images of the th scene by using the micro-polarization array distribution method of a split focal plane polarization camera sensor to obtain a split focal plane image of the th scene ; Taking the sub-focal plane images of the th scene and their corresponding true camouflage images to form a set of polarization image data; Step 2: Construct a deep convolutional neural network for detecting camouflaged targets based on split focal plane polarization imaging, including: an encoding module and a decoding module; among them, the encoding module includes: a polarization information extraction module, a Res2Net50 backbone network, and receptive field modules; the polarization information extraction module includes: a number of convolutional layers, and each convolutional layer is followed by a BN layer and a ReLU activation layer; the receptive field module includes K branches, a standard convolutional layer, and a ReLU layer; the decoding module includes: decoding branches; Step 2.

1. The encoding module is used to process the sub-focal plane images of the th scene, and obtain H enhanced features of the th scene; ​ Step 2.2, the decoding module processes the H enhanced features and outputs the camouflage prediction map of the th scene ; Step 3, training a camouflage target detection model based on defocused plane polarization imaging; Based on the defocused plane images corresponding to N scenarios and their corresponding true camouflage images, an adaptive motion amount random optimization method is used to train the camouflage target detection model, and weighted binary cross-entropy loss and weighted loss are jointly used as the loss function of the model to calculate the loss between the camouflage prediction image and the true camouflage image, so as to update the model parameters until the loss function of the model converges, thereby obtaining an optimal dual-input network camouflage target detection model using polarization images, which is used to detect camouflage targets for any intensity image and polarization image to be predicted.

2. The method for detecting camouflaged targets based on split focal plane polarization imaging according to claim 1, wherein The encoding module in the said step 2.1 includes: a polarization information extraction module, a Res2Net50 backbone network, and receptive field modules, and the processing is carried out according to the following steps: Step 2.1.1, the polarization information extraction module includes: a plurality of convolutional layers, and a BN layer and a ReLU activation layer are connected after each convolutional layer; The sub-focal plane image of the first scene, after being processed by the polarization information extraction module, obtains the shallow feature map of the first scene, where respectively represent the height, width, and number of channels of the shallow feature map; Step 2.1.2, the Res2Net50 backbone network is composed of H-level residual convolution blocks, where the level residual convolution block is composed of the layer two-dimensional convolution layers connected in series; Among them, the layer two-dimensional convolutional layer successively includes: a convolutional layer, a BN layer, and a ReLU activation function layer; When it is, input it into the layer convolution layer of the nth-level residual convolution module for processing and output the nth-level image feature map ;​ When the -th input is processed in the -th residual convolution module, and the -th image feature map is obtained; thus, the -th residual convolution module outputs the -th image feature map ; Step 2.1.3, construct receptive field modules, where each receptive field module includes K branches, a standard convolutional layer, and a ReLU layer; The level residual convolution block output level image feature map is respectively used as the input of the receptive field modules; When the level image feature map is input into the th branch of the th receptive field module and processed by a standard 2D convolution, the feature output by the th branch of the th receptive field module is obtained; When the level image feature map is input into the number of standard convolutional layers in the th branch of the th receptive field module for processing, obtaining the feature After cascading the features output by the K branches of the th receptive field module and processing them through a standard convolutional layer, the output features are ; After adding the features output by the second branch of the th receptive field module to and then inputting them into a ReLU layer for processing, finally, an enhanced feature of the th scene output by the th receptive field module is obtained ; Thus, H enhanced features of the th scene are output by the receptive field modules ; .

3. The method for detecting camouflaged targets based on split focal plane polarization imaging according to claim 2, characterized in that, The decoding module in the said step 2.2 includes: decoding branches; and input enhanced features into the said decoding module and process them according to the following process: When the th enhanced feature is directly output as the th feature; When is reached, the th decoding branch directly outputs the enhanced feature, denoted as the th feature ; meanwhile, the th decoding branch performs upsampling on the th enhanced feature and then performs a standard convolution operation to obtain the th feature ; ​ When the th enhanced feature is upsampled by 2a times and then a standard convolution operation is performed, the th feature is obtained; at the same time, the th enhanced feature is upsampled by a times and then a standard convolution operation is performed, the th feature is obtained, and the th enhanced feature is upsampled by a times and then a standard convolution operation is performed, the th enhanced feature is upsampled by 2a times and then a standard convolution operation is performed, the th feature is obtained; at the same time, the th enhanced feature is upsampled by a times and then a standard convolution operation is performed, the th feature is obtained; and the th enhanced feature is upsampled by a times and then a standard convolution operation is performed, the ; Multiply the feature to obtain a new feature ; After multiplying the feature a new feature is obtained; The feature and After cascading the features, first perform a standard convolution operation, and then perform upsampling by a factor of a and a standard convolution operation in sequence to obtain the cascaded feature ; After cascading the feature with and the feature , and then performing a number of convolution operations, the predicted camouflage map of the th scene is obtained at the output of the decoder module .

4. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program for supporting the processor to execute any one of the camouflage target detection methods described in claims 1-3, and the processor is configured to execute the program stored in the memory.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it executes the steps of any one of the camouflage target detection methods described in claims 1-3.

Images