A camouflaged target segmentation method and system based on light intensity and polarization cues

Abstract

The present invention discloses a camouflaged target segmentation method and system based on light intensity and polarization cues. The method includes: obtaining a test picture and preprocessing the test picture to obtain a polarization information map of the test picture, where the polarization information map includes an intensity map, a degree of polarization map, and a polarization angle map; inputting the polarization information map of the test picture into a pre-trained camouflaged target segmentation network model, and successively performing multi-layer feature extraction processing, compression processing, multi-layer feature fusion processing, multi-branch search processing, and three-modal fusion processing on each polarization information map to obtain a mask image of the camouflaged target. By introducing polarization information including light intensity and polarization cues and using a two-stage fusion method of multi-layer fusion and multi-branch search, the problem of camouflaged target segmentation using light intensity and spectral polarization as cues is solved, helping the polarization vision system to accurately find camouflaged targets highly similar to the current scene.

Description

Technical Field

[0001] This invention relates to a method and system for segmenting camouflaged targets based on light intensity and polarization cues, belonging to the field of scene segmentation technology in computer vision. Background Technology

[0002] In recent years, with the rapid development of intelligent manufacturing, imaging systems such as polarization, hyperspectral, and infrared imaging have been gradually applied in various fields, playing a crucial role. Among them, polarization imaging technology primarily detects polarization features closely related to the target's physical characteristics by analyzing the polarization state of reflected and radiated light. This technology can improve the contrast between the target and the background, enriching the image's detail. Therefore, polarization imaging is increasingly being applied to camouflage target detection tasks, such as underwater camouflage target detection, defect detection, and military camouflage target identification.

[0003] In early traditional methods, researchers designed feature extraction operators applicable to polarization images for target detection. With the development of deep learning technology, many scholars have combined polarization images with deep learning. However, current research still has many shortcomings. First, most current methods for fusing polarization and intensity features use direct addition, which cannot fully leverage the advantages of each modality's information, nor can it fully fuse the effective information from both. Second, in complex environments such as forests and grasslands, while polarization information increases the contrast between the camouflaged target and the background, it also increases the detail information of other objects in the environment, resulting in a large amount of interference information in the polarization image. Summary of the Invention

[0004] The purpose of this invention is to address the problem that camouflage target segmentation models in complex environments struggle to efficiently integrate the advantages of light intensity and polarization information. It provides a camouflage target segmentation method and system based on light intensity and polarization cues. This innovatively utilizes light intensity and polarization cues to segment camouflage targets in images acquired by polarization cameras in special scenes, achieving effective and accurate camouflage target detection.

[0005] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0006] In a first aspect, the present invention provides a method for segmenting camouflaged targets based on light intensity and polarization cues, comprising:

[0007] The image to be tested is acquired and preprocessed to obtain a polarization information map of the image to be tested, which includes an intensity map, a degree of polarization map, and a polarization angle map.

[0008] The polarization information map of the image to be tested is input into a pre-trained camouflage target segmentation network model. Multi-layer feature extraction is performed on each polarization information map to obtain multi-layer modal features of each polarization information map. Each modal feature of each polarization information map is compressed to obtain multi-layer compressed features of each polarization information map. Multi-layer feature fusion is performed on the multi-layer compressed features of each polarization information map to obtain cross-fusion features of each polarization information map. Multi-branch search is performed on the cross-fusion features of each polarization information map to obtain residual fusion features of each polarization information map. Three-modal fusion is performed on the residual fusion features of the polarization information map to obtain the mask image of the camouflage target.

[0009] Furthermore, the step of acquiring the image to be tested and preprocessing it to obtain the polarization information map of the image to be tested specifically includes:

[0010] Select the scene to be tested and use a professional grayscale polarization camera to acquire the light intensity in four polarization directions: 0°, 45°, 90° and 135°.

[0011] Based on the light intensity in the four polarization directions, the Stokes parameters are calculated to obtain the polarization information map of the image under test.

[0012] Furthermore, the multi-layer feature extraction includes using a Res2Net-50 feature extraction network to extract features from each polarization information map, thereby obtaining multi-layer modal features for each polarization information map.

[0013] Furthermore, the multi-layer feature fusion processing of the compressed features of each polarization information map to obtain the cross-fusion features of each polarization information map specifically includes:

[0014] Based on the compression features of adjacent layers in each polarization information map, the common spatial attention map of each polarization information map is obtained, and the calculation formula is as follows:

[0015] F SA =SA(Concat((F1+U(F2)),(F1×U(F2)))),

[0016] In the formula, F SA This represents the common spatial attention map; F1 and F2 are two compressed features from adjacent layers; U(·) denotes the upsampling operation; Concat(·) denotes the connection operation; SA(·) denotes the spatial attention operation;

[0017] The common spatial attention map is used as the weight for spatial alignment of the combined features. Channel attention operations are then performed on the resulting spatially aligned features. Finally, the cross-fusion features of adjacent layers of each polarization information map are obtained through a join operation. The calculation formula is as follows:

[0018] F = Concat(CA(F) SA ×(F1+U(F2))),CA(F SA ×(F1×U(F2)))),

[0019] In the formula, F represents the cross-fusion feature of adjacent layers; CA(·) represents the channel attention operation;

[0020] By continuously fusing the compressed features of adjacent layers, the final cross-fusion feature of each polarization information map is obtained.

[0021] Furthermore, the multi-branch search processing of the cross-fusion features of each polarization information map to obtain the residual fusion features of each polarization information map specifically includes:

[0022] The cross-fusion features of the intensity maps are subjected to three-branch operations to obtain high-resolution feature maps F. u Original resolution feature map F c and low-resolution feature map F d The three-branch operation includes an upsampling branch operation, a convolution branch operation, and a pooling branch operation.

[0023] High-resolution feature map F u Original resolution feature map F c and low-resolution feature map F d The convolutional group features F are obtained by sequentially performing three branch operations: multi-scale channel attention processing, multiplicative fusion, and addition fusion. ucd ;

[0024] The convolutional group features F ucd Cross-fusion features of light intensity maps Residual fusion is performed to obtain the residual fusion feature F of the intensity map. I The calculation formula is:

[0025]

[0026] In the formula, F I This refers to the residual fusion characteristics of the intensity map; It represents the cross-fusion feature of the intensity map; U(·) represents the upsampling operation; D(·) represents the downsampling operation; Conv3(·) represents the convolution operation with a kernel size of 3×3; M(·) represents the multi-scale channel attention operation;

[0027] Based on the cross-fusion characteristics of the polarization degree map, the cross-fusion characteristics of the polarization angle map, and the multi-branch search process, the residual fusion characteristics F of the polarization degree map are obtained respectively. P The residual fusion feature F of the polarization angle map A .

[0028] Furthermore, the process of performing three-modal fusion processing on the residual fusion features of the polarization information map to obtain a mask image of the camouflaged target specifically includes:

[0029] The residual fusion features of the polarization information map are merged and fused to obtain the mask image of the camouflaged target. The calculation formula is as follows:

[0030]

[0031] In the formula, Ψ(·) represents a convolutional group containing a Conv layer, a BN layer, and a ReLU layer; Concat(·) represents a connection operation; Conv3(·) represents a convolutional layer with a kernel size of 3×3; M S (·) denotes a multi-scale spatial attention operation; pred denotes the final mask image; A i A p And A a The residual fusion features F of the intensity map are respectively I Residual fusion feature F of polarization degree map P The residual fusion feature F of the polarization angle map A Characteristics of the merged result.

[0032] Furthermore, the training method for the camouflaged target segmentation network model includes:

[0033] Obtain a training set, which includes polarization maps acquired by multiple professional polarization cameras;

[0034] The polarization maps in the training set are processed according to the camouflage target segmentation method, and the parameters of the camouflage target segmentation network model are optimized and adjusted according to the loss function value until the loss function value tends to stabilize, and then training is stopped.

[0035] The formula for calculating the loss function is as follows:

[0036] L=λ1L wbce +λ2L wDL ,

[0037] In the formula, L is the loss function in this embodiment; L wbce L is the cross-entropy loss function; wDL λ1 is the weight of the Dice loss function; λ2 is the weight of the Dice loss function.

[0038] Secondly, the present invention provides a camouflage target segmentation system based on light intensity and polarization cues, comprising:

[0039] Preprocessing module: used to preprocess the image to be tested to obtain the polarization information map of the image to be tested, which includes a light intensity map, a degree of polarization map and a polarization angle map;

[0040] Multi-layer feature extraction module: used to perform multi-layer feature extraction processing on each polarization information map in sequence to obtain multi-layer modal features of each polarization information map;

[0041] Multi-layer feature compression module: used to perform multi-layer feature compression processing on the multi-layer modal features of each polarization information map to obtain multi-layer compressed features of each polarization information map;

[0042] Multi-layer feature fusion module: used to perform multi-layer feature fusion processing on the compressed features of each polarization information map to obtain the cross-fusion features of adjacent layers of each polarization information map;

[0043] Multi-branch search module: used to perform multi-branch search processing on the cross-fusion features of adjacent layers of each polarization information map to obtain the residual fusion features of each polarization information map;

[0044] The three-modal fusion module is used to perform three-modal fusion processing on the residual fusion features of the polarization information map to obtain a mask image of the camouflaged target.

[0045] Thirdly, a computer device, including a processor and a storage medium;

[0046] The storage medium is used to store instructions;

[0047] The processor is configured to operate according to the instructions to perform the steps of the method according to any of the foregoing.

[0048] Fourthly, a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of any of the methods described above.

[0049] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0050] This invention uses images captured by a polarization camera in a special scene to introduce polarization information, including light intensity and polarization cues. It employs a two-stage fusion method of multi-layer fusion and multi-branch search to solve the problem of segmenting camouflaged targets based on light intensity and spectral polarization cues. This helps the polarization vision system accurately identify camouflaged targets that are highly similar to the current scene. At the same time, it effectively fuses multi-level features of various modalities, applies multi-layer fusion and multi-branch search to multi-level features, and finally adopts a three-modal merging and fusion method to achieve accurate segmentation of camouflaged targets. Attached Figure Description

[0051] Figure 1 This is a flowchart of the camouflaged target segmentation method in Embodiment 1 of the present invention;

[0052] Figure 2 This is a flowchart of the processing of the camouflaged target segmentation network model in Embodiment 1 of the present invention;

[0053] Figure 3 This is a structural diagram of the feature compression coding network block in Embodiment 1 of the present invention;

[0054] Figure 4 This is a structural diagram of the multi-layer feature fusion network block in Embodiment 1 of the present invention;

[0055] Figure 5 This is a structural diagram of the multi-branch search network block in Embodiment 1 of the present invention;

[0056] Figure 6 This is a structural diagram of the three-modal fusion network block in Embodiment 1 of the present invention;

[0057] Figure 7 This is the output test image of the camouflaged target segmentation network model in Embodiment 1 of the present invention;

[0058] Figure 8 This is the output side view of the deep neural network segmentation model in the prior art in Embodiment 1 of the present invention. Detailed Implementation

[0059] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0060] Example 1:

[0061] This embodiment provides a method for segmenting camouflaged targets based on light intensity and polarization cues, specifically including:

[0062] The image to be tested is acquired and preprocessed to obtain a polarization information map of the image to be tested, which includes an intensity map, a degree of polarization map, and a polarization angle map.

[0063] The polarization information map of the image to be tested is input into a pre-trained camouflage target segmentation network model. Multi-layer feature extraction is performed on each polarization information map to obtain multi-layer modal features of each polarization information map. Multi-layer feature compression is performed on the multi-layer modal features of each polarization information map to obtain multi-layer compressed features of each polarization information map. Multi-layer feature fusion is performed on the multi-layer compressed features of each polarization information map to obtain cross-fusion features of each polarization information map. Multi-branch search is performed on the cross-fusion features of each polarization information map to obtain residual fusion features of each polarization information map. Three-modal fusion is performed on the residual fusion features of the polarization information map to obtain the mask image of the camouflage target.

[0064] The above steps mainly include three stages: data acquisition and preprocessing, model building and training, and application. Each stage will be described in detail below.

[0065] I. Data Acquisition and Preprocessing Stage

[0066] In this embodiment, a professional grayscale polarization camera is used as the receiving source to record polarization information in the scene and acquire the image to be tested or the image for training. During preprocessing, a mask containing four polarization directions is used to record the light intensity (IL) in the four polarization directions of 0°, 45°, 90°, and 135°, primarily by using a mask layer covering the camera's sensing surface. 0° ,I 45° ,I 90° and I 135° Then, the Stokes parameters are calculated using the light intensity in four directions. Finally, the light intensity, degree of polarization, and polarization angle of the image are obtained using the Stokes parameters, thus obtaining the polarization information map of the image under test or used to train the network model. Specifically, this includes the light intensity map, the degree of polarization map, and the polarization angle. Figure 3 Polarization information diagram.

[0067] II. Model Building and Training Phases

[0068] In this embodiment, as Figure 2 As shown, the camouflaged target segmentation network model mainly includes a multi-layer feature extraction network block, a feature compression coding network block, a multi-layer feature fusion network block, a multi-branch search network block, and a three-modal fusion network block connected in sequence.

[0069] The multi-layer feature extraction network primarily utilizes the existing Res2Net-50 network, which possesses feature extraction capabilities. Its input consists of three polarization information maps: an intensity map, a polarization degree map, and a polarization angle map containing the camouflaged target. Feature extraction is then performed on these maps to obtain multi-layer modal features E for each polarization information map. Figure 2 In the diagram, the modal feature size of each polarization information map decreases sequentially from bottom to top, and from bottom to top, they are first-level modal features, second-level modal features, third-level modal features, fourth-level modal features, and fifth-level modal features.

[0070] The feature compression coding network block employs multi-scale convolution and dilated convolution to perform multi-layer feature compression (CFE) on the multi-layer modal features of each polarization information map, resulting in multi-layer compressed features for each polarization information map, where each compressed feature has 64 channels. Its network structure is as follows: Figure 3As shown, the modal features E of each polarization information map are first passed through four 1×1 convolutional layers (unless otherwise specified, the 1×1 convolutional layers described in this paper have 1×1 kernels and a stride of 1). Except for the first 1×1 convolutional layer, no other structures are connected. After the other 1×1 convolutional layers, 3×3 convolutional layers (unless otherwise specified, the 3×3 convolutional layers described in this paper have 3×3 kernels and a stride of 1) and 5×5 convolutional layers (unless otherwise specified, the 5×5 convolutional layers described in this paper have 3×3 kernels and a stride of 1) are connected respectively. The convolutional layers are 5×5 kernels with a stride of 1 and 7×7 kernels (unless otherwise specified, the 7×7 convolutional layer described in this paper has a 7×7 kernel with a stride of 1), and then connected to dilated convolutional layers of the corresponding kernel size. The convolutional layers with four branches are then processed by channel splicing and passed through a 1×1 convolutional layer. The result is then added to the modal features processed by the 1×1 convolutional layer. Finally, the ReLU activation function is used to activate the modal features, resulting in compressed features that correspond one-to-one with the modal features in each polarization information map.

[0071] The multi-layer feature fusion network block performs multi-layer feature fusion (CLFM) on the compressed features of each polarization information map, obtaining the cross-fused features of each polarization information map. By introducing an attention mechanism, it alleviates the scale variation of features at different levels and effectively fuses compact features at each level. Its network structure is as follows: Figure 4 As shown, the diagram includes a first convolutional group and a second convolutional group. The first convolutional group consists of a 3×3 convolutional layer, a global max pooling layer (CGMP layer) along the channel direction, a 3×3 convolutional layer, and a sigmoid function layer connected in sequence. The second convolutional group consists of a global max pooling layer (GMP layer), a 1×1 convolutional layer, and a sigmoid function layer connected in sequence. Taking the largest first-level compressed feature F1 and the second-level compressed feature F2 from the previous layer as an example, the second-level compressed feature F2 is first subjected to pre-interpolation (UP×2) to make it the same size as the first-level compressed feature F1. Then, it is added to and multiplied by the first-level compressed feature F1, respectively, to obtain the additive compressed feature and the multiplicative compressed feature. The additive compressed feature and the multiplicative compressed feature are then concatenated through the channels and input into the first convolutional group for spatial attention operation to obtain the common spatial attention map F. SA Public Space Attention Map F SA After multiplying the summative and multiplicative compressed features respectively, channel attention is performed. Finally, the two are concatenated again and convolved (in this paper, unless otherwise specified, the convolutional layer represents a 3×3 convolution), resulting in the cross-fusion feature F of two adjacent layers. This process of upward fusion is continued to obtain the final cross-fusion feature of each polarization information map.

[0072] In a multi-layer feature fusion network block, the common space attention map F SA The calculation formula is:

[0073] F SA =SA(Concat((F1+U(F2)),(F1×U(F2)))),

[0074] In the formula, F SA Here is the common spatial attention map; U(·) represents the upsampling operation; Concat(·) represents the connection operation; SA(·) represents the spatial attention operation; the general definition formula for the spatial attention operation is:

[0075] SA(x)=Sigmoid(conv3(CGMP(conv1(x)))),

[0076] In the formula, Conv1(·) represents a convolutional layer with a kernel size of 1×1; CGMP(·) represents a global max pooling operation along the channel direction; Conv3(·) represents a convolutional layer with a kernel size of 3×3; and Sigmoid(·) represents the Sigmoid activation function.

[0077] In a multi-layer feature fusion network block, the formula for calculating the cross-fusion feature F between adjacent layers is:

[0078] F = Concat(CA(F) SA ×(F1+U(F2))),CA(F SA ×(F1×U(F2)))),

[0079] In the formula, F represents the cross-fusion feature of adjacent layers; CA(·) represents the channel attention operation; and the general definition formula for the channel attention operation is:

[0080] CA(x)=Sigmoid(conv1(GMP(x))),

[0081] In the formula, GMP(·) represents the global max pooling operation; Conv1(·) represents a convolutional layer with a kernel size of 1×1; and Sigmoid(·) represents the Sigmoid activation function.

[0082] In the multi-branch search network block, multi-branch search (MSM) processing is performed on the cross-fusion features of each polarization information map to obtain the residual fusion features of each polarization information map. This process can extract richer global information and improve the performance of camouflaged target detection. Its network structure is as follows: Figure 5 As shown, the cross-fusion features of each polarization information map The process involves three branches: the first branch performs upsampling, the second branch performs convolutional processing, and the third branch performs pooling processing, resulting in high-resolution feature maps F. u Original resolution feature map F c and low-resolution feature map F d High-resolution feature map F u Original resolution feature map F c and low-resolution feature map F d The features were processed using the multi-scale channel attention (MS-CA) mechanism and compared with the high-resolution feature map F that had not undergone MS-CA processing. u Original resolution feature map F c and low-resolution feature map F d Perform multiplication fusion; then, the first branch performs downsampling, the third branch performs upsampling, and the second branch performs no other processing. The results from the three branches are then added together to obtain the convolutional group feature F. ucd Convolutional group features F ucd After convolutional processing by the convolutional layer, the features are then fused with the cross-feedback of each polarization information map. Residual fusion is performed, and finally, the image is processed by a convolutional layer to obtain residual fusion features, which are the residual fusion features F of the intensity map. I Residual fusion feature F of polarization degree map P and the residual fusion feature F of the polarization angle map A Based on the residual fusion feature F of the intensity map I For example, the calculation formula is:

[0083]

[0084] In the formula, F I This refers to the residual fusion characteristics of the intensity map; These are cross-fusion features of the intensity map; U(·) represents upsampling; D(·) represents downsampling; Conv3(·) represents convolution with a kernel size of 3×3; M(·) represents multi-scale channel attention; where, for example... Figure 5 As shown, the general definition formula for multi-scale channel attention (MS-CA) operation is:

[0085] F b =F a ×Sigmoid(L(F a )+g(F a )),

[0086] In the formula, F aHere, L(·) represents the channel attention operation for local features, and g(·) represents the channel attention operation for global features. Their general definition formula is:

[0087]

[0088] In the formula, PWConv1(·) represents reducing the number of channels of the input feature to the original value. 1×1 point convolution; B(·) represents the BatchNorm layer; δ(·) represents the ReLU activation function; PWConv2(·) represents a 1×1 point convolution that restores the number of channels to the same number of channels as the original input; GAP(·) represents the global average pooling operation.

[0089] Finally, the residual fusion features F of the intensity maps were obtained respectively. I Residual fusion feature F of polarization degree map P and the residual fusion feature F of the polarization angle map A .

[0090] In the trimodal fusion network block, the residual fusion features of the polarization information map are subjected to trimodal fusion processing (TFM) to obtain a mask image of the camouflaged target. The trimodal fusion network block can selectively aggregate specific information from each modality to explore key semantic cues in different modalities, enhance feature representation, and obtain an accurate prediction map. Its network structure is as follows: Figure 6 As shown, the residual fusion feature F of the obtained intensity map I Residual fusion feature F of polarization degree map P and the residual fusion feature F of the polarization angle map A After being processed by the third convolutional group, the corresponding F values ​​are obtained respectively. I '、F P 'and F A The third convolutional group consists of sequentially connected convolutional layers, BN layers, and ReLU activation function layers; the results are then merged by the fourth convolutional group to obtain the corresponding A. i A p And A a Ultimately, F I '、F P 'and F A 'and its corresponding A i A p And A a The final mask image is obtained by fusion processing; its calculation formula is as follows:

[0091]

[0092] In the formula, Ψ(·) represents the convolution operation of the third convolutional group; Concat(·) represents the concatenation operation; Conv3(·) represents a convolutional layer with a kernel size of 3×3; M S (·) indicates the Multi-Scale Spatial Attention (MS-SA) operation; pred represents the final mask image.

[0093] The above describes the specific structure of the camouflage target segmentation network model in this embodiment. After construction, the network model needs to be optimized and trained using the PyTorch framework. Both training and testing were performed on a single 6-core computer equipped with an Intel(R) Xeon(R) E5-2609 v3 1.9GHz CPU and an NVIDIA GeForce RTX3090Ti GPU (24GB RAM). The momentum SGD optimizer was used, with weight decay of 5e-4, an initial learning rate of 1e-3, and a momentum of 0.9. Furthermore, the batch size was set to 4, and the learning rate was adjusted using a poly policy with a factor of 0.9. The network was trained for 50 epochs. The specific method for model training is as follows:

[0094] Obtain a training set, which includes polarization maps acquired by multiple professional polarization cameras;

[0095] The polarization maps in the training set are processed according to the camouflage target segmentation method. The parameters of the camouflage target segmentation network model are optimized and adjusted according to the loss function value until the loss function value tends to stabilize, and then training is stopped.

[0096] The training set consists of IP camouflage target segmentation polarization maps (4390 images in total). Various scenes within the polarization maps are randomly segmented into 2930 training images and 1470 test images. Images of different sizes within the polarization maps are uniformly scaled to 352×352 pixels during training, and the image segmentation output is resized to the original size of the input images. The parameters of the feature extraction network are initialized using a pre-trained Res2Net-50 network, while other parameters are randomly initialized.

[0097] This embodiment uses a combination of cross-entropy loss and Dice loss as the loss function, and the calculation formula is as follows:

[0098] L=λ1L wbce +λ2L wDL ,

[0099] In the formula, L is the loss function in this embodiment; L wbce L is the cross-entropy loss function; wDL λ1 is the weight of the cross-entropy loss function, which is a fixed value; λ2 is the weight of the Dice loss function, which is a fixed value.

[0100] After the training images undergo preprocessing in the segmentation method and are input into the camouflaged target segmentation network model for segmentation, a predicted image is obtained. The parameters of the camouflaged target segmentation network model are optimized and adjusted based on the loss function value until the loss function value tends to stabilize, at which point training stops. In this embodiment, to improve the training effect, the predicted images during the training process are supervised by manually annotated mask images corresponding to the training set.

[0101] III. Application Stage

[0102] In this embodiment, 1470 test images were used for testing. To verify the effectiveness of the invention, the results were compared with other state-of-the-art methods in the field. Other deep neural networks used for segmentation in the field (EAFNet, PGSNet, RD3D, MIDD, and SwinNet) were retrained and trained and tested on the same dataset using publicly available code. During the prediction process, the invention uses the acquired grayscale polarization image as the input image. After preprocessing, three modes—light intensity, polarization degree, and polarization angle—are obtained. These are then fed into the network of the invention to finally obtain an accurate segmented image of the camouflaged target. The specific process is shown in Figure 7. To compare the invention with other deep neural networks in the field, the same data source was used for prediction through different deep neural networks. Figure 8 It can be clearly seen that the segmentation results of this invention are more accurate and the edges are clearer than those of other networks, while other networks have false positives and false negatives.

[0103] Furthermore, this invention is applicable to the segmentation of camouflaged targets in various scenarios, and has broad application value in different fields such as manufacturing (apparent defect detection), agriculture (e.g., locust detection), and computer vision (e.g., search and rescue missions). The polarization imaging technology utilized in this invention primarily detects polarization features closely related to the target's physical characteristics by analyzing the polarization state of the reflected and radiated light. This technology can improve the contrast between the target and the background, enriching the image's detail information. It is particularly effective for identifying special materials, far surpassing RGB cameras. Moreover, the method presented in this paper generates more accurate and complete camouflaged target segmentation maps for various sizes of natural camouflaged targets (small camouflaged targets, large camouflaged targets), with clear boundaries and coherent details. This demonstrates the effectiveness and practicality of the method in different scenarios.

[0104] Example 2:

[0105] A camouflaged target segmentation system based on light intensity and polarization cues, comprising:

[0106] Preprocessing module: used to preprocess the image to be tested to obtain the polarization information map of the image to be tested, which includes a light intensity map, a degree of polarization map and a polarization angle map;

[0107] Multi-layer feature extraction module: used to perform multi-layer feature extraction processing on each polarization information map in sequence to obtain multi-layer modal features of each polarization information map;

[0108] Multi-layer feature compression module: used to perform multi-layer feature compression processing on the multi-layer modal features of each polarization information map to obtain multi-layer compressed features of each polarization information map;

[0109] Multi-layer feature fusion module: used to perform multi-layer feature fusion processing on the compressed features of each polarization information map to obtain the cross-fusion features of adjacent layers of each polarization information map;

[0110] Multi-branch search module: used to perform multi-branch search processing on the cross-fusion features of adjacent layers of each polarization information map to obtain the residual fusion features of each polarization information map;

[0111] The three-modal fusion module is used to perform three-modal fusion processing on the residual fusion features of the polarization information map to obtain a mask image of the camouflaged target.

[0112] The specific network structure of each of the above modules can be found in the camouflage target segmentation network model in Example 1, and will not be repeated here.

[0113] Example 3:

[0114] This invention also provides a computer device, including a processor and a storage medium;

[0115] The storage medium is used to store instructions;

[0116] The processor is configured to operate according to the instructions to perform the steps of the following method:

[0117] The image to be tested is acquired and preprocessed to obtain a polarization information map of the image to be tested, which includes an intensity map, a degree of polarization map, and a polarization angle map.

[0118] The polarization information map of the image to be tested is input into a pre-trained camouflage target segmentation network model. Multi-layer feature extraction is performed on each polarization information map to obtain multi-layer modal features of each polarization information map. Multi-layer feature compression is performed on the multi-layer modal features of each polarization information map to obtain multi-layer compressed features of each polarization information map. Multi-layer feature fusion is performed on the multi-layer compressed features of each polarization information map to obtain cross-fusion features of adjacent layers of each polarization information map. Multi-branch search is performed on the cross-fusion features of adjacent layers of each polarization information map to obtain residual fusion features of each polarization information map. Three-modal fusion is performed on the residual fusion features of the polarization information map to obtain the mask image of the camouflage target.

[0119] Example 4:

[0120] This invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the following method:

[0121] The image to be tested is acquired and preprocessed to obtain a polarization information map of the image to be tested, which includes an intensity map, a degree of polarization map, and a polarization angle map.

[0122] The polarization information map of the image to be tested is input into a pre-trained camouflage target segmentation network model. Multi-layer feature extraction is performed on each polarization information map to obtain multi-layer modal features of each polarization information map. Multi-layer feature compression is performed on the multi-layer modal features of each polarization information map to obtain multi-layer compressed features of each polarization information map. Multi-layer feature fusion is performed on the multi-layer compressed features of each polarization information map to obtain cross-fusion features of adjacent layers of each polarization information map. Multi-branch search is performed on the cross-fusion features of adjacent layers of each polarization information map to obtain residual fusion features of each polarization information map. Three-modal fusion is performed on the residual fusion features of the polarization information map to obtain the mask image of the camouflage target.

[0123] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0126] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0127] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for segmenting camouflaged targets based on light intensity and polarization cues, characterized in that, include: The image to be tested is acquired and preprocessed to obtain a polarization information map of the image to be tested, which includes an intensity map, a degree of polarization map, and a polarization angle map. The polarization information map of the image to be tested is input into a pre-trained camouflage target segmentation network model. Multi-layer feature extraction is performed on each polarization information map to obtain multi-layer modal features of each polarization information map. Each modal feature of each polarization information map is compressed to obtain multi-layer compressed features of each polarization information map; Multi-layer feature fusion processing is performed on the multi-layer compressed features of each polarization information map to obtain the cross-fusion features of each polarization information map; Multi-branch search processing is performed on the cross-fusion features of each polarization information map to obtain the residual fusion features of each polarization information map; three-mode fusion processing is performed on the residual fusion features of the polarization information map to obtain the mask image of the camouflaged target. The multi-branch search processing of the cross-fusion features of each polarization information map to obtain the residual fusion features of each polarization information map specifically includes: The cross-fusion features of the light intensity map are subjected to three-branch operations respectively to obtain a high-resolution feature map F u , an original-resolution feature map F c , and a low-resolution feature map F d ; the three-branch operations include up-sampling branch operation, convolution branch operation, and pooling branch operation; High-resolution feature map F u Original resolution feature map F c and low-resolution feature map F d The convolutional group features F are obtained by sequentially performing three branch operations: multi-scale channel attention processing, multiplicative fusion, and addition fusion. ucd ; The convolutional group features F ucd Cross-fusion features of light intensity maps Residual fusion is performed to obtain the residual fusion feature F of the intensity map. I The calculation formula is: ； In the formula, F I This refers to the residual fusion characteristics of the intensity map; It is a cross-fusion feature of the intensity map; U(·) represents an upsampling operation; D(·) represents a downsampling operation; Conv3(·) represents a convolution operation with a kernel size of 3×3; M(·) represents a multi-scale channel attention operation; Based on the cross-fusion characteristics of the polarization degree map, the cross-fusion characteristics of the polarization angle map, and the multi-branch search process, the residual fusion characteristics F of the polarization degree map are obtained respectively. P The residual fusion feature F of the polarization angle map A .

2. The camouflage target segmentation method based on light intensity and polarization cues according to claim 1, characterized in that, The process of acquiring the image to be tested and preprocessing it to obtain a polarization information map of the image to be tested specifically includes: Select the scene to be tested and use a professional grayscale polarization camera to acquire the light intensity in four polarization directions: 0˚, 45˚, 90˚, and 135˚. Based on the light intensity in the four polarization directions, the Stokes parameters are calculated to obtain the polarization information map of the image under test.

3. The camouflage target segmentation method based on light intensity and polarization cues according to claim 1, characterized in that, The multi-layer feature extraction includes using a Res2Net-50 feature extraction network to extract features from each polarization information map, thereby obtaining multi-layer modal features for each polarization information map.

4. The camouflage target segmentation method based on light intensity and polarization cues according to claim 3, characterized in that, The process of performing multi-layer feature fusion on the compressed features of each polarization information map to obtain the cross-fusion features of each polarization information map specifically includes: Based on the compression features of adjacent layers in each polarization information map, the common spatial attention map of each polarization information map is obtained, and the calculation formula is as follows: ， In the formula, This represents the common spatial attention map; F1 and F2 are two compressed features from adjacent layers; U(·) denotes the upsampling operation; Concat(·) denotes the connection operation; SA(·) denotes the spatial attention operation. The common spatial attention map is used as the weight for spatial alignment of the combined features. Channel attention operations are then performed on the resulting spatially aligned features. Finally, the cross-fusion features of adjacent layers of each polarization information map are obtained through a join operation. The calculation formula is as follows: ， In the formula, F represents the cross-fusion feature of adjacent layers; CA(·) represents the channel attention operation; By continuously fusing the compressed features of adjacent layers, the final cross-fusion feature of each polarization information map is obtained. .

5. The camouflage target segmentation method based on light intensity and polarization cues according to claim 4, characterized in that, The process of performing three-modal fusion processing on the residual fusion features of the polarization information map to obtain a mask image of the camouflaged target specifically includes: The residual fusion features of the polarization information map are merged and fused to obtain the mask image of the camouflaged target. The calculation formula is as follows: ， In the formula, Ψ(·) represents the convolution process of a convolutional group, including a Conv layer, a BN layer, and a ReLU layer; Concat(·) represents the concatenation operation; Conv3(·) represents a convolutional layer with a kernel size of 3×3; M S (·) denotes a multi-scale spatial attention operation; pred denotes the final mask image; A i A p And A a The residual fusion features F of the intensity map are respectively I Residual fusion feature F of polarization degree map P The residual fusion feature F of the polarization angle map A Characteristics of the merged result.

6. The method for segmenting camouflaged targets based on light intensity and polarization cues according to any one of claims 1-5, characterized in that, The training method for the camouflaged target segmentation network model includes: Obtain a training set, which includes polarization maps acquired by multiple professional polarization cameras; The polarization maps in the training set are processed according to the camouflage target segmentation method, and the parameters of the camouflage target segmentation network model are optimized and adjusted according to the loss function value until the loss function value tends to stabilize, and then training is stopped. The formula for calculating the loss function is as follows: ， In the formula, L is the loss function in this embodiment; L wbce L is the cross-entropy loss function; wDL λ1 is the weight of the Dice loss function; λ2 is the weight of the Dice loss function.

7. A camouflaged target segmentation system based on light intensity and polarization cues, characterized in that, include: Preprocessing module: used to preprocess the image to be tested to obtain the polarization information map of the image to be tested, which includes a light intensity map, a degree of polarization map and a polarization angle map; Multi-layer feature extraction module: used to perform multi-layer feature extraction processing on each polarization information map in sequence to obtain multi-layer modal features of each polarization information map; Multi-layer feature compression module: used to perform multi-layer feature compression processing on the multi-layer modal features of each polarization information map to obtain multi-layer compressed features of each polarization information map; Multi-layer feature fusion module: used to perform multi-layer feature fusion processing on the compressed features of each polarization information map to obtain the cross-fusion features of adjacent layers of each polarization information map; Multi-branch search module: used to perform multi-branch search processing on the cross-fusion features of adjacent layers of each polarization information map to obtain the residual fusion features of each polarization information map; The three-modal fusion module is used to perform three-modal fusion processing on the residual fusion features of the polarization information map to obtain a mask image of the camouflaged target. The multi-branch search processing of the cross-fusion features of each polarization information map to obtain the residual fusion features of each polarization information map specifically includes: The cross-fusion features of the intensity maps are subjected to three-branch operations to obtain high-resolution feature maps F. u Original resolution feature map F c and low-resolution feature map F d The three-branch operation includes an upsampling branch operation, a convolution branch operation, and a pooling branch operation. High-resolution feature map F u Original resolution feature map F c and low-resolution feature map F d The convolutional group features F are obtained by sequentially performing three branch operations: multi-scale channel attention processing, multiplicative fusion, and addition fusion. ucd ; The convolutional group features F ucd Cross-fusion features of light intensity maps Residual fusion is performed to obtain the residual fusion feature F of the intensity map. I The calculation formula is: ， In the formula, F I This refers to the residual fusion characteristics of the intensity map; It is a cross-fusion feature of the intensity map; U(·) represents an upsampling operation; D(·) represents a downsampling operation; Conv3(·) represents a convolution operation with a kernel size of 3×3; M(·) represents a multi-scale channel attention operation; Based on the cross-fusion characteristics of the polarization degree map, the cross-fusion characteristics of the polarization angle map, and the multi-branch search process, the residual fusion characteristics F of the polarization degree map are obtained respectively. P The residual fusion feature F of the polarization angle map A .

8. A computer device, characterized in that, Including processor and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1 to 6.

Images