Camouflage Target Detection Method and System Based on the Joint of Cross-Scale Global and Local Features

Abstract

The present invention discloses a camouflaged target detection method and system based on the joint of cross-scale global and local features. The method of the present invention includes using an encoder to extract feature maps of M scales from the input image #imgabs0#, inputting them into a multi-branch convolution block to obtain enhanced features, inputting them into a pre-trained decoder for multi-scale fusion to obtain the final fused features, upsampling to obtain the decoded result, and then classifying to obtain the camouflaged target detection result. Among them, the decoder has #imgabs1# layers, and any #imgabs2# layer includes #imgabs3# decoder nodes in total. Each decoder node performs upsampling layer by layer and level by level, fuses through double convolution operations, and then passes through a multi-level joint scanning feature fusion block to obtain the output features of the decoder node. The present invention aims to solve the detection difficulties when the camouflaged target is highly similar to the background, the edges are not obvious, or there are a large number of occlusions in a complex background, and overcome the problem of low detection accuracy of camouflaged targets due to blurred edges and small target sizes.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a camouflaged target detection method and system based on the joint of cross-scale global and local features. Background Art

[0002] Camouflaged target detection aims to identify and segment imperceptible targets in visual scenes, especially those camouflaged objects that are highly similar to the background or occluded by the background. Its main purpose is to quickly and accurately detect these targets, which are widely used in multiple fields such as medical image analysis and ecological protection. In existing camouflaged target detection methods, the detection task of camouflaged targets is usually completed based on the use of local features or edge features of camouflaged targets. For example, the F2-EDNet model enhances multi-scale context features through a feature enhancement module, and combines an edge prediction branch guided by cross-layer features to extract edge features, thereby improving the accuracy of camouflaged target detection. However, in the face of complex backgrounds, too small camouflaged object sizes, or low contrast with the background edge, relying on edge features may lead to missed detections. The Vim model uses a bidirectional state space model (SSM) for data-dependent global visual context modeling, but ignores the preservation of local two-dimensional dependencies. Therefore, the camouflaged target detection method based on the joint scanning and fusion of cross-scale global and local features has become an important approach. To sum up, how to effectively utilize the local features and global features of images at different scales, accurately capture the dependencies at different distances of the images, and achieve high-precision detection of camouflaged targets of different sizes in complex environments has become a key technical problem to be solved urgently. Summary of the Invention

[0003] The technical problem to be solved by the present invention: Aiming at the above problems of the prior art, a camouflaged target detection method and system based on the joint of cross-scale global and local features are provided. The present invention aims to solve the detection difficulties of camouflaged targets that are highly similar to the background, have unclear edges, or have a large number of occlusions in complex backgrounds, and overcome the problem of low accuracy of camouflaged target detection due to blurred edges and small target sizes.

[0004] To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0005] A camouflaged target detection method based on the joint of cross-scale global and local features, comprising the following steps:

[0006] Input the image Use a pre-trained encoder to extract feature maps of M scales;

[0007] Input the feature maps of M scales into a multi-branch convolutional block MDCS to obtain enhanced features;

[0008] Input the enhanced features of M scales into the pre-trained decoder for multi-scale fusion to obtain the final fused feature;

[0009] Upsample the final fused feature to obtain the decoding result, and classify the decoding result to obtain the camouflage target detection result;

[0010] Among them, the decoder has layers, and any layer includes decoder nodes in total. The first decoder node of any layer inputs the enhanced feature of the th scale. For the remaining any decoder nodes, the output feature of the th decoder node of the th layer is upsampled and then concatenated with the output features of the 1st to th decoder nodes of the th layer, and then fused through a double convolution operation, and then passed through a multi-level joint scanning feature fusion block LVBlock to obtain the output feature of this decoder node. The processing of the input feature by the joint scanning feature fusion block LVBlock includes: randomly selecting four directions with the highest preference probability from a preset variety of scanning methods for the input feature, flattening the input feature into a sequence and inputting it into the selective spatial state model SSM to obtain one-dimensional scanning sequences generated by scanning in four directions, and fusing the one-dimensional scanning sequences in four directions through an attention feature fusion block with channel attention and spatial attention, and then concatenating to obtain the output feature of the decoder node.

[0011] Optionally, before extracting the feature maps of M scales from the input image using the pre-trained encoder, it further includes the step of preprocessing the original image to obtain the input image: converting the original image into an RGB three-channel matrix to obtain the image input ; reorganizing the shape or size of the image input to a specified size to obtain the input image

[0012] Optionally, the multi-branch convolution block MDCS includes three parallel convolutional branches with different receptive fields, a feature fusion convolutional layer, a residual connection convolutional layer, and integrates channel attention and spatial attention. The first branch contains a convolutional layer for channel adjustment; the second branch contains a convolutional layer, a convolutional layer, a convolutional layer, and a deformable convolution; the third branch contains a convolutional layer, convolution layer, a convolution layer, and a deformable convolution layer; the output features of two parallel convolution branches including the deformable convolution layer are concatenated along the channel dimension by a feature fusion convolution layer, and then feature fusion and channel adjustment are performed through a 3×3 convolution, and then input into a residual connection convolution layer. The output features of the convolution branch without the dilated convolution layer are added by residual connection and passed through activation function processing, and an enhanced feature with the same spatial size as the input feature map and compressed channel number is obtained by combining channel attention and spatial attention; the output features of the three branches have different receptive fields, and these features are concatenated along the channel dimension, and then passed through a convolution module with a convolution kernel size of for feature fusion and channel adjustment. The features after feature fusion and channel adjustment are added to the input feature map by residual connection and passed through activation function processing, and by combining channel attention and spatial attention, an enhanced feature with the same spatial size as the input feature map and 64 channels is finally obtained .

[0013] Optionally, the encoder is composed of cascaded M-level backbone networks, and each level of backbone network is used to extract a feature map of one scale. N-level adapters are inserted at the input end of the backbone network, and the input features of the backbone network are concatenated with the original input features of the backbone network after passing through the N-level adapters as the input of the subsequent network of the backbone network. The adapter consists of a linear layer for downsampling, an activation function, a random dropout layer, a linear layer for upsampling, and an activation function in sequence.

[0014] Optionally, the backbone network is a Hiera backbone network. The Hiera backbone network includes a normalization module, an attention module, a splicing module, a normalization module, a multi-layer perceptron MLP, and a splicing module connected in sequence. The first splicing module splices and outputs the output features of an attention module and the input features of the Hiera backbone network. The second splicing module is used to splice the output features of the first splicing module and the output features of the multi-layer perceptron MLP as the output features of the Hiera backbone network.

[0015] Optionally, the double convolution operation in the decoder includes two convolution operations, and after each convolution operation, batch normalization and activation function processing are performed in sequence; when using probability to select the four directions with the highest preference probability, the expression of the probability calculation function is:

[0016] ,

[0017] Among them, is the probability of the scanning method in the -th level joint scanning feature fusion block LVBlock, is the weight of the scanning method in the -th level joint scanning feature fusion block LVBlock, is the weight of the scanning method in the -th

[0018] level joint scanning feature fusion block LVBlock, and

[0019] is a set of scanning methods composed of a plurality of preset scanning methods.

[0020]

[0021] Among them, is the total loss function, is the weighted loss of the -th layer decoder node, is the binary cross-entropy loss of the

[0022] ，

[0023] ，

[0024] ， ​

[0025] Among them, and are the height and width of the input image respectively, represents the output image of the last decoder node in the th layer, is the true value of the pixel at the position in the true mask G of the output image , is the predicted value of the pixel at the position in the output image , is the adjustment parameter, is the weight of the pixel at the position in the output image ; is the category, , being 0 represents the background, being 1 represents the foreground; are the model parameters of the network model, is the log probability that the pixel at the position belongs to the category when the given model parameters are provided, is the set of pixels around the position; is the true value of the pixel at the position in the true mask G of the output image is the indicator function, whose value is 1 when the input is True and 0 when the input is False.

[0026] In addition, the present invention also provides a camouflaged target detection system based on the joint of cross-scale global and local features, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the camouflaged target detection method based on the joint of cross-scale global and local features.

[0027] In addition, the present invention also provides a computer-readable storage medium, in which a computer program or instruction is stored, and the computer program or instruction is programmed or configured to execute the camouflaged target detection method based on the joint of cross-scale global and local features through a processor.

[0028] Compared with the prior art, the present invention can mainly achieve the following beneficial effects: Considering that the camouflage target is highly similar to the background, with unclear edges or a large amount of occlusion, object detection of camouflage objects in complex environments is more complex and challenging than traditional prominent object detection. The present invention introduces multi-scale information, provides representations of the original image at different resolutions, and fuses multi-scale features at the same resolution and multi-scale features at different resolutions through tight skip connections between decoder nodes, enabling analysis and processing of images at different scales, and being able to fully extract important clues in the image in complex environments, effectively overcoming the problem of low detection accuracy of camouflage targets caused by variable or small sizes; the present invention fuses global information and local information, realizes the fusion of global features and local features under different scanning methods, can effectively discover the correlation and semantic information between different distances in the image, can more accurately locate the target, and improve the boundary integrity of the detection result; the present invention is applicable to the detection of concealed targets in complex environments, can effectively overcome the blurred edges and size changes of the target, and provides the possibility for accurate recognition and positioning of concealed targets. The present invention can solve the problem of detecting camouflage targets in images where the edges of camouflage targets are unclear or the sizes are uncertain due to complex environments, realize accurate detection of small-sized objects, and overcome the problem of low detection accuracy of camouflage targets caused by boundary and size changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the basic process of the method according to an embodiment of the present invention.

[0030] Figure 2 It is a schematic diagram of the network structure of the encoder in an embodiment of the present invention.

[0031] Figure 3 It is a schematic diagram of the network structure of the adapter in an embodiment of the present invention.

[0032] Figure 4 It is a schematic diagram of the network structure of the decoder in an embodiment of the present invention.

[0033] Figure 5 It is a schematic diagram of the network structure of the decoder node in an embodiment of the present invention.

[0034] Figure 6 It is a schematic diagram of the network structure of the attention feature fusion block in an embodiment of the present invention.

[0035] Figure 7 It is a schematic diagram of the network structure of the multi-branch convolution block MDCS in an embodiment of the present invention.

[0036] Figure 8 It is a schematic diagram of the process of image enhancement in an embodiment of the present invention.

[0037] Figure 9 This is the example test result of the present invention embodiment on the CAMO dataset, where (a1)-(a8) are the target images on the CAMO test set respectively, and (b1)-(b8) are the corresponding mask images GT of (a1)-(a8); (c1)-(c8) are the segmentation results of the method of this embodiment for the corresponding target images of (a1)-(a8). Detailed implementation manners

[0038] To enable those skilled in the art of this technology to better understand the technical solution of the present invention, the technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0039] As Figure 1 shown, the camouflage target detection method based on the joint cross-scale global-local features in this embodiment includes the following steps:

[0040] S1, extracting feature maps of M scales from the input image using a pre-trained encoder;

[0041] S2, inputting the feature maps of M scales into a multi-branch convolutional block MDCS to obtain enhanced features;

[0042] S3, inputting the enhanced features of M scales into a pre-trained decoder for multi-scale fusion to obtain the final fused feature;

[0043] S4, upsampling the final fused feature to obtain a decoded result, and classifying the decoded result to obtain the camouflage target detection result.

[0044] In this embodiment, before extracting feature maps of M scales from the input image using a pre-trained encoder, it further includes the step of preprocessing the original image to obtain the input image: converting the original image into an RGB three-channel matrix to obtain an image input ; reorganizing the shape or size of the image input to a specified size to obtain the input image . For example, in this embodiment, the size of the input image is scaled by bilinear interpolation, and the input label is scaled by bicubic interpolation to avoid the loss of edge information in the label. It should be noted that interpolation is a well-known data processing method, and the required interpolation algorithm can be adopted according to needs. For example, the bilinear interpolation algorithm adopted in this embodiment is an interpolation method based on the weighted average of known pixel values within a local area. It takes into account the changes in pixel values in the horizontal and vertical directions, so as to better retain the details and quality of the image. Specifically, bilinear interpolation will find the four pixel points closest to the target pixel position in the original image, and then perform a weighted average according to the pixel values of these four pixel points to obtain the value of the target pixel. First, interpolation is implemented on the X-axis. Along the , Find two points with the same horizontal coordinate as the insertion point and :

[0045] ,

[0046] ,

[0047] in, and Points and The interpolation calculation result in the X-axis direction, point The coordinates are , The coordinates are , The coordinates of the point to be inserted are, ~ These are four points near the insertion point, with coordinates corresponding to Then use the interpolation obtained in the previous step , , perform linear interpolation in the vertical direction and then interpolate in the Y axis:

[0048] ,

[0049] in, This is the value of the insertion point obtained by final interpolation. For example, in this embodiment, the input image will be scaled to a specified size based on this. , indicating that the number of channels is 3 and the image size is , get the input image .

[0050] In this embodiment, the input image The feature maps of M scales extracted using the pre-trained encoder are expressed as As an optional implementation, in this embodiment The value is 4, and the preprocessed image in step S2 Input the trained encoder to encode the features at 4 scales , and there is:

[0051] ,

[0052] Among them, the subscript Indicates the Layer scale, Indicates the The number of channels at the layer scale, and Represent the height and width of the feature map respectively, and Both values are 352

[0053] As Figure 2 shown, the encoder in this embodiment is composed of a backbone network. N-level adapters are inserted at the input end of the backbone network to achieve efficient parameter fine-tuning. After passing through the N-level adapters, the input features of the backbone network are concatenated with the original input features of the backbone network and used as the input of the subsequent network of the backbone network. As Figure 3 shown, the adapter consists of a linear layer for downsampling, an activation function, a dropout layer, a linear layer for upsampling, and an activation function connected in sequence. By wrapping each multi-scale block of the encoder into an adapter, the feature extraction ability of the encoder can be enhanced, and layer-scale features are output.

[0054] The backbone network in this embodiment can select the backbone network that can extract feature maps of M scales as needed. For example, as an optional implementation manner, the backbone network in this embodiment is the Hiera backbone network pre-trained by SAM2. The Hiera backbone network includes a normalization module, an attention module, a concatenation module, a normalization module, a multi-layer perceptron MLP, and a concatenation module connected in sequence. The first concatenation module concatenates and outputs the output features of an attention module and the input features of the Hiera backbone network. The second concatenation module is used to concatenate the output features of the first concatenation module and the output features of the multi-layer perceptron MLP as the output features of the Hiera backbone network. Hiera is a new type of hierarchical vision transformer (Vit), which constructs the model only using standard Vit blocks and learns spatial biases through pre-training tasks (such as masked autoencoder MAE), rather than relying on complex architecture designs. Hiera adopts an efficient multi-scale design, capturing the details and global information of the image by using feature maps of different resolutions at different stages. In addition, other encoders can also be used.

[0055] As Figure 4 shown, the decoder in this embodiment has layers, and any layer includes a total of decoder nodes. The first decoder node of any layer inputs the enhanced features of the th scale. As Figure 5 shown, for the remaining any th decoder node, the output features of the th decoder node of the th layer are upsampled and then concatenated with the output features of the The first to The output features of the decoder nodes are concatenated and then fused through a double convolution operation, and then passed through a multi-level joint scanning feature fusion block LVBlock to obtain the output features of the decoder nodes. The processing of the input features by the joint scanning feature fusion block LVBlock includes: for the input features, Four directions with the highest preference probability are selected from a preset variety of scanning methods with probability. After the input features are tiled in sequence, they are input into the selective spatial state model SSM to obtain one-dimensional scanning sequences generated by scanning in four directions. The one-dimensional scanning sequences in the four directions are fused through an attention feature fusion block with channel attention and spatial attention to achieve global and local feature fusion, and then concatenated to obtain the output features of the decoder nodes. In this embodiment, the image features Are input into the decoder for upsampling layer by layer and feature fusion to obtain the fused features of each decoder node And the final output image features of the decoder ; There are Layers in total for the decoder. Each decoder node is represented as , and the output features corresponding to the decoder node are denoted as , and the features Are respectively input into the decoder node . For multi-layer input features Are upsampled layer by layer and feature fusion is performed to obtain the fused features Of each decoder node, where Represents the downsampling layer along the encoder, and Represents the convolutional layer along the fully connected block of the skip connection. As Figure 3 Shown, there are four layers in total for the decoder in this embodiment. The features , , , Are respectively input into the decoder nodes , , , , as the output features , , , Of the corresponding decoder nodes. As Figure 5 Shown, the non-first-layer decoder nodes All perform upsampling, feature connection, and joint scanning, including: the multi-layer features Are first respectively input into the first node Of the corresponding Layer of the decoder as the output features Of this node; Inside the decoder, for the features Perform bilinear interpolation upsampling to double the width and height of the image, obtaining the upsampled features ; (In this embodiment, the output feature map sizes corresponding to the decoder nodes of layers 1 to 4 are ). The upsampled features are fused with the features of layer to obtain the intermediate features of the encoder node . The fusion process includes a double convolution operation, followed by batch normalization and activation function processing each time, reducing the number of channels to 64; among them, the expression of the batch normalization function is:

[0056] ,

[0057] In the above formula, is each pixel of the input feature map, is the input mean, is the input variance, is a small constant used to prevent the denominator from being zero. The activation function is expressed as:

[0058] ,

[0059] In the above formula, is each pixel of the input feature map. The intermediate features are input into two layers of the joint scanning feature fusion block LVBlock to obtain the node features at .

[0060] As Figure 6 shown, the attention feature fusion block includes a global branch and a local branch. The global branch refers to performing global average pooling on the entire input feature sequence and then performing linear dimensionality reduction, so that the value of each channel represents the information of the entire feature map. The processing steps in the global branch include:

[0061] Step 1, perform global average pooling on the input feature map to obtain a feature vector with a shape of . Among them, the function expression of global average pooling is:

[0062] ,

[0063] In the above formula, is the feature vector of the input feature map at position , is the feature vector after global average pooling.

[0064] Step 2, through a fully connected layer, the output shape is , where is a scaling ratio. In this embodiment, is defaulted to 0.125, which is used to reduce the number of parameters. This is the shape of the decoder node feature map. Then, through activation function for activation processing;

[0065] Step 3, linearly reduce the dimensionality of the feature vector activated by the activation function through a fully connected layer, and map the feature vector back to shape, and use activation function to generate channel attention weights.

[0066] The local branch refers to linearly reducing the dimensionality for each token in the feature sequence to retain local features. The processing steps in the local branch include:

[0067] Step 1, pass the input feature map through a fully connected layer to obtain a feature map with a shape of , followed by activation function;

[0068] Step 2, for each token in the local features output by the activation function in the local branch, concatenate the global features output in Step 2 of the global branch with the local features output in Step 2 of the local branch, so that each token in the concatenated global features has both global and local features;

[0069] Step 3, calculate the spatial attention weights from the concatenated global features.

[0070] Finally, multiply the channel attention and the spatial attention to obtain the final attention weights. Apply the final attention weights to the input feature map to obtain a feature with an output shape of .

[0071] In this embodiment, the double convolution operation in the decoder includes two convolution operations, and after each convolution operation, batch normalization and activation function processing are performed in sequence.

[0072] The joint scanning feature fusion block LVBlock is a selective spatial state model with four scanning directions and spatial and channel attention feature fusion blocks. In this embodiment, when the joint scanning feature fusion block LVBlock uses probability to select the four directions with the highest preference probability, the calculation function expression of the probability is:

[0073] ,

[0074] wherein, is the probability of the th level joint scanning feature fusion block LVBlock regarding the scanning method , probability, is the weight of the th level joint scanning feature fusion block LVBlock regarding the scanning method , is the weight of the th level joint scanning feature fusion block LVBlock regarding the scanning method , is a set of scanning methods composed of a plurality of preset scanning methods.

[0075] In this embodiment, the preset multiple scanning methods include: row-by-row scanning: scanning the image row by row from left to right, reverse row-by-row scanning: scanning the image row by row from right to left, column-by-column scanning: scanning the image column by column from top to bottom, reverse column-by-column scanning: scanning the image column by column from bottom to top, 2×2 local scanning: performing local scanning with a 2×2 window size, reverse 2×2 local scanning: performing reverse local scanning with a 2×2 window size, 7×7 local scanning: performing local scanning with a 7×7 window size, reverse 7×7 local scanning: performing reverse local scanning with a 7×7 window size. Through the 8 scanning methods in the preset set of scanning methods, local features and global features are respectively captured, and the four directions with the highest preference probability are selected from the 8 scanning methods at this layer, thereby constructing a search space , where represents the number of LVBlocks. A differentiable search mechanism is set, and probability represents the selection preference of each direction. Finally, the four directions with the highest probability are selected as the scanning directions of this layer to generate corresponding one-dimensional scanning sequences. The feature sequences in the four directions are respectively input into the spatial and channel attention feature fusion block for feature merging to obtain the output feature . Feature merging can be expressed as:

[0076] ,

[0077] wherein, represents the th layer of the joint scanning feature fusion block LVBlock, represents the scanning result obtained by weighted summation of the scanning results in the four directions of the th layer, represents the probability of the th layer and the th selected scanning direction, Indicates according to a specific scanning direction for the input feature after scanning processing to obtain the output. For the feature merging in the joint scanning feature fusion block LVBlock, a combination of channel attention and spatial attention is adopted. The layer output feature is used as the layer input feature, and the layer operations of the joint scanning feature fusion block LVBlock are repeated. Finally, the final node output feature can be obtained. It should be noted that the spatial state model (SSM) used in this embodiment is specifically selected as the Selective State Space Model. The Selective State Space Model introduces input-dependent parameterization and a selective mechanism compared to the traditional spatial state model, solving the limitations of the traditional model in dealing with complex dynamic systems, so that the model can more flexibly adapt to the changes of input data. Finally, for the node feature at the decoding operations of each decoder node are repeated, and finally the features of each decoder node can be obtained; the decoder finally outputs the fused feature , denoted as the output feature . In this embodiment, the decoder output feature is input into a two-dimensional convolutional layer to adjust the number of channels to obtain the decoding result and classify to obtain the camouflage target detection result.

[0078] In step S3 of this embodiment, the M types of scale image features are respectively input into the multi-branch convolutional block MDCS to obtain the image feature ; in this embodiment, M = 4. As Figure 7 shown, the multi-branch convolutional block MDCS includes three parallel convolutional branches with different receptive fields, a feature fusion convolutional layer, a residual connection convolutional layer and incorporates channel attention and spatial attention. The first branch contains a convolutional layer for channel adjustment; the second branch contains convolutional layer,[[]] convolutional layer,[[]] convolutional layer and a deformable convolution; the third branch contains convolutional layer,[[]] convolutional layer,[[]] a convolutional layer and a deformable convolutional layer; the output features of two parallel convolutional branches containing the deformable convolutional layer are concatenated along the channel dimension through a feature fusion convolutional layer and then undergo feature fusion and channel adjustment through a 3×3 convolution, and then are input into the residual connection convolutional layer to be added to the output features of the convolutional branch without the dilated convolutional layer through residual connection and passed through activation function processing, and an enhanced feature with the same spatial size as the input feature map and a compressed number of channels is obtained by combining channel attention and spatial attention. The output features of the three branches have different receptive fields, and these features are concatenated along the channel dimension and then passed through a convolutional module with a convolutional kernel size of for feature fusion and channel adjustment. The fused features are added to the input feature map through residual connection and passed through activation function processing, and by combining channel attention and spatial attention, an enhanced feature with the same spatial size as the input feature map and 64 channels is finally obtained .

[0079] This embodiment also includes training an end-to-end image segmentation model composed of an encoder, a multi-branch convolutional block MDCS, a decoder, an upsampling module for upsampling the final fused features to obtain a decoded result, and a classifier for classifying the decoded result, which is supervised by a real mask G, and the functional expression of the training loss function used during training is:

[0080] ,

[0081] where is the total loss function, is the weighted loss of the decoder node at the layer, is the binary cross-entropy loss of the decoder node at the layer, and there is:

[0082] ，

[0083] ，

[0084] ，

[0085] where and are the height and width of the input image respectively, represents the output image of the last decoder node at the layer, is the output image in the true mask G the true value of the pixel at the is the output image in the predicted value of the pixel at the is the adjustment parameter is the output image in the weight of the pixel at the is the category , when it is 0, it represents the background when it is 1, it represents the foreground is the model parameter of the network model represents the given model parameter at this time the probability that the pixel at the belongs to the category is the set of pixels around the is the output image in the true mask G the true value of the pixel at the is the indicator function. When the input is True, the value of the indicator function is 1. When the input is False, the value of the indicator function is 0. In the training phase, the segmentation output of the model adopts a deep supervision mechanism, and the segmentation output of each stage is input into the loss function and supervised by the true mask G. The loss function combines weighted loss and binary cross-entropy loss as the training objective In the numerator of, the weighted intersection is calculated, that is, the weighted sum of the pixels correctly predicted by the model as the positive class. In the denominator, the weighted union is calculated, that is, the weighted sum of the pixels predicted by the model as the positive class or with the true label as the positive class; in this embodiment the value of is 4 , , correspond to the decoder nodes , , of the output image

[0086] It should be noted that as Figure 8 shown, in the training phase, this embodiment also includes the samples of the input image are enhanced to obtain new samples of the input image , including: S101, set the random probability , if Then input the samples of the image into the image Perform a mirror symmetry transformation along the horizontal axis of the image; if Then do not flip the image and directly return the original data; for example, in this embodiment , perform a mirror symmetry transformation on the image to obtain an enhanced image ; S102, set a random probability , if Then perform a vertical flip transformation on the image along the horizontal axis of the image; if Then do not flip the image and directly return the original data; for example, in this embodiment , perform a mirror symmetry transformation on the image to obtain an enhanced image ; S103, convert the enhanced image into a tensor , and normalize the pixel value range from to , and convert the data format from to ; for example, in this embodiment, convert an image of size into a tensor of size ; S104, perform standardization on each channel of the image using predefined mean and standard deviation. For example, in this embodiment, the mean is set to , and the standard deviation is set to ; obtain the input feature image .

[0087] To verify the effectiveness of the camouflage target detection method based on the joint cross-scale global-local features in this embodiment, in this embodiment, model training and testing are performed on the well-known CAMO dataset to obtain the optimal image segmentation model. The CAMO dataset is designed specifically for the camouflaged object segmentation task. The camouflaged object images consist of 1250 images (1000 images in the training set and 250 images in the test set). The existing F2-EDNet method is used as a comparison for the method in this embodiment, and structural similarity metric ( ), enhancement metric ( ), harmonic mean of precision and recall ( ), mean absolute error ( ) to measure the recognition accuracy. The higher the structural similarity measure, enhancement measure, and adaptive weight index, the better the segmentation effect, while the smaller the mean absolute error value indicates the smaller the difference between the predicted result and the true result; the structural similarity measure mainly evaluates the performance by comparing the structural similarity between the predicted segmentation result and the true segmentation; the enhancement measure is an index used to evaluate the performance of image segmentation algorithms, especially for the measurement of boundary accuracy and regional similarity. It combines concepts such as structural similarity, regional similarity, and mutual information in information theory, and can more comprehensively evaluate the accuracy of image segmentation results; the harmonic mean of precision and recall takes into account the imbalance of sample categories of each class; the mean absolute error is the average of the absolute values of the differences between the predicted values and the true values. In the camouflaged target detection task, it measures the average distance between the predicted boundary position and the true boundary position. The final result obtained is shown in Table 1.

[0088] Table 1 Comparison table of recognition results of the method of this embodiment and the F2-EDNet method under the CAMO dataset

[0089]

[0090] As can be seen from Table 1, the structural similarity measure ( ) of the hidden target segmentation of the method of this embodiment, the enhancement measure ( ), the harmonic mean of precision and recall ( ), and the mean absolute error ( ) are all higher than the existing F2-EDNet method.

[0091] Figure 9 The following is the instance test effect of this embodiment on the CAMO dataset. Among them, (a1) to (a8) are the target images on the CAMO test set respectively, and (b1) to (b8) are the corresponding mask images GT of (a1) to (a8); (c1) to (c8) are the segmentation results of the method of this embodiment for the corresponding target images of (a1) to (a8). Among them, the mask image GT is a mask image corresponding to the image to be segmented, where the significant regions are marked as white or high brightness values, and the non-significant regions are marked as black or low brightness values. This mask image provides the correct annotation of the significant regions for the model, and is used to train and evaluate the performance of the model. It can be seen that the method of this embodiment has good detection effects in the cases where the target size is small, the light is blurred, the target is in a complex environment and hidden in the environment, fully explores the correlation between local features, extracts important semantic information in the input, can more accurately locate the target, and improves the boundary integrity of the segmentation result.

[0092] In summary, in the camouflaged object detection task, the changes in object categories and the diversity of complex scenes make the detection of target objects more challenging compared to traditional salient object detection or other segmentation tasks. The camouflaged object detection method based on the joint cross-scale global and local features in this embodiment considers the different expressions of the scene at different scales and adopts a multi-branch scanning method to capture the local dependencies of the image, which helps to improve the understanding and judgment of target objects. First, the multi-scale information of the image is extracted through a hierarchical decoder, and then a multi-branch convolutional block is used to enhance the feature representation. Next, in the hierarchical cross-scale decoder, the cross-scale semantics are further mined through strategies of grouping, mixing, and fusing. The camouflaged object detection method based on the joint cross-scale feature global and local scanning fusion in this embodiment utilizes the correlation between different scales at the same time, extracts semantic information according to the correlation between channels and the selective scanning mechanism, and provides an accurate and effective solution for camouflaged object detection in complex environments.

[0093] In addition, this embodiment also provides a camouflaged object detection system based on the joint cross-scale global and local features, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the camouflaged object detection method based on the joint cross-scale global and local features.

[0094] In addition, this embodiment also provides a computer-readable storage medium, in which a computer program or instruction is stored, and the computer program or instruction is programmed or configured to execute the camouflaged object detection method based on the joint cross-scale global and local features through a processor.

[0095] In addition, this embodiment also provides a computer program product, including a computer program or instruction, and the computer program or instruction is programmed or configured to execute the camouflaged object detection method based on the joint cross-scale global and local features through a processor.

[0096] Those skilled in the art should understand that the technical solutions provided by the present invention can be in the form of a method, a system, or a computer program product. Therefore, the present invention can be implemented in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can be in the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code. The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means realizes the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks. These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0097] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements should also be regarded as within the protection scope of the present invention.

Claims

1. A camouflaged target detection method based on the joint of cross-scale global and local features, characterized in that, Including the following steps: The input image Use a pre-trained encoder to extract feature maps of M scales; Input feature maps of M scales into the multi-branch convolutional block MDCS to obtain enhanced features; Input the enhanced features of M scales into the pre-trained decoder for multi-scale fusion to obtain the final fused features; Upsample the final fused features to obtain the decoding result, and classify the decoding result to obtain the camouflage target detection result; Among them, the decoder has a total of layers, and any layer includes a total of decoder nodes. The first decoder node of any layer inputs enhanced features of types. The remaining any decoder nodes upsample the output features of the th decoder node of the th layer, then splice them with the output features of the 1st to th decoder nodes of the th layer, and fuse them through a double convolution operation, and then obtain the output features of the decoder node through a multi-level joint scanning feature fusion block LVBlock. The processing of the input features by the joint scanning feature fusion block LVBlock includes: using probability to select the four directions with the highest preference probability from a preset variety of scanning methods for the input features, flattening the input features into a sequence and inputting them into a selective spatio-temporal model SSM to obtain one-dimensional scanning sequences generated by scanning in four directions, fusing the one-dimensional scanning sequences in the four directions through an attention feature fusion block with channel attention and spatial attention to achieve the fusion of global and local features, and then splicing them to obtain the output features of the decoder node.

2. The camouflage target detection method based on the joint of cross-scale global and local features according to claim 1, characterized in that Before extracting the feature maps of M scales from the input image using the pre-trained encoder, it further includes the step of preprocessing the original image to obtain the input image: converting the original image into an RGB three-channel matrix to obtain the image input ; Reorganizing the shape or size of the image input to a specified size to obtain the input image .

3. The camouflage target detection method based on the joint cross-scale global-local features according to claim 1, wherein, The multi-branch convolution block MDCS includes three parallel convolution branches with different receptive fields, a feature fusion convolution layer, a residual connection convolution layer, and incorporates channel attention and spatial attention. The first branch contains a convolution layer for channel adjustment; the second branch contains convolution layer, convolution layer, convolution layer, and a deformable convolution; the third branch contains convolution layer, convolution layer, convolution layer, and a deformable convolution layer; the output features of the two parallel convolution branches containing the deformable convolution layer are concatenated along the channel dimension through the feature fusion convolution layer and then undergo feature fusion and channel adjustment through a 3×3 convolution, and then are input into the residual connection convolution layer and added to the output features of the convolution branch without the dilated convolution layer through residual connection and passed through activation function processing, and an enhanced feature with the same spatial size as the input feature map and compressed channel number is obtained by combining channel attention and spatial attention; the output features of the three branches have different receptive fields, these features are concatenated along the channel dimension, and then undergo feature fusion and channel adjustment through a convolution module with a convolution kernel size of , the features after feature fusion and channel adjustment are added to the input feature map through residual connection, and passed through activation function processing, and finally an enhanced feature with the same spatial size as the input feature map and a channel number of 64 is obtained by combining channel attention and spatial attention , where is the number of scales.

4. The camouflaged target detection method based on the joint of cross-scale global and local features according to claim 1, wherein The encoder is composed of an M-level backbone network in cascade, and each level of the backbone network is used to extract a feature map of a certain scale. An N-level adapter is inserted at the input end of the backbone network, and the input features of the backbone network are concatenated with the original input features of the backbone network after passing through the N-level adapter as the input of the subsequent network of the backbone network. The adapter consists of a linear layer for downsampling, an activation function, a dropout layer, a linear layer for upsampling, and an activation function in sequence.

5. The camouflage target detection method based on the joint cross-scale global-local features according to claim 4, wherein The backbone network is the Hiera backbone network, and the Hiera backbone network includes a normalization module, an attention module, a splicing module, a normalization module, a multi-layer perceptron MLP, and a splicing module connected in sequence. The first splicing module splices and outputs the output features of an attention module and the input features of the Hiera backbone network, and the second splicing module is used to splice the output features of the first splicing module and the output features of the multi-layer perceptron MLP as the output features of the Hiera backbone network.

6. The camouflaged target detection method based on the joint of cross-scale global and local features according to claim 1, wherein The double convolution operation in the decoder includes two convolution operations, and after each convolution operation, batch normalization and activation function processing are performed in sequence; the use of When probabilistically selecting the four directions with the highest preference probabilities, The functional expression for calculating the probability is: ， Among them, is the probability of the scanning method in the -th level joint scanning feature fusion block LVBlock, probability, is the weight of the scanning method in the -th level joint scanning feature fusion block LVBlock, is the weight of the scanning method in the -th level joint scanning feature fusion block LVBlock, is a set of scanning methods composed of a preset variety of scanning methods.

7. The camouflage target detection method based on the joint of cross-scale global and local features according to claim 1, wherein, The preset multiple scanning methods include: row-by-row scanning: scanning the image row by row from left to right, reverse row-by-row scanning: scanning the image row by row from right to left, column-by-column scanning: scanning the image column by column from top to bottom, reverse column-by-column scanning: scanning the image column by column from bottom to top, 2×2 local scanning: performing local scanning with a 2×2 window size, reverse 2×2 local scanning: performing reverse local scanning with a 2×2 window size, 7×7 local scanning: performing local scanning with a 7×7 window size, reverse 7×7 local scanning: performing reverse local scanning with a 7×7 window size.

8. The camouflage target detection method based on the joint of cross-scale global and local features according to claim 1, wherein It also includes training the end-to-end image segmentation model composed of an encoder, a multi-branch convolutional block MDCS, a decoder, an upsampling module for upsampling the final fused features to obtain the decoding result, and a classifier for classifying the decoding result, which is supervised by the real mask G, and the functional expression of the training loss function used during training is: ， Among them, is the total loss function, is the number of scales, is the weighted loss of the decoder node at the th layer, is the binary cross-entropy loss of the decoder node at the th layer, and there is: ， ， ， Wherein, and are the height and width of the input image respectively, represents the output image of the last decoder node of the th layer, is the output image in the true mask G of the true value of the pixel at the position, is the predicted value of the pixel at the position in the output image , is the adjustment parameter, is the weight of the pixel at the position in the output image ; is the category, , being 0 represents the background, being 1 represents the foreground; are the model parameters of the network model, is the log probability that the pixel at the position belongs to the category when the given model parameters are provided, is the set of pixels around the position; is the true value of the pixel at the position in the true mask G of the output image ; is the indicator function, which has a value of 1 when the input is True and a value of 0 when the input is False.

9. A camouflaged target detection system based on the joint of cross-scale global and local features, comprising a microprocessor and a memory connected to each other, characterized in that, The microprocessor is programmed or configured to execute the camouflage target detection method based on the joint of cross-scale global and local features according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instruction is programmed or configured to execute the camouflage target detection method based on the joint of cross-scale global and local features according to any one of claims 1 to 8 through a processor.

Images