Camouflage target identification system and method based on feature aggregation, medium and equipment

Abstract

The embodiment of the invention discloses a camouflage target recognition system and method based on feature aggregation, a medium and equipment, and the system comprises an encoder module which comprises a preset number of encoders and is used for obtaining a feature vector of an initial image; the target area amplification module comprises a preset number of amplification modules and is used for carrying out secondary convolution and batch normalization operation on the feature vectors to obtain amplification features, and the amplification features comprise fusion amplification features and highest-layer amplification features; the decoder module comprises a preset number of multi-head self-attention modules, and is used for obtaining a foreground feature map and a background feature map of the current level amplification feature through matrix multiplication and element multiplication operations, obtaining a segmentation map of the current level amplification feature through independent convolution and depth separable convolution operations, and outputting the segmentation map of the current level amplification feature; and aggregating the foreground feature map, the background feature map and the segmentation map, and obtaining a segmentation prediction map of the camouflage target in combination with the amplified features output by the previous level, thereby obtaining a more refined prediction result.

Description

Technical Field

[0001] The present invention relates to the technical field of camouflaged target recognition, and in particular to a camouflaged target recognition system, method, medium and device based on feature aggregation. Background Art

[0002] With the development of computer vision, the camouflaged target recognition task has attracted more and more attention as a more challenging segmentation task. Camouflaged target recognition aims to segment targets hidden in the background and is applied to medical image analysis, agricultural pest detection, and military target detection. Biological studies have shown that because camouflaged objects are highly similar to the surrounding environment, human visual perception can be easily deceived by various camouflage strategies, which makes traditional target detection methods perform poorly in the process of identifying camouflaged targets.

[0003] The existing camouflaged target recognition algorithms are based on the CNN (Convolutional Neural Network) architecture, including: SINet (Search Identify Network) with a two-stage search and recognition strategy and RefCOD (Referring Camouflaged Object Detection) that uses salient targets as references to assist in camouflaged target recognition. However, recent studies have shown that the CNN architecture loses structural information during feature extraction and the actual receptive field is smaller than the theoretical receptive field. Therefore, CNN-based camouflaged target detection methods usually cannot fully capture global information. Summary of the invention

[0004] The embodiments of the present application provide a camouflaged target recognition system, method, medium and device based on feature aggregation to solve the problem that the camouflaged target detection method cannot fully capture global information.

[0005] In a first aspect, an embodiment of the present application provides a camouflaged target recognition system based on feature aggregation, the system comprising an internally cascaded encoder module, a target area magnification module, and a decoder module; The encoder module includes a preset number of encoders, which are used to obtain the feature vector of the initial image.

[0006] The target area magnification module includes a preset number of magnification modules, which are used to perform secondary convolution and batch normalization operations on the feature vector to obtain magnification features, and the magnification features include fusion magnification features and top-level magnification features.

[0007] The decoder module includes a preset number of multi-head self-attention modules, which are used to obtain the foreground feature map and the background feature map of the enlarged feature of the current level through matrix multiplication and element multiplication operations, obtain the segmentation map of the enlarged feature of the current level through independent convolution and depth-separable convolution operations, aggregate the foreground feature map, the background feature map and the segmentation map, and combine the enlarged features output by the previous level to obtain the segmentation prediction map of the camouflaged target.

[0008] Wherein, the amplification modules include a dual-branch amplification module and a single-branch amplification module; The dual-branch amplification module is used to use the feature vector obtained by the encoder of the adjacent layer as the dual-branch input, respectively obtain the compressed feature vector of each branch, and upsample the compressed feature vector, splice the upsampled compressed feature vectors of the two branches, and then perform a convolution and batch normalization operation to obtain the fused amplified feature.

[0009] The single-branch amplification module is used to perform a convolution and batch normalization operation on the feature vector obtained by the encoder at the highest level, and then perform a convolution operation again to obtain the highest-level amplified features.

[0010] Wherein, the decoders all include a foreground self-attention head, a background self-attention head and a standard self-attention head; The foreground self-attention head and the background self-attention head are used to obtain prediction masks according to the enlarged features of the current level respectively. The query matrix and key matrix of the prediction mask and the enlarged features obtain the channel attention map through element multiplication and matrix multiplication, and then perform matrix multiplication with the value matrix to obtain the foreground feature map and the background feature map of the current level.

[0011] The standard self-attention head is used to perform independent convolution and depth-separable convolution operations on the enlarged features of the current level, determine the attention of multiple convolution heads, and obtain the segmentation map of the current level according to the attention of multiple convolution heads.

[0012] Wherein, the foreground feature map is based on Determine, among which, MA F is the foreground feature map, V F is the foreground value matrix, Q F is the foreground query matrix, T is the transposed amount, K F is the foreground key matrix, α is a learnable parameter, and softmax() is the normalized exponential function.

[0013] The background feature map is based on Determine, among which, MA B is the background feature map, V Bis the background value matrix, Q B is the background query matrix, T is the transposed amount, K B is the background key matrix, α is a learnable parameter, and softmax() is the normalized exponential function.

[0014] In a second aspect, an embodiment of the present application provides a camouflaged target recognition method based on feature aggregation, the method comprising: Get the feature vector of the initial image.

[0015] Performing secondary convolution and batch normalization operations on the feature vector to obtain amplified features, wherein the amplified features include fused amplified features and top-level amplified features.

[0016] Through matrix multiplication and element multiplication operations, the foreground feature map and background feature map of the enlarged feature of the current level are obtained, and through independent convolution and depth-separable convolution operations, the segmentation map of the enlarged feature of the current level is obtained. The foreground feature map, the background feature map and the segmentation map are aggregated, and combined with the enlarged feature output of the previous level, a segmentation prediction map of the camouflaged target is obtained.

[0017] The step of obtaining the feature vector of the initial image specifically includes: The initial image is divided into a plurality of image blocks, and each of the image blocks is linearly projected to obtain an image block with spatial position embedding.

[0018] A feature size redefinition operation is performed on the image block with spatial position embedding to obtain a first-level feature vector.

[0019] A feature size redefinition operation is performed on the feature vectors of the first level until a preset number of feature vectors are obtained.

[0020] The feature vector is subjected to secondary convolution and batch normalization operations to obtain a magnified feature, wherein the magnified feature includes a fused magnified feature and a top-level magnified feature, specifically including: The feature vectors of adjacent levels are used as dual-branch inputs to obtain compressed feature vectors of each branch respectively, and up-sample the compressed feature vectors. The up-sampled compressed feature vectors of the two branches are concatenated and then subjected to a convolution and batch normalization operation to obtain fused and amplified features.

[0021] After performing a convolution and batch normalization operation on the feature vector of the highest level, a convolution operation is performed again to obtain the highest level amplified features.

[0022] Among them, the foreground feature map and the background feature map of the enlarged feature of the current level are obtained through matrix multiplication and element multiplication operations, the segmentation map of the enlarged feature of the current level is obtained through independent convolution and depth-separable convolution operations, the foreground feature map, the background feature map and the segmentation map are aggregated, and the enlarged feature output of the previous level is combined to obtain the segmentation prediction map of the camouflaged target, which specifically includes: Independent convolution and depth-separable convolution operations are performed on the enlarged features of the current level, multi-convolution head attention is determined, and a segmentation map of the current level is obtained according to the multi-convolution head attention.

[0023] The enlarged features of the current level generate a prediction map of the current level according to a prediction loss function under the supervision of the initial input mask.

[0024] A convolution operation is performed on the prediction map of the current level to obtain a prediction mask.

[0025] After normalizing the amplified features of the current level, independent convolution and depth-separable convolution operations are performed again to obtain the query matrix, key matrix and value matrix.

[0026] The prediction mask of the current level is element-wise multiplied with the query matrix and the key matrix respectively to obtain a query matrix mapping and a key matrix mapping.

[0027] The key matrix mapping is matrix transposed and then matrix multiplied with the query matrix mapping to obtain the channel attention map of the current level.

[0028] After matrix multiplication of the channel attention map of the current level and the value matrix, the foreground feature map and the background feature map of the current level are obtained.

[0029] The segmentation map of the current level is added to the foreground feature map and the background feature map to obtain the global interaction features of the current level.

[0030] The global interaction features of the current level are aggregated with the fused amplified features of the previous level to obtain a segmentation prediction map for the camouflaged target.

[0031] In a third aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the method described above.

[0032] In a fourth aspect, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method described above.

[0033] The embodiments of the present invention have the following beneficial effects: The present invention extracts image features layer by layer through an internally cascaded encoder module to capture multi-scale features from coarse to fine. Furthermore, the feature vector output by the encoder module is further processed to enhance the feature representation of the target area. Through secondary convolution and batch normalization operations, not only the features are refined, but also the stability and robustness of the features are improved. Finally, through an internally cascaded decoder module with a multi-head self-attention module, the amplified features of the previous layer and the aggregated features of the foreground feature map, background feature map and segmentation map of each layer processed by the multi-head self-attention module are used to generate a segmentation prediction map for the camouflaged target and assist the foreground area processing and background area processing through the segmentation prediction map, thereby improving the feature prediction results of each level and obtaining a more refined prediction result. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] in: Figure 1 A schematic diagram of the structure of an embodiment of a camouflaged target recognition system based on feature aggregation provided by the present invention; Figure 2 A schematic diagram of the structure of another embodiment of a camouflaged target recognition system based on feature aggregation provided by the present application; Figure 3 A schematic diagram of the structure of an embodiment of a target area magnification module provided by the present invention; Figure 4 A schematic diagram of the structure of an embodiment of a foreground self-attention module provided by the present invention; Figure 5 A schematic diagram of the structure of another embodiment of a camouflage target recognition system based on feature aggregation provided by the present invention; Figure 6 A schematic diagram of a flow chart of an embodiment of a camouflaged target recognition method based on feature aggregation provided by the present invention; Figure 7 A schematic diagram of a flow chart of another embodiment of a camouflaged target recognition method based on feature aggregation provided by the present invention; Figure 8 A schematic structural diagram of an embodiment of the device provided by the present invention; Fig. 9 A schematic structural diagram of an embodiment of the medium provided by the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] like Figure 1 As shown, Figure 1 The present invention provides a structural diagram of a disguised target recognition system based on feature aggregation according to an embodiment of the present invention. A disguised target recognition system based on feature aggregation comprises an internally cascaded encoder module 11, a target region magnification module 12 and a decoder module 13.

[0038] The encoder module 11 includes a preset number of encoders and is used to obtain a feature vector of an initial image.

[0039] For example, referring to Figure 2 , Figure 2 The schematic diagram of another embodiment of a camouflage target recognition system based on feature aggregation provided by the present application. The encoder module 11 includes four cascaded ViT (Vision Transformer) encoders, which are the first level, the second level, the third level and the fourth level from top to bottom, and the fourth level is the highest level. Taking the first level ViT encoder as an example, the ViT encoder is used to extract features, with a size of The initial image is taken as input and the generated image size is The generated image block is linearly projected to obtain an image block of size , the number of channels is The image block with spatial position embedding is further subjected to feature size redefinition operation to obtain a size of The first-level feature vector of .

[0040] By analogy, the feature vector output by the encoder of the first level is input into the encoder of the second level, and the feature size is redefined to obtain the feature vector of the second level. The feature vector of the second level is input into the encoder of the third level, and the feature size is redefined to obtain the feature vector of the third level. The feature vector of the third level is input into the encoder of the fourth level, and the feature size is redefined to obtain the feature vector of the fourth level. At this time, four feature vectors are obtained.

[0041] The target area magnification module 12 includes a preset number of magnification modules, which are used to perform secondary convolution and batch normalization operations on the feature vector to obtain magnification features, and the magnification features include fusion magnification features and top-level magnification features.

[0042] Exemplarily, each amplification module includes a dual-branch amplification module and a single-branch amplification module. The dual-branch amplification module is used to use the feature vectors obtained by the encoders of adjacent levels as dual-branch inputs, respectively obtain the compressed feature vectors of each branch, and upsample the compressed feature vectors, concatenate the upsampled compressed feature vectors of the two branches, and then perform a convolution and batch normalization operation to obtain a fused amplified feature; the single-branch amplification module is used to perform a convolution and batch normalization operation on the feature vector obtained by the encoder of the highest level, and then perform a convolution operation to obtain the highest-level amplified feature.

[0043] Specifically, refer to Figure 2 and Figure 3 , Figure 3 The target region enlargement module 12 is a schematic diagram of an embodiment of the present invention. The target region enlargement module 12 includes four enlargement modules arranged in cascade. In the enlargement stage, the feature vector of the third level and the feature vector of the fourth level are used as two branches inputted by the enlargement module of the third level. The feature vectors of each branch are firstly enlarged once. The convolution operation , apply batch normalization BN and GeLU activation function to the convolutional features to compress the feature vector and obtain the compressed feature vector; perform upsampling operation UpSample of bicubic interpolation on the compressed feature vector to complete the scale enlargement of the feature vector; concatenate (add elements) the compressed feature vectors compressed and scaled by the two branch channels to complete the feature fusion of adjacent levels, and perform another upsampling operation on the fused features. The convolution operation , batch normalization BN and GeLU activation function are applied to the convolutional features to further compress the number of channels, complete the fusion and amplification of the feature vectors of the third level and the feature vectors of the fourth level, and obtain the fused and amplified features of the third level.

[0044] By analogy, the same operation is performed on the feature vectors of the first level and the feature vectors of the second level as the two branches of the input of the amplification module at the first level, and the feature vectors of the second level and the feature vectors of the third level as the two branches of the input of the amplification module at the second level, to complete the fusion and amplification of the feature vectors of all features at adjacent levels, and obtain the fused amplified features of the first level and the fused amplified features of the second level.

[0045] At the same time, the feature vectors at the fourth level are separately The convolution operation is performed, batch normalization and ReLU activation function are applied to the convolutional features, and then The convolution operation is performed to obtain the highest-level amplified features of the fourth level.

[0046] The decoder module 13 includes a preset number of multi-head self-attention modules, which are used to obtain the foreground feature map and the background feature map of the enlarged feature of the current level through matrix multiplication and element multiplication operations, obtain the segmentation map of the enlarged feature of the current level through independent convolution and depth-separable convolution operations, aggregate the foreground feature map, the background feature map and the segmentation map, and combine the enlarged features output by the previous level to obtain the segmentation prediction map of the camouflaged target.

[0047] Exemplarily, the multi-head self-attention module includes a foreground self-attention head, a background self-attention head and a standard self-attention head. The foreground self-attention head and the background self-attention head are used to obtain the prediction mask according to the enlarged features of the current level respectively. The query matrix and the key matrix of the prediction mask and the enlarged features are used to obtain the channel attention map through element multiplication and matrix multiplication, and then matrix multiplication is performed with the value matrix to obtain the foreground feature map and the background feature map of the current level; the standard self-attention head is used to perform independent convolution and depth-separable convolution operations on the enlarged features of the current level, determine the attention of multiple convolution heads, and obtain the segmentation map of the current level according to the attention of multiple convolution heads.

[0048] Specifically, refer to Figure 2 , taking the highest level amplified features of the fourth level as an example, the highest level amplified features of the fourth level are used as the input of the multi-head self-attention module of the fourth level. In the standard self-attention head, the highest level amplified features input to the multi-head self-attention module at the highest level are subjected to independent convolution and depth-wise separable convolution operations, the multi-convolution head attention is determined, and the segmentation map of the current level is obtained according to the multi-convolution head attention.

[0049] At the same time, the highest-level amplified features output by the multi-head self-attention module at the highest level (the fourth level) are supervised by the initial input mask GT, and the prediction map of the fourth level is generated using BCE (Binary Cross-Entropy) and IoU (Intersection over Union) as the prediction loss function. From this, the prediction loss function can be obtained as shown in the following formula: ; Among them, Loss is the prediction loss function, i is the number of levels, GT is the initial input mask, BCE is the binary cross entropy, IoU is the intersection over union ratio, P i is the prediction graph of the i-th level, loss bce () is the binary cross entropy loss function, loss iou() is the intersection-over-union loss function.

[0050] The prediction graph of the fourth level is The size of the convolution operation is The predicted mask P4 of the fourth level is input into the foreground self-attention head.

[0051] It should be noted that since the foreground self-attention head and the background self-attention head have similar structures, the foreground self-attention head is taken as an example for explanation.

[0052] Specifically, Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of an embodiment of the foreground self-attention module provided by the present invention. The fourth-level highest-level amplified feature is input into the foreground self-attention head and then normalized to generate a size of The feature map is further analyzed through three independent convolution and Depthwise Separable Convolution Operation Generate a foreground query matrix, a foreground key matrix and a foreground value matrix; perform element-wise multiplication of the fourth-level prediction mask P4 with the foreground query matrix and the foreground key matrix to obtain a scale size of , the number of channels is The foreground query matrix mapping and the foreground key matrix mapping are , at this time, the foreground query matrix mapping and the foreground key matrix mapping have been transposed once. After the foreground key matrix mapping is transposed, it is restored to the foreground key matrix before the transposition and matrix multiplication is performed with the foreground query matrix mapping to obtain a scale size of The channel attention map is obtained by matrix multiplication of the channel attention map and the foreground value matrix. The foreground feature map after processing the foreground area. Similarly, the background self-attention head also uses the highest-level amplified features of the fourth level as input to perform similar operations, and can obtain the background feature map after processing the background area.

[0053] Furthermore, the features output by the three attention heads are added to obtain the fourth-level global interaction features with global interaction for processing the foreground area and the background area. The fourth-level global interaction features are aggregated with the fused amplified features of the previous level (the third level) and input into the multi-head self-attention module of the previous level. At the same time, the fused amplified features of the third level generate the prediction map of the third level according to the prediction loss function under the supervision of the initial input mask GT. The prediction map of the third level is convolved to obtain the prediction mask P3 of the third level. The prediction mask is used as the input of the foreground self-attention head and the background self-attention head in the multi-head self-attention module of the third level. The rest of the process is the same as the above multi-head self-attention module processing process. By analogy, through the cascaded multi-head self-attention module, the segmentation prediction map of the camouflaged target is generated by using the aggregated features of the amplified features of each layer and the features of the previous layer processed by the multi-head self-attention module.

[0054] It can be seen from the above description that the present invention extracts image features layer by layer through an internally cascaded encoder module to capture multi-scale features from coarse to fine. Furthermore, the feature vector output by the encoder module is further processed to enhance the feature representation of the target area. Through secondary convolution and batch normalization operations, not only the features are refined, but also the stability and robustness of the features are improved. Finally, through an internally cascaded decoder module with a multi-head self-attention module, the amplified features of the previous level and the aggregated features of the foreground feature map, background feature map and segmentation map of each layer after processing by the multi-head self-attention module are used to generate a segmentation prediction map for the camouflaged target and assist the foreground area processing and background area processing through the segmentation prediction map, so as to improve the feature prediction results of each level and obtain a more refined prediction result.

[0055] like Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of another embodiment of a camouflage target recognition system based on feature aggregation provided by the present invention. The multi-head self-attention modules each include a foreground self-attention head, a background self-attention head and a standard self-attention head.

[0056] The foreground self-attention head and the background self-attention head are used to obtain the prediction mask according to the enlarged features of the current level respectively. The query matrix and the key matrix of the prediction mask and the enlarged features obtain the channel attention map through element multiplication and matrix multiplication, and then perform matrix multiplication with the value matrix to obtain the foreground feature map and background feature map of the current level.

[0057] For example, referring to Figure 2 and Figure 4, taking the fourth-level highest-level enlarged feature as an example, the fourth-level highest-level enlarged feature is used as the input of the fourth-level multi-head self-attention module. The fourth-level highest-level enlarged feature is input into the foreground self-attention head and then normalized to generate a size of The feature map is further analyzed through three independent convolution and Depthwise Separable Convolution Generate a foreground query matrix, a foreground key matrix, and a foreground value matrix; the highest level amplified features generate a fourth-level prediction map according to the prediction loss function under the supervision of the initial input mask GT; perform a convolution operation on the fourth-level prediction map to obtain a fourth-level prediction mask P4; perform element-wise multiplication of the fourth-level prediction mask P4 with the foreground query matrix and the foreground key matrix to obtain a scale of , the number of channels is The foreground query matrix mapping and the foreground key matrix mapping are , at this time, the foreground query matrix mapping and the foreground key matrix mapping have been transposed once. After the foreground key matrix mapping is transposed, it is restored to the foreground key matrix before the transposition and matrix multiplication is performed with the foreground query matrix mapping to obtain a scale size of The channel attention map is obtained by matrix multiplication of the channel attention map and the foreground value matrix. The foreground feature map after processing the foreground area. Similarly, the background self-attention head also uses the highest-level amplified features of the fourth level as input to perform similar operations, and can obtain the background feature map after processing the background area.

[0058] Specifically, the foreground feature map is determined according to the following formula: ; Among them, MA F is the foreground feature map, V F is the foreground value matrix, Q F is the foreground query matrix, T is the transposed amount, K F is the foreground key matrix, α is a learnable parameter, and softmax() is the normalized exponential function.

[0059] The background feature map is determined according to the following formula: ; Among them, MA B is the background feature map, V B is the background value matrix, Q B is the background query matrix, T is the transposed amount, K B is the background key matrix, α is a learnable parameter, and softmax() is the normalized exponential function.

[0060] The standard self-attention head is used to perform independent convolution and depth-wise separable convolution operations on the enlarged features of the current level, determine the attention of multiple convolution heads, and obtain the segmentation map of the current level based on the attention of multiple convolution heads.

[0061] Exemplarily, taking the highest-level amplified features of the fourth level as an example, the highest-level amplified features of the fourth level are used as the input of the multi-head self-attention module of the fourth level.

[0062] It should be noted that the standard self-attention head is expanded to obtain the foreground self-attention head and the background self-attention head, and the structures of the foreground self-attention head and the background self-attention head are similar. For the standard self-attention head, it can be defined as: ; Among them, MA is the multi-convolutional head attention, Q is the query matrix, T is the transposed amount, K is the key matrix, and V is the value matrix. Convolution and The depth-wise separable convolution operation generates the query matrix, key matrix and value matrix, and α is a learnable parameter.

[0063] Specifically, in the standard self-attention head, independent convolution and depth-separable convolution operations are performed on the highest-level amplified features input to the decoder at the fourth level, the attention of multiple convolution heads is determined, and the segmentation map of the fourth level is obtained according to the attention of multiple convolution heads.

[0064] like Figure 6 As shown, Figure 6 A flow chart of an embodiment of a method for identifying a camouflaged target based on feature aggregation provided by the present invention. A method for identifying a camouflaged target based on feature aggregation, the method comprising: S101: Obtain a feature vector of an initial image.

[0065] Exemplarily, the initial image is divided into several image blocks, and each image block is linearly projected to obtain an image block with spatial position embedding; a feature size redefinition operation is performed on the image block with spatial position embedding to obtain a first-level feature vector.

[0066] A feature size redefinition operation is performed on the feature vectors of the first level until a preset number of feature vectors are obtained.

[0067] S102: Perform secondary convolution and batch normalization operations on the feature vector to obtain amplified features, where the amplified features include fused amplified features and top-level amplified features.

[0068] Exemplarily, the feature vectors of adjacent levels are used as dual-branch inputs, and the compressed feature vectors of each branch are obtained respectively, and the compressed feature vectors are upsampled, and the upsampled compressed feature vectors of the two branches are concatenated and then convolved and batch normalized to obtain fused amplified features. After performing a convolution and batch normalization operation on the feature vector of the highest level, a convolution operation is performed again to obtain the highest-level amplified features.

[0069] S103: Obtain the foreground feature map and background feature map of the enlarged feature of the current level through matrix multiplication and element multiplication operations, obtain the segmentation map of the enlarged feature of the current level through independent convolution and depth-separable convolution operations, aggregate the foreground feature map, background feature map and segmentation map, and combine the enlarged features output by the previous level to obtain the segmentation prediction map of the camouflaged target.

[0070] Exemplarily, independent convolution and depth-separable convolution operations are performed on the enlarged features of the current level, the attention of multiple convolution heads is determined, and the segmentation map of the current level is obtained according to the attention of multiple convolution heads; the enlarged features of the current level generate the prediction map of the current level according to the prediction loss function under the supervision of the initial input mask GT; the prediction map of the current level is convolved to obtain the prediction mask; the enlarged features of the current level are standardized and then the query matrix, key matrix and value matrix are obtained again through independent convolution and depth-separable convolution operations; the prediction mask of the current level is element-wise multiplied with the query matrix and key matrix respectively to obtain the query matrix Mapping and key matrix mapping, at this time, the query matrix mapping and the key matrix mapping have been transposed once; after the key matrix mapping is transposed, it is restored to the key matrix before transposition and matrix multiplication is performed with the query matrix mapping to obtain the channel attention map of the current level; after the channel attention map of the current level is matrix multiplied with the value matrix, the foreground feature map and background feature map of the current level are obtained; the segmentation map of the current level is added to the foreground feature map and the background feature map to obtain the global interaction feature of the current level; the global interaction feature of the current level is aggregated with the fused amplification feature of the previous level to obtain the segmentation prediction map of the camouflaged target.

[0071] like Figure 7 As shown, Figure 7 A flow chart of another embodiment of a camouflaged target recognition method based on feature aggregation provided by the present invention. A camouflaged target recognition method based on feature aggregation, the method comprising: S201: Divide the initial image into a number of image blocks, perform linear projection on each image block, and obtain an image block with spatial position embedding.

[0072] For example, referring to Figure 2The encoder module includes four cascaded ViT (Vision Transformer) encoders. Taking the first-level ViT encoder as an example, feature extraction is performed through the ViT encoder. Specifically, with a size of The initial image is taken as input and the generated image size is Further, the generated image block is linearly projected to obtain an image block of size , the number of channels is Image patches with spatial position embeddings.

[0073] S202: performing a feature size redefinition operation on the image block with spatial position embedding to obtain a first-level feature vector.

[0074] For example, the image block with spatial position embedding is subjected to feature size redefinition operation to obtain a size of The first-level feature vector of .

[0075] S203: performing a feature size redefinition operation on the feature vectors of the first level until a preset number of feature vectors are obtained.

[0076] Exemplarily, the feature vector of the first level is used as the input of the encoder at the second level, and the feature size is redefined to obtain the feature vector of the second level. Similarly, the feature vector output by the encoder of the previous level is input into the encoder of the next level to perform the feature size redefined operation until four feature vectors are obtained.

[0077] S204: Using the feature vectors of adjacent levels as dual-branch inputs, respectively obtain the compressed feature vectors of each branch, and perform upsampling operations on the compressed feature vectors. After concatenating the upsampled compressed feature vectors of the two branches, perform a convolution and batch normalization operation to obtain fused and amplified features.

[0078] For example, referring to Figure 2 and Figure 3 The target area amplification module includes four amplification modules in cascade. In the amplification stage, the feature vector of the third level and the feature vector of the fourth level are used as two branches of the input of the amplification module at the third level. The convolution operation , apply batch normalization BN and GeLU activation function to the convolutional features to compress the feature vector and obtain the compressed feature vector; perform bicubic interpolation upsampling on the compressed feature vector to complete the scale enlargement of the feature vector; after splicing the compressed feature vectors compressed and scaled by the two branch channels, perform another The convolution operation , batch normalization BN and GeLU activation function are applied to the convolutional features to further compress the number of channels, complete the fusion and amplification of the feature vectors of the third level and the feature vectors of the fourth level, and obtain the fused and amplified features of the third level.

[0079] By analogy, the same operation is performed on the feature vectors of the first level and the feature vectors of the second level as the two branches of the input of the amplification module at the first level, and the feature vectors of the second level and the feature vectors of the third level as the two branches of the input of the amplification module at the second level, to complete the fusion and amplification of all features and obtain the fused amplified features of the first level and the fused amplified features of the second level.

[0080] S205: After performing a convolution and batch normalization operation on the feature vector of the highest level, a convolution operation is performed again to obtain the highest level amplified features.

[0081] For example, the feature vectors of the fourth level are individually processed once The convolution operation is performed, batch normalization and ReLU activation function are applied to the convolutional features, and then The convolution operation is performed to obtain the highest-level amplified features of the fourth level.

[0082] S206: Perform independent convolution and depth-separable convolution operations on the enlarged features of the current level, determine the attention of multiple convolution heads, and obtain a segmentation map of the current level according to the attention of multiple convolution heads.

[0083] Exemplarily, each multi-head self-attention module includes a foreground self-attention head, a background self-attention head, and a standard self-attention head. Taking the highest level magnified feature of the fourth level as an example, the highest level magnified feature of the fourth level is used as the input of the multi-head self-attention module of the fourth level. In the standard self-attention head, the highest level magnified feature is subjected to independent convolution and depth-separable convolution operations, the multi-convolution head attention is determined, and the segmentation map of the current level is obtained according to the multi-convolution head attention.

[0084] S207: The enlarged features of the current level generate a prediction map of the current level according to the prediction loss function under the supervision of the initial input mask.

[0085] Exemplarily, the highest-level amplified features output by the multi-head self-attention module at the fourth level are supervised by the initial input mask GT, and the prediction graph of the fourth level is generated using BCE (Binary Cross-Entropy) and IoU (Intersection over Union) as the prediction loss function, thereby obtaining the prediction loss function as shown in the following formula: ; Among them, Loss is the prediction loss function, i is the number of levels, GT is the initial input mask, BCE is the binary cross entropy, IoU is the intersection over union ratio, P i is the prediction graph of the i-th level.

[0086] S208: Perform a convolution operation on the prediction map of the current level to obtain a prediction mask.

[0087] Exemplarily, the prediction graph of the fourth level is The size of the convolution operation is The fourth-level prediction mask P4 is input into the foreground self-attention head and the background self-attention head.

[0088] S209: After normalizing the amplified features of the current level, the query matrix, key matrix and value matrix are obtained again through independent convolution and depth-separable convolution operations.

[0089] For example, the fourth level’s highest level amplifies the feature input foreground and normalizes it after the attention head, generating a size of The feature map is further analyzed through three independent Convolution and Depthwise separable convolution generates foreground query matrix, foreground key matrix, and foreground value matrix.

[0090] The fourth layer’s highest layer amplifies the feature input background and then normalizes it after the self-attention head, generating a size of The feature map is further analyzed through three independent Convolution and Depthwise separable convolution generates background query matrix, background key matrix, and background value matrix.

[0091] S210: Perform element-wise multiplication of the prediction mask of the current level with the query matrix and the key matrix respectively to obtain a query matrix mapping and a key matrix mapping.

[0092] For example, the fourth-level prediction mask P4 is element-wise multiplied with the foreground query matrix and the foreground key matrix to obtain a scale of , the number of channels is The foreground query matrix mapping and foreground key matrix mapping are now matrix-transposed.

[0093] The fourth-level prediction mask P4 is element-wise multiplied with the background query matrix and the background key matrix to obtain a scale of , the number of channels is The background query matrix mapping and background key matrix mapping are obtained by performing a matrix transposition on the foreground query matrix mapping and the foreground key matrix mapping.

[0094] S211: After performing matrix transposition on the key matrix mapping, perform matrix multiplication with the query matrix mapping to obtain the channel attention map of the current level.

[0095] For example, after performing matrix transposition on the foreground key matrix mapping, the foreground key matrix is ​​restored to the foreground key matrix before transposition and matrix multiplication is performed with the foreground query matrix mapping to obtain a scale size of After the background key matrix mapping is transposed, the background key matrix is ​​restored to the original untransposed matrix and matrix multiplied with the background query matrix mapping to obtain a scale of Background channel attention map.

[0096] S212: After performing matrix multiplication of the channel attention map of the current level with the value matrix, the foreground feature map and the background feature map of the current level are obtained.

[0097] For example, the foreground channel attention map is multiplied by the foreground value matrix to obtain a size of The foreground feature map after processing the foreground area. The background channel attention map is multiplied by the background value matrix to obtain a size of Background feature map after processing the background area.

[0098] S213: Add the segmentation map of the current level to the foreground feature map and the background feature map to obtain the global interaction features of the current level.

[0099] Exemplarily, the segmentation map of the current level is added to the foreground feature map and the background feature map to obtain the fourth level global interaction features with global interaction for processing the foreground area and the background area.

[0100] S214: Aggregate the global interaction features of the current level with the fused amplified features of the previous level to obtain a segmentation prediction map of the camouflaged target.

[0101] Exemplarily, the global interaction features of the fourth level are aggregated with the fused amplified features of the third level and then input into the multi-head self-attention module of the third level. At the same time, the fused amplified features of the third level generate a prediction map of the third level according to the prediction loss function under the supervision of the initial input mask GT. The prediction map of the third level is convolved to obtain a prediction mask P3 of the third level. The prediction mask is used as the input of the foreground self-attention head and the background self-attention head in the multi-head self-attention module of the third level. The rest of the process is the same as the above-mentioned multi-head self-attention module processing process.

[0102] like Figure 8 As shown, Figure 8The device 20 includes a memory 21 and a processor 22. The memory 21 stores a computer program, and the processor 22 executes the computer program when working to implement the following. Figure 6 and Figure 7 The method shown.

[0103] The specific technical details of a camouflaged target recognition method based on feature aggregation implemented when the above-mentioned device 20 executes a computer program have been discussed in detail in the aforementioned method steps, so they will not be repeated here.

[0104] like Fig. 9 As shown, Fig. 9 The structure diagram of an embodiment of the medium provided by the present invention is shown in FIG. The medium 30 stores at least one computer program 31, and the computer program 31 is executed by the processor 22 to implement the following Figure 6 and Figure 7 In one embodiment, the medium 30 may be a storage chip, a hard disk, a mobile hard disk, a USB flash drive, an optical disk, or other readable and writable storage tools, or a server, etc.

[0105] Additionally, the processes depicted in the figures do not necessarily have to be performed in the particular order shown or in sequential order to achieve desired results. In certain implementations, multitasking and parallel processing are also possible or may be advantageous.

[0106] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0107] The apparatus, device, non-volatile computer-readable storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, device, and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the corresponding apparatus, device, and non-volatile computer storage medium will not be repeated here.

[0108] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0109] For the convenience of description, the above device is described by being divided into various units according to their functions and described separately. Of course, when implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware. It should be understood by those skilled in the art that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0110] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0111] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0113] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0114] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0115] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0116] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0117] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0118] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0119] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A camouflage target recognition system based on feature aggregation, characterized in that: The system includes an internally cascaded encoder module, a target area magnification module, and a decoder module; The encoder module includes a preset number of encoders for obtaining a feature vector of an initial image; The target area magnification module includes a preset number of magnification modules, which are used to perform secondary convolution and batch normalization operations on the feature vector to obtain magnification features, wherein the magnification features include fusion magnification features and top-level magnification features; The decoder module includes a preset number of multi-head self-attention modules, which are used to obtain the foreground feature map and the background feature map of the enlarged feature of the current level through matrix multiplication and element multiplication operations, obtain the segmentation map of the enlarged feature of the current level through independent convolution and depth-separable convolution operations, aggregate the foreground feature map, the background feature map and the segmentation map, and combine the enlarged feature output of the previous level to obtain the segmentation prediction map of the camouflaged target.

2. The camouflage target recognition system based on feature aggregation according to claim 1 is characterized in that: The amplification modules include a dual-branch amplification module and a single-branch amplification module; The dual-branch amplification module is used to use the feature vectors obtained by the encoders of adjacent levels as dual-branch inputs, respectively obtain compressed feature vectors of each branch, and perform upsampling operations on the compressed feature vectors, concatenate the upsampled compressed feature vectors of the two branches, and then perform convolution and batch normalization operations to obtain fused amplified features; The single-branch amplification module is used to perform a convolution and batch normalization operation on the feature vector obtained by the encoder at the highest level, and then perform a convolution operation again to obtain the highest-level amplified features.

3. The camouflage target recognition system based on feature aggregation according to claim 1 is characterized in that: The multi-head self-attention modules each include a foreground self-attention head, a background self-attention head and a standard self-attention head; The foreground self-attention head and the background self-attention head are used to obtain prediction masks according to the enlarged features of the current level, respectively. The query matrix and the key matrix of the prediction mask and the enlarged features obtain a channel attention map through element multiplication and matrix multiplication, and then perform matrix multiplication with the value matrix to obtain the foreground feature map and the background feature map of the current level; The standard self-attention head is used to perform independent convolution and depth-separable convolution operations on the enlarged features of the current level, determine the attention of multiple convolution heads, and obtain the segmentation map of the current level according to the attention of multiple convolution heads.

4. The camouflage target recognition system based on feature aggregation according to claim 3 is characterized in that: The foreground feature map is based on Determine, among which, MA F is the foreground feature map, V F is the foreground value matrix, Q F is the foreground query matrix, T is the transposed amount, K F is the foreground key matrix, α is a learnable parameter, and softmax() is the normalized exponential function; The background feature map is based on Determine, among which, MA B is the background feature map, V B is the background value matrix, QB is the background query matrix, T is the transposed amount, K B is the background key matrix, α is a learnable parameter, and softmax() is the normalized exponential function.

5. A camouflage target recognition method based on feature aggregation, characterized in that: The method comprises: Get the feature vector of the initial image; Performing secondary convolution and batch normalization operations on the feature vector to obtain amplified features, wherein the amplified features include fused amplified features and top-level amplified features; Through matrix multiplication and element multiplication operations, the foreground feature map and background feature map of the enlarged feature of the current level are obtained, and through independent convolution and depth-separable convolution operations, the segmentation map of the enlarged feature of the current level is obtained. The foreground feature map, the background feature map and the segmentation map are aggregated, and combined with the enlarged feature output of the previous level, a segmentation prediction map of the camouflaged target is obtained.

6. The method for identifying camouflaged targets based on feature aggregation according to claim 5 is characterized in that: The step of obtaining the feature vector of the initial image specifically includes: Dividing the initial image into a plurality of image blocks, performing linear projection on each of the image blocks, and obtaining an image block with spatial position embedding; Performing a feature size redefinition operation on the image block with spatial position embedding to obtain a first-level feature vector; A feature size redefinition operation is performed on the feature vectors of the first level until a preset number of feature vectors are obtained.

7. The method for identifying camouflaged targets based on feature aggregation according to claim 6, characterized in that: The feature vector is subjected to secondary convolution and batch normalization operations to obtain amplified features, wherein the amplified features include fused amplified features and top-level amplified features, specifically including: Taking the feature vectors of adjacent levels as dual-branch inputs, respectively obtaining compressed feature vectors of each branch, and performing upsampling operations on the compressed feature vectors, concatenating the upsampled compressed feature vectors of the two branches, and then performing convolution and batch normalization operations to obtain fused amplified features; After performing a convolution and batch normalization operation on the feature vector of the highest level, a convolution operation is performed again to obtain the highest level amplified features.

8. The method for identifying camouflaged targets based on feature aggregation according to claim 7 is characterized in that: The foreground feature map and the background feature map of the enlarged feature of the current level are obtained by matrix multiplication and element multiplication operations, the segmentation map of the enlarged feature of the current level is obtained by independent convolution and depth-separable convolution operations, the foreground feature map, the background feature map and the segmentation map are aggregated, and the enlarged feature outputted by the previous level is combined to obtain the segmentation prediction map of the camouflaged target, which specifically includes: Performing independent convolution and depth-separable convolution operations on the enlarged features of the current level, determining multi-convolution head attention, and obtaining a segmentation map of the current level according to the multi-convolution head attention; The enlarged features of the current level generate a prediction map of the current level according to a prediction loss function under the supervision of the initial input mask; Performing a convolution operation on the prediction map of the current level to obtain a prediction mask; After normalizing the amplified features of the current level, a query matrix, a key matrix, and a value matrix are obtained again through independent convolution and depth-separable convolution operations; Performing element-wise multiplication of the prediction mask of the current level with the query matrix and the key matrix respectively to obtain a query matrix mapping and a key matrix mapping; The key matrix mapping is matrix transposed and then matrix multiplied with the query matrix mapping to obtain a channel attention map of the current level; After performing matrix multiplication of the channel attention map of the current level and the value matrix, a foreground feature map and a background feature map of the current level are obtained; Adding the segmentation map of the current level to the foreground feature map and the background feature map to obtain the global interaction feature of the current level; The global interaction features of the current level are aggregated with the fused amplified features of the previous level to obtain a segmentation prediction map for the camouflaged target.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 5 to 8.

10. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 5 to 8.

Images