A camouflaged target detection method, system, device and medium

Abstract

The present invention provides a camouflaged target detection method, system, device, and medium, belonging to the field of camouflaged target detection. The method includes the following steps: obtaining an RGB image and a depth image of a camouflaged target to be detected; constructing a camouflaged target detection model, the camouflaged target detection model comprising a backbone network PVTv2 network, a depth feature enhancement module, a multimodal cross-fusion module, and a fusion and refinement module; inputting the RGB image and the depth image into two weight-shared backbone networks PVTv2 to extract multi-layer RGB features and multi-layer depth features, and performing feature enhancement processing on the depth features of each layer using the depth feature enhancement module; performing multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature map using the multimodal cross-fusion module, and then enhancing and refining the fused feature map using the fusion and refinement module to obtain a camouflaged target image. The present invention can improve the detection accuracy of camouflaged targets.

Description

Technical Field

[0001] The present invention belongs to the field of disguised target detection, and in particular relates to a disguised target detection method, system, equipment and medium. Background Art

[0002] Camouflage is a common phenomenon in nature and a common tactic used in military operations, significantly impacting ecological balance, national defense, and security. The goal of camouflaged target detection is to identify camouflaged targets hidden within an environment, which may blend into the environment through natural or artificial means. Unlike conventional salient targets, camouflaged targets represent a class of non-salient targets with insufficient semantic information and features. These targets blend closely with the surrounding background, contain less pixel information in the image, have low resolution, and exhibit poor visual edge recognition, making detection highly complex. However, camouflaged target detection technology is highly practical and widely used in medical image analysis, crop and locust monitoring, video polyp segmentation, rescue assistance, and military security. Therefore, improving the performance and accuracy of camouflaged target detection is a key technical challenge in this field.

[0003] Currently, camouflaged object detection in RGB images has received widespread attention and research. The paper "Camouflaged object detection. In CVPR, 2020" proposes SINet, which introduces a receptive field module to expand the local receptive field, thereby facilitating the segmentation of camouflaged objects. However, while the receptive field module effectively enhances the extraction of local features, it still has a significant limitation: it cannot fully capture global features. The paper "Attention-induced semantic and boundary interaction network for camouflaged object detection. Computer Vision and Image Understanding, 2023" proposes an attention-induced semantic and boundary interaction network, which retains more semantic information and generates more powerful classification features. However, it does not consider targets hidden by specific color or texture patterns and cannot capture the deep information of these features. Chinese invention patent number CN116228702A discloses a camouflaged object detection method based on an attention mechanism and a convolutional neural network. While it considers aggregating multi-layer features and refining boundary contours, it does not consider the complexity of the camouflaged background and ignores spatial information.

[0004] In summary, the existing camouflaged target detection algorithms are unable to capture the complex and deep information of the camouflaged background, resulting in low camouflaged target detection accuracy. Summary of the Invention

[0005] In order to overcome the above-mentioned shortcomings of low accuracy in conventional disguised target detection, the present invention provides a disguised target detection method, comprising the following steps:

[0006] Obtain the RGB image and depth image of the camouflaged target to be detected;

[0007] Construct a disguised target detection model, which includes a backbone network PVTv2 network, multiple deep feature enhancement modules, a multimodal cross-fusion module, and a fusion refinement module;

[0008] The RGB image and depth image are simultaneously input into the camouflaged target detection model. Multi-layer RGB feature extraction and multi-layer depth feature extraction are performed through two weight-sharing backbone networks PVTv2. The depth feature enhancement module is used to enhance the depth features of each layer. The multi-modal cross fusion module is used to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature maps. The fusion refinement module is then used to enhance and refine the fused feature maps to obtain the detected camouflaged target image.

[0009] Preferably, the depth feature enhancement module includes a channel attention mechanism and a spatial attention mechanism, and uses the channel attention mechanism to perform feature enhancement processing on the depth feature, specifically: performing maximum pooling and average pooling based on width and height on the input depth feature to obtain two 1×1×C feature maps, and then adjusting the number of channels through the shared multi-layer perceptron ShareMLP, and adding the output feature maps, and then performing a Sigmoid activation function to obtain channel information, and finally multiplying the channel information with the input depth feature to obtain the output feature map of the channel attention mechanism;

[0010] The spatial attention mechanism is used to enhance the deep features. Specifically, the output feature map of the channel attention mechanism is subjected to channel-based maximum pooling and average pooling to obtain two H×W×1 feature maps, which are then concatenated from the channel direction. The feature map is reduced in dimension by 7×7 convolution, and the weight parameters are generated by a Sigmoid function to obtain spatial information.

[0011] Preferably, after obtaining the spatial information, the method further includes multiplying the spatial information with the output feature map of the channel attention mechanism to obtain the output feature map of the hybrid attention mechanism.

[0012] Preferably, the multi-modal cross fusion module is used to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature map, specifically: the fusion unit RU of the multi-modal cross fusion module is used to perform convolution operations of different sizes, expand the receptive field of each layer of feature map, and then perform weighted fusion through parameters, and finally use the upsampling operation to splice the sampled feature map with the feature map of the previous layer to obtain the fused feature map F s .

[0013] Preferably, the step of enhancing and refining the fused feature map using the fusion and refinement module comprises the following steps:

[0014] Use three 1×1 convolutions to extract the query information Q, key information K, and value information V of the input feature map respectively;

[0015] The query information Q, key information K and value information V of the input feature map are enhanced respectively to obtain the three-dimensional query information Q∈R C×H×W , three-dimensional bond information K∈R C×H×W and three-dimensional value information V∈R C×H×W ;

[0016] For three-dimensional query information Q∈R C×H×W , three-dimensional bond information K∈R C×H×W and three-dimensional value information V∈R C×H×W Reconstruct and obtain the corresponding two-dimensional matrix q∈R C×N , k∈R C×N and v∈R C×N ;

[0017] For a two-dimensional matrix q∈R C×N and k∈R C×N Perform matrix multiplication to obtain a mapping R representing the long-range dependency relationship, and for the two-dimensional matrix v∈R C×N Perform matrix multiplication with the mapping R to obtain the enhanced matrix E;

[0018] Reconstruct the enhanced matrix E to its original shape and size to obtain the enhanced features F e , and F e With F s Add element by element to get the final output feature F f .

[0019] Preferably, before inputting the RGB image and the depth image into two weight-sharing backbone networks PVT-V2 for multi-layer RGB feature extraction and multi-layer depth feature extraction, the method further includes training the camouflaged target detection model. In the training phase, the batch size of the input model is set to 16, the number of epochs is set to 200, and the initial learning rate is set to 10. -4 .

[0020] The present invention also provides a camouflaged target detection system, comprising:

[0021] An image acquisition module is used to acquire an RGB image and a depth image of the camouflaged target to be detected;

[0022] The model building module is used to build a disguised target detection model. The disguised target detection model includes a backbone network PVTv2 network, multiple deep feature enhancement modules, a multimodal cross fusion module, and a fusion refinement module.

[0023] The camouflaged target detection module is used to input the RGB image and the depth image into the camouflaged target detection model at the same time, perform multi-layer RGB feature extraction and multi-layer depth feature extraction by inputting two weight-sharing backbone networks PVTv2, and use the depth feature enhancement module to enhance the depth features of each layer; use the multimodal cross fusion module to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature map, and then use the fusion refinement module to enhance and refine the fused feature map to obtain the detected camouflaged target image.

[0024] The present invention also provides a computer device, comprising a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the camouflaged target detection method.

[0025] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the camouflaged target detection method.

[0026] The camouflaged target detection method, system, device, and medium provided by the present invention have the following beneficial effects:

[0027] The present invention simultaneously extracts features from RGB images and depth images in two weight-sharing backbone networks PVTv2, and can obtain multi-scale RGB features and multi-scale depth features containing camouflaged target images. The RGB features and depth features can share and complement information at multiple levels. By using a depth feature enhancement module to perform feature enhancement processing on the depth features of each layer, the compatibility of the depth features can be enhanced, while the multi-level RGB features can be retained, which is conducive to feature fusion and improves the accuracy of camouflaged target detection. By using a multimodal cross-fusion module to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature maps, the fused features can not only retain the appearance information of the target, but also better perceive the spatial position and three-dimensional structure of the target through the depth features, so that the model has stronger recognition ability in complex scenes. It can not only solve the problem that a single modality is easily affected by changes in illumination, color or texture, but also enhance the perception of the depth difference between the target and the background, avoid the fusion of the camouflaged target and the background, thereby achieving higher accuracy and reliability in camouflaged target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] To more clearly illustrate the embodiments of the present invention and its design, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0029] Figure 1 Flowchart of a camouflaged target detection method according to an embodiment of the present invention;

[0030] Figure 2 This is the flow chart of the deep feature enhancement module;

[0031] Figure 3 This is the flow chart of the multimodal cross-fusion module;

[0032] Figure 4 Flowchart of module refinement for integration;

[0033] Figure 5 The results of the detection of different camouflaged target images using six mainstream camouflaged target segmentation models are compared with the method of the present invention. Figure 5 (a) represents an RGB image, Figure 5 (b) represents the Depth depth image, Figure 5 (c) represents the GT disguised target saliency label, Figure 5 (d) shows the camouflaged target prediction result of Ours invention. Figure 5 (e)- Figure 5(j) represents the image segmentation results of six mainstream camouflaged target segmentation models SINetV2, ZoomNet, DGNet, DTINet, SegMaR, and ICON on the camouflaged target datasets CAMO, CHAMELEON, COD10K, and NC4K. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the technical solution of the present invention and to be able to implement it, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and are not intended to limit the scope of protection of the present invention.

[0035] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the technical solutions of the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0036] In addition, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meaning of the above terms in the present invention can be understood according to the specific circumstances. In the description of the present invention, unless otherwise specified, "plurality" means two or more, which will not be described in detail here.

[0037] Example

[0038] The present invention provides a method for detecting a disguised target, specifically Figure 1 As shown, the following steps are included:

[0039] Step 1: Obtain the RGB image and depth image of the camouflaged target to be detected.

[0040] Step 2: Construct a disguised target detection model. The disguised target detection model includes the backbone network PVTv2 network, deep feature enhancement module, multimodal cross fusion module and fusion refinement module. The backbone network is the PVTv2 network in the Transformer structure.

[0041] Step 3: The RGB image and the depth image are simultaneously input into the camouflaged target detection model. Multi-layer RGB feature extraction and multi-layer depth feature extraction are performed through the two weight-sharing backbone networks PVTv2. The depth feature enhancement module is used to enhance the depth features of each layer. The multi-modal cross-fusion module is used to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature maps. The fusion refinement module is then used to enhance and refine the fused feature maps to obtain the detected camouflaged target image.

[0042] The overall structure of the camouflaged target detection model is as follows: Figure 1 As shown in the figure, the backbone network responsible for feature extraction is the PVTv2 network in the Transformer structure. Two weight-sharing backbone networks, PVTv2, extract four layers of features from the RGB image and the depth image respectively. The features are then enhanced by the depth feature enhancement module. To prevent the semantic information in the original RGB image features from being blurred by useless information in the depth features, thereby affecting the model accuracy, an adaptive fusion strategy is adopted. During the feature fusion process, different weights are assigned to the RGB features and the depth features to identify the dominant mode. The characteristic formula can be expressed as:

[0043] F i =α i F ri +β i F DFEM (F di );

[0044] Among them, F i represents the fusion feature of the i-th (i=1,2,3,4) layer, F ri represents the RGB features of the i-th (i=1,2,3,4) layer, F di represents the depth feature of the i-th (i=1,2,3,4) layer, F DFEM (·) represents the features enhanced by the deep feature enhancement module. α i and β i is the adaptive weight coefficient learned during the network optimization process.

[0045] Then, F iThe features are input into the multimodal cross fusion module (MCFM) for multi-channel and multi-level feature fusion. In the multimodal cross fusion module, the receptive field of each layer of features is gradually expanded and spliced ​​to the previous layer of features through upsampling. The output fusion features are then captured by the fusion refinement module (FRM) guided by semantic position information to obtain more spatial fine-grained information. Finally, 3×3 convolution is used to combine the output features to obtain the refined prediction map S.

[0046] Among them, DFEM is the deep feature enhancement module, and its structure is as follows Figure 2 As shown in the figure, the module is composed of a channel attention mechanism and a spatial attention mechanism, in which each layer of deep features passes through a deep feature enhancement module to enhance the compatibility of deep features while retaining multi-level information. The DFEM module includes a channel attention mechanism and a spatial attention mechanism, and its formula is as follows:

[0047] F DFEM (F di )=S att (C att (F di ));

[0048] Among them, C att (·) represents channel attention, S att (·) represents spatial attention, and their specific expression formulas are as follows:

[0049]

[0050] Among them, MaxPool(·) represents the global maximum pooling feature of each feature map, AvgPool(·) represents the global average pooling feature of each feature map, MLP(·) represents multi-layer perceptron, σ represents the Sigmoid activation function, and F represents the input feature map.

[0051] The detailed process of obtaining the camouflaged target image of the present invention is as follows:

[0052] (1) Train the camouflaged target detection model as follows:

[0053] Obtain a disguised target image dataset, and divide the disguised target image dataset into a training set and a test set;

[0054] The experimental platform is carried out under the environment configuration of Cuda10.2, PyTorch1.12, Python3.8, and GPU is NVIDIAGeForce RTXA5000. All input images in the training set are resized to 352×352 and input into the disguised target detection model to train the disguised target detection model. During the training phase, the sample batch size of the input model is set to 16, the number of epochs is set to 200, and the initial learning rate is set to 10 -4 .

[0055] (2) Input the RGB image and depth image of the camouflaged target to be detected into the trained camouflaged target detection model, and use the two weight-sharing backbone networks PVTv2 of the trained camouflaged target detection model to extract the multi-scale RGB features F containing the camouflaged target image. ri ,i∈{1,2,3,4} and multi-scale deep features F di ,i∈{1,2,3,4}.

[0056] (3) The deep features F extracted from the backbone network di ,i∈{1,2,3,4} are respectively input into the deep feature enhancement module DFEM for feature enhancement;

[0057] The implementation flow chart of the deep feature enhancement module DFEM described in the embodiment of the present invention is as follows: Figure 2 As shown in the figure, the module consists of a channel attention mechanism and a spatial attention mechanism. Each layer of deep features passes through a deep feature enhancement module to enhance the compatibility of deep features while retaining multi-level information. The DFEM module contains both a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism assigns different weights to different channels. First, the input feature map is subjected to maximum pooling and average pooling based on width and height to obtain two 1×1×C feature maps. The number of channels is then adjusted through a shared multi-layer perceptron (Share MLP). The outputs are then summed and the channel information is obtained after a Sigmoid activation function. Finally, the channel information is multiplied with the input feature map to obtain the output feature map of the channel attention mechanism. The spatial attention mechanism assigns different weights to different spaces. First, the feature map is subjected to channel-based maximum pooling and average pooling to obtain two H×W×1 feature maps. These are concatenated along the channel direction. The feature map is then reduced in dimension through a 7×7 convolution. A Sigmoid function is then used to generate weight parameters to obtain spatial information. Finally, the spatial information is multiplied by the output feature map of the channel attention mechanism to obtain the output feature map of the hybrid attention mechanism. The output after feature enhancement can be expressed as:

[0058]

[0059] F DFEM (F di )=S att (C att (F di ));

[0060] Among them, MaxPool(·) represents the global maximum pooling feature of each feature map, AvgPool(·) represents the global average pooling feature of each feature map, MLP(·) represents multi-layer perceptron, σ represents the Sigmoid activation function, and F represents the input feature map.

[0061] (4) RGB feature F ri ,i∈{1,2,3,4} and the depth feature F after depth enhancement DFEM (F di ), i∈{1,2,3,4} are assigned different weights α i ,β i , i∈{1,2,3,4} is added element by element, and its characteristic formula can be expressed as:

[0062] F i =α i F ri +β i F DFEM (F di );

[0063] Among them, F i represents the fusion feature of the i-th (i=1,2,3,4) layer, F ri represents the RGB features of the i-th (i=1,2,3,4) layer, F di represents the depth feature of the i-th (i=1,2,3,4) layer, F DFEM (·) represents the features enhanced by the deep feature enhancement module. The adaptive parameter α i ,β i ,i∈{1,2,3,4} is the adaptive weight coefficient learned during the network optimization process.

[0064] (5) The added feature F i ,i∈{1,2,3,4} is input to the multimodal cross fusion module MCFM for multi-channel and multi-level feature fusion;

[0065] Figure 3 This is a flow chart of the implementation of the multimodal cross fusion module MCFM described in the real-time example of the present invention, as shown in FIG. Figure 3As shown in the figure, in order to better capture more contextual information of each layer of features, a fusion unit module (RU) is used to expand the receptive field of the feature map of each level. In the fusion unit module, the receptive field of the feature map of each level is increased through convolution operations of different sizes, and then weighted fusion is performed through parameters. Finally, an upsampling operation is used to splice the sampled feature map with the feature map of the previous level. The fusion unit module consists of a 1×1 convolution, a 3×3 convolution, and two 5×5 convolutions. Taking into account the different requirements for contextual information of features at different levels, four learnable parameters are introduced. These parameters are automatically learned during the network optimization process. Through the learnable parameters, the fusion ratio of feature maps at different levels can be controlled to ensure that the contribution of features at each level can be fully reflected in the final fused feature map. The formula can be expressed as:

[0066] F si =α i *Conv1(F i )+β i *Conv3(F i )+γ i *Conv5(F i )+δ i *Conv5(F i );

[0067] Among them, F si and F i They represent the output features and input features of the fusion unit of the i-th layer (i=1, 2, 3, 4), α i , β i , γ i and δ i denotes learnable parameters, Conv1(·), Conv3(·), and Conv5(·) denote 1×1, 3×3, and 5×5 convolution operations, respectively.

[0068] (6) The fusion feature F obtained in step 5 si ,i∈{1,2,3,4} is input to the fusion refinement module for refinement to obtain a fine feature map F f ;

[0069] The fusion refinement module FRM described in the real-time example of the present invention is as follows Figure 4 As shown in Figure 1, first, three 1×1 convolutions are used to extract the query (Q), key (K), and value information (V) of the input feature map. The output channels of the first two convolutions are reduced to one eighth of the original channels, which can reduce the computational cost to a certain extent. By using the prior knowledge of the coarse saliency map, the converted probability map (S C) is multiplied with the query (Q), key (K), and value information (V) in sequence to obtain the enhanced query (Q), key (K), and value information (V). The formula is as follows:

[0070] Q=S C ×Q,K=S C ×K,V=S C ×V;

[0071] Then the enhanced three-dimensional query Q∈R C×H×W Reconstruct into a two-dimensional matrix q∈R C×N , where N = H × W, and the formula for transposing q is as follows:

[0072] q = Transpose(Reshape(Q));

[0073] At the same time, the enhanced three-dimensional bond information K∈R C×H×W Reconstruct into a two-dimensional matrix k∈R C×N , the formula is as follows:

[0074] k = Reshape(K);

[0075] Perform matrix multiplication on q and k and use the Softmax function to obtain the mapping R representing the long-range dependency relationship. The formula is as follows:

[0076]

[0077] After obtaining the relationship mapping R, the three-dimensional value information V∈R C×H×W Reconstruct into a two-dimensional matrix v∈R C×N , then perform matrix multiplication on the two-dimensional matrix v and the mapping R to obtain the enhanced matrix, the formula is as follows:

[0078] v = Reshape(V);

[0079]

[0080] Reconstruct the enhanced matrix E to its original shape and size to obtain the enhanced features F e , in order to retain the original information, and F s Add element by element to get the final output feature F f :

[0081] F e =Reshape(E);

[0082] F f =F e +F s .

[0083] (7) The enhanced feature map F obtained in step 6f Perform convolution to obtain the disguised target prediction map S, and perform deep supervision on it through the binary label map of the disguised target;

[0084] Among them, the F obtained above f The final disguised target prediction map S is obtained through 3×3 convolution, and deep supervision is performed through the binary label map GT of the disguised target to calculate its structured loss. The selected loss function is a combination of weighted binary cross entropy loss, weighted IoU loss and dice loss to construct a loss function. The total loss function of the final model is:

[0085] L total =L BCE +L IoU +L dice ;

[0086] L BCE (P,G)=GlogP+(1-G)log(1-P);

[0087]

[0088] Among them, P represents the predicted situation, G represents the actual situation, and N represents the number of samples.

[0089] In order to verify the effectiveness of the present invention, the segmentation results of six mainstream camouflaged target segmentation models, SINetV2, ZoomNet, DGNet, DTINet, SegMaR, and ICON, were tested on some images of the camouflaged target datasets CAMO, CHAMELEON, COD10K, and NC4K, and compared with the camouflaged target prediction results of the present invention. Figure 5 As shown, Figure 5 (a) represents an RGB image, Figure 5 (b) represents the Depth depth image, Figure 5 (c) represents the GT disguised target saliency label, Figure 5 (d) shows the camouflaged target prediction result of Ours invention. Figure 5 (e)- Figure 5 (j) represents the segmentation results of some images of six mainstream camouflaged target segmentation models SINetV2, ZoomNet, DGNet, DTINet, SegMaR, and ICON on the camouflaged target datasets CAMO, CHAMELEON, COD10K, and NC4K. By comparison, it can be seen that the proposed method can effectively detect camouflaged targets and improve the accuracy of camouflaged target detection. It has obvious advantages over other segmentation models in more complex situations.

[0090] This invention combines global and local feature extraction, using the PVTv2 network as the backbone network for feature extraction, capturing global information and obtaining more comprehensive context. Channel-wise and spatial-wise attention mechanisms enhance features at specific channels and spatial locations, allowing global context to be considered when processing local details. A multimodal cross-fusion module fuses features at different levels, combining local details with global information. A fusion refinement module further integrates global and local information to guide the final segmentation results, thereby improving camouflaged target detection performance. This invention introduces depth images and fuses depth features with RGB features for camouflaged target detection. Through the multimodal cross-fusion module, RGB and depth features can share and complement information at multiple levels. The fused features not only preserve the target's appearance but also, through depth features, better perceive the target's spatial position and three-dimensional structure, enhancing the model's recognition capabilities in complex scenes. This fusion strategy addresses the vulnerability of single modalities to changes in lighting, color, or texture. It also enhances the perception of depth differences between the target and the background, preventing camouflaged targets from blending into the background, thereby achieving higher accuracy and reliability in camouflaged target detection.

[0091] The present invention also provides a camouflaged target detection system, comprising an image acquisition module, a model construction module, and a camouflaged target detection module. The image acquisition module is used to acquire an RGB image and a depth image of a camouflaged target to be detected; the model construction module is used to construct a camouflaged target detection model, which includes a backbone network PVTv2 network, multiple depth feature enhancement modules, a multimodal cross-fusion module, and a fusion refinement module; the camouflaged target detection module is used to simultaneously input the RGB image and the depth image into the camouflaged target detection model, perform multi-layer RGB feature extraction and multi-layer depth feature extraction in the backbone network PVTv2 with two shared weights, and perform feature enhancement processing on the depth features of each layer using the depth feature enhancement module; perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature map using the multimodal cross-fusion module, and then enhance and refine the fused feature map using the fusion refinement module to obtain a detected camouflaged target image.

[0092] The present invention also provides a computer device, comprising a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute a camouflaged target detection method.

[0093] The present invention also provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded by a processor to execute the disguised target detection method.

[0094] The above-described embodiments are only preferred specific implementation methods of the present invention, and the protection scope of the present invention is not limited thereto. Any simple changes or equivalent replacements of the technical solutions that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention fall within the protection scope of the present invention.

Claims

1. A method for detecting a disguised target, characterized in that: The steps include: Obtain the RGB image and depth image of the camouflaged target to be detected; Construct a disguised target detection model, which includes the backbone network PVTv2 network, deep feature enhancement module, multimodal cross fusion module and fusion refinement module; The RGB image and the depth image are simultaneously input into the camouflaged target detection model. The multi-layer RGB feature extraction and the multi-layer depth feature extraction are performed through the two weight-sharing backbone networks PVTv2. The depth feature enhancement module is used to enhance the depth features of each layer. The multi-modal cross fusion module is used to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature maps. The fusion refinement module is then used to refine the fused feature maps F. s Enhance and refine to obtain the detected camouflaged target image; The fusion refinement module is used to refine the fused feature map F s Enhancement refinement includes the following steps: Use three 1×1 convolutions to extract the query information Q, key information K, and value information V of the input feature map respectively; The query information Q, key information K and value information V of the input feature map are enhanced respectively to obtain the three-dimensional query information Q∈R C×H×W , three-dimensional bond information K∈R C×H×W and three-dimensional value information V∈R C×H×W ; For three-dimensional query information Q∈R C×H×W , three-dimensional bond information K∈R C×H×W and three-dimensional value information V∈R C×H×W Reconstruct and obtain the corresponding two-dimensional matrix q∈R C×N , k∈R C×N and v∈R C×N ; For a two-dimensional matrix q∈R C×N and k∈R C×N Perform matrix multiplication to obtain a mapping R representing the long-range dependency relationship, and for the two-dimensional matrix v∈R C×N Perform matrix multiplication with the mapping R to obtain the enhanced matrix E; Reconstruct the enhanced matrix E to its original shape and size to obtain the enhanced features F e , and F e With the F s Add element by element to get the final output feature F f .

2. The method for detecting a disguised target according to claim 1, wherein: The deep feature enhancement module includes a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism is used to enhance the deep features. Specifically, the input deep features are subjected to maximum pooling and average pooling based on width and height to obtain two 1×1×C feature maps. The number of channels is then adjusted through a shared multi-layer perceptron (Share MLP). The output feature maps are summed and then subjected to a Sigmoid activation function to obtain channel information. Finally, the channel information is multiplied by the input deep features to obtain the output feature map of the channel attention mechanism. The spatial attention mechanism is used to enhance the deep features. Specifically, the output feature map of the channel attention mechanism is subjected to channel-based maximum pooling and average pooling to obtain two H×W×1 feature maps, which are then concatenated from the channel direction. The feature map is reduced in dimension by 7×7 convolution, and the weight parameters are generated by a Sigmoid function to obtain spatial information.

3. The method for detecting a disguised target according to claim 2, wherein: After obtaining the spatial information, the method further includes multiplying the spatial information with the output feature map of the channel attention mechanism to obtain the output feature map of the hybrid attention mechanism.

4. The method for detecting a disguised target according to claim 1, wherein: The multi-modal cross fusion module is used to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature maps. Specifically, the fusion unit RU of the multi-modal cross fusion module is used to perform convolution operations of different sizes to expand the receptive field of each layer of feature maps, and then perform weighted fusion through parameters. Finally, an upsampling operation is used to splice the sampled feature map with the feature map of the previous layer to obtain the fused feature map F. s .

5. The method for detecting a disguised target according to claim 1, wherein: Before the RGB image and the depth image are respectively input into the two weight-sharing backbone networks PVTv2 for multi-layer RGB feature extraction and multi-layer depth feature extraction, the camouflaged target detection model is trained. In the training phase, the sample batch size of the input model is set to 16, the number of epochs is set to 200, and the initial learning rate is set to 10 -4 .

6. A disguised target detection system, characterized in that: include: An image acquisition module is used to acquire an RGB image and a depth image of the camouflaged target to be detected; The model building module is used to build a disguised target detection model. The disguised target detection model includes a backbone network PVTv2 network, multiple deep feature enhancement modules, a multimodal cross fusion module, and a fusion refinement module. The disguised target detection module is used to input the RGB image and the depth image into the disguised target detection model at the same time. It uses the two weight-sharing backbone networks PVTv2 to perform multi-layer RGB feature extraction and multi-layer depth feature extraction. The depth feature enhancement module uses the depth feature enhancement module to enhance the depth features of each layer. The multi-modal cross fusion module is used to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature map, and then the fusion refinement module is used to enhance and refine the fused feature map to obtain the detected camouflaged target image; the fusion refinement module is used to enhance and refine the fused feature map F s Enhancement and refinement include the following steps: using three 1×1 convolutions to extract the query information Q, key information K and value information V of the input feature map respectively; enhancing the query information Q, key information K and value information V of the input feature map respectively to obtain the three-dimensional query information Q∈R C×H×W , three-dimensional bond information K∈R C×H×W and three-dimensional value information V∈R C×H×W ; respectively for the three-dimensional query information Q∈R C×H×W , three-dimensional bond information K∈R C×H×W and three-dimensional value information V∈R C×H×W Reconstruct and obtain the corresponding two-dimensional matrix q∈R C×N , k∈R C×N and v∈R C×N ; For a two-dimensional matrix q∈R C×N and k∈R C×N Perform matrix multiplication to obtain the mapping R representing the long-range dependency relationship, and for the two-dimensional moment v∈R C×N Perform matrix multiplication on the matrix and mapping R to obtain the enhanced matrix E; reconstruct the enhanced matrix E to its original shape and size to obtain the enhanced feature F e , and F e With the F s Add element by element to get the final output feature F f .

7. A computer device, characterized in that: It includes a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the camouflaged target detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the camouflaged target detection method according to any one of claims 1 to 5.

Images