Camouflage target detection method, system and device and medium

Abstract

The invention provides a camouflage target detection method, system and device and a medium, and belongs to the field of camouflage target detection, and the method comprises the following steps: obtaining an RGB image and a depth image of a to-be-detected camouflage target; constructing a camouflage target detection model, wherein the camouflage target detection model comprises a backbone network PVTv2 network, a depth feature enhancement module, a multi-modal cross fusion module and a fusion refining module; respectively inputting the RGB image and the depth image into two weight-shared backbone networks PVTv2 to carry out multi-layer RGB feature extraction and multi-layer depth feature extraction, and carrying out feature enhancement processing on the depth feature of each layer by using a depth feature enhancement module; and performing multi-channel and multi-level feature fusion on the multi-layer RGB features and the feature map after enhancement processing by using a multi-modal cross fusion module, and performing enhancement and refinement on the fused feature map by using a fusion refinement module to obtain a camouflage target image. The detection precision of the camouflage target can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of disguised target detection, and in particular relates to a disguised target detection method, system, equipment and medium. Background Art

[0002] Camouflage is a very common phenomenon in nature and a common means in military operations. It has an important impact on ecological balance, national defense security and other fields. The purpose of camouflaged target detection is to identify camouflaged targets hidden in the environment. These targets may be integrated with the environment through natural or artificial means. Unlike conventional salient targets, camouflaged targets represent a class of non-salient targets with insufficient semantic information and lack of features. Such targets have a high degree of integration with the surrounding background environment, contain less pixel information in the image, have low resolution and poor visual edge, which makes detection very complicated. However, the practicability of camouflaged target detection technology is very high, and it is widely used in medical image analysis, crop locust monitoring, video polyp segmentation, rescue assistance and military security. Therefore, improving the performance and accuracy of camouflaged target detection is one of the important technical topics in this field.

[0003] At present, camouflaged object detection for RGB images has received extensive attention and research. The paper "Camouflaged object detection. In CVPR, 2020" proposes that SINet introduces a receptive field module to expand the local receptive field, thereby promoting the segmentation of camouflaged objects. However, although the receptive field module effectively enhances the extraction of local features, it still has a significant limitation, that is, it cannot fully capture global features. The paper "Attention-induced semantic and boundary interaction network for camouflaged object detection. Computer Vision and Image Understanding, 2023" proposes an attention-induced semantic and boundary interaction network, which retains more semantic information and generates more powerful classification features, but does not consider targets hidden by specific color or texture patterns, and cannot capture the deep information of these features. The Chinese invention patent with patent number CN116228702A discloses a camouflaged object detection method based on an attention mechanism and a convolutional neural network. Although it considers aggregating multi-layer features and refining boundary contours, it does not consider the complexity of the camouflaged background and ignores spatial information.

[0004] In summary, the existing camouflaged target detection algorithms are unable to capture the complex and deep information of the camouflaged background, resulting in low camouflaged target detection accuracy. Summary of the invention

[0005] In order to overcome the above-mentioned disadvantage of low detection accuracy of the existing disguised target, the present invention provides a disguised target detection method, comprising the following steps:

[0006] Obtain the RGB image and depth image of the camouflaged target to be detected;

[0007] Construct a disguised target detection model, which includes a backbone network PVTv2 network, multiple deep feature enhancement modules, a multimodal cross fusion module, and a fusion refinement module;

[0008] The RGB image and the depth image are simultaneously input into the camouflaged target detection model, and multi-layer RGB feature extraction and multi-layer depth feature extraction are performed through two weight-sharing backbone networks PVTv2. The depth feature enhancement module is used to enhance the depth features of each layer. The multi-modal cross-fusion module is used to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature maps, and the fusion refinement module is used to enhance and refine the fused feature maps to obtain the detected camouflaged target image.

[0009] Preferably, the deep feature enhancement module includes a channel attention mechanism and a spatial attention mechanism, and uses the channel attention mechanism to perform feature enhancement processing on the deep features, specifically: performing maximum pooling and average pooling based on width and height on the input deep features to obtain two 1×1×C feature maps, and then adjusting the number of channels through the shared multi-layer perceptron ShareMLP, and adding the output feature maps, and then obtaining channel information through the Sigmoid activation function, and finally multiplying the channel information with the input deep features to obtain the output feature map of the channel attention mechanism;

[0010] The spatial attention mechanism is used to enhance the deep features. Specifically, the output feature map of the channel attention mechanism is subjected to channel-based maximum pooling and average pooling to obtain two H×W×1 feature maps, which are concatenated from the channel direction. The feature map is reduced in dimension through 7×7 convolution, and the weight parameters are generated through a Sigmoid function to obtain spatial information.

[0011] Preferably, after obtaining the spatial information, the method further includes multiplying the spatial information with the output feature map of the channel attention mechanism to obtain the output feature map of the hybrid attention mechanism.

[0012] Preferably, the multi-modal cross fusion module is used to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the feature map after enhancement processing, specifically: the fusion unit RU of the multi-modal cross fusion module is used to perform convolution operations of different sizes, expand the receptive field of each layer of the feature map, and then perform weighted fusion through parameters, and finally use the upsampling operation to splice the sampled feature map with the feature map of the previous level to obtain the fused feature map F s .

[0013] Preferably, the step of enhancing and refining the fused feature map using a fusion and refinement module comprises the following steps:

[0014] Use three 1×1 convolutions to extract the query information Q, key information K, and value information V of the input feature map respectively;

[0015] The query information Q, key information K and value information V of the input feature map are enhanced respectively to obtain the three-dimensional query information Q∈R C×H×W , three-dimensional bond information K∈R C×H×W and three-dimensional value information V∈R C×H×W ;

[0016] For the three-dimensional query information Q∈R C×H×W , three-dimensional bond information K∈R C×H×W and three-dimensional value information V∈R C×H×W Reconstruct and obtain the corresponding two-dimensional matrix q∈R C×N , k∈R C×N and v∈R C×N ;

[0017] For a two-dimensional matrix q∈R C×N and k∈R C×N Perform matrix multiplication to obtain a mapping R representing the long-range dependency relationship. For the two-dimensional matrix v∈R C×N Perform matrix multiplication with the mapping R to obtain the enhanced matrix E;

[0018] Reconstruct the enhanced matrix E to its original shape and size to obtain the enhanced features F e , and F e With F s Add element by element to get the final output feature F f .

[0019] Preferably, before the RGB image and the depth image are respectively input into two weight-sharing backbone networks PVT-V2 for multi-layer RGB feature extraction and multi-layer depth feature extraction, the method further includes training the camouflage target detection model. In the training stage, the sample batch size of the input model is set to 16, the number of epochs is set to 200, and the initial learning rate is set to 10 -4 .

[0020] The present invention also provides a camouflaged target detection system, comprising:

[0021] An image acquisition module is used to acquire an RGB image and a depth image of a camouflaged target to be detected;

[0022] A model building module is used to build a disguised target detection model. The disguised target detection model includes a backbone network PVTv2 network, multiple deep feature enhancement modules, a multi-modal cross fusion module, and a fusion refinement module;

[0023] The camouflaged target detection module is used to input the RGB image and the depth image into the camouflaged target detection model at the same time, perform multi-layer RGB feature extraction and multi-layer depth feature extraction by inputting two weight-sharing backbone networks PVTv2, and use the depth feature enhancement module to perform feature enhancement processing on the depth features of each layer; use the multimodal cross-fusion module to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature map, and then use the fusion refinement module to enhance and refine the fused feature map to obtain the detected camouflaged target image.

[0024] The present invention also provides a computer device, comprising a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the camouflaged target detection method.

[0025] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the camouflaged target detection method.

[0026] The camouflaged target detection method, system, device and medium provided by the present invention have the following beneficial effects:

[0027] The present invention simultaneously extracts features from RGB images and depth images in two weight-sharing backbone networks PVTv2, and can obtain multi-scale RGB features and multi-scale depth features containing camouflaged target images. The RGB features and depth features can share and complement information at multiple levels; by using a deep feature enhancement module to perform feature enhancement processing on the depth features of each layer, the compatibility of the depth features can be enhanced, and the multi-level RGB features can be retained, which is conducive to feature fusion and improves the accuracy of camouflaged target detection; by using a multi-modal cross-fusion module to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the feature map after enhancement processing, the fused features can not only retain the appearance information of the target, but also better perceive the spatial position and three-dimensional structure of the target through the depth features, so that the model has stronger recognition ability in complex scenes, which can not only solve the problem that a single mode is easily affected by changes in illumination, color or texture, but also enhance the perception of the depth difference between the target and the background, avoid the situation where the camouflaged target is fused with the background, thereby achieving higher accuracy and reliability in camouflaged target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 is a flow chart of a camouflaged target detection method according to an embodiment of the present invention;

[0030] Figure 2 This is the flow chart of the deep feature enhancement module;

[0031] Figure 3 This is the flow chart of the multimodal cross-fusion module;

[0032] Figure 4 To refine the module flow chart for integration;

[0033] Figure 5 The results of detecting different camouflaged target images using six mainstream camouflaged target segmentation models are compared with the method of the present invention, where: Figure 5 (a) represents an RGB image, Figure 5 (b) represents the Depth image, Figure 5 (c) represents the GT disguised target saliency label, Figure 5 (d) represents the camouflaged target prediction result of Ours invention. Figure 5 (e)- Figure 5(j) represents the segmentation results of some images of six mainstream camouflaged target segmentation models SINetV2, ZoomNet, DGNet, DTINet, SegMaR, and ICON on the camouflaged target datasets CAMO, CHAMELEON, COD10K, and NC4K. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.

[0035] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the technical solutions of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0036] In addition, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless otherwise clearly specified or limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. In the description of the present invention, unless otherwise specified, "plurality" means two or more, which will not be described in detail here.

[0037] Example

[0038] The present invention provides a method for detecting a camouflaged target, specifically Figure 1 As shown, the following steps are included:

[0039] Step 1: Obtain the RGB image and depth image of the camouflaged target to be detected.

[0040] Step 2: Construct a disguised target detection model. The disguised target detection model includes a backbone network PVTv2 network, a deep feature enhancement module, a multimodal cross fusion module, and a fusion refinement module. The backbone network is the PVTv2 network in the Transformer structure.

[0041] Step 3: Input the RGB image and the depth image into the camouflaged target detection model at the same time, perform multi-layer RGB feature extraction and multi-layer depth feature extraction through two weight-sharing backbone networks PVTv2, and use the deep feature enhancement module to enhance the depth features of each layer; use the multimodal cross-fusion module to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature maps, and then use the fusion refinement module to enhance and refine the fused feature maps to obtain the detected camouflaged target image.

[0042] The overall structure of the camouflaged target detection model is as follows: Figure 1 As shown in the figure, the backbone network responsible for extracting features is the PVTv2 network in the Transformer structure. Four layers of features are extracted from the RGB image and the depth image respectively through two weight-sharing backbone networks PVTv2, and then feature enhancement is performed through the deep feature enhancement module. In order to avoid the semantic information in the original RGB image features being blurred by the useless information in the deep features, thereby affecting the model accuracy, an adaptive fusion strategy is adopted. In the feature fusion process, different weights are assigned to the RGB features and the depth features to identify the dominant mode. The feature formula can be expressed as:

[0043] F i =α i F ri +β i F DFEM (F di );

[0044] Among them, F i represents the fusion feature of the i-th (i=1,2,3,4) layer, F ri represents the RGB features of the i-th (i=1,2,3,4) layer, F di represents the depth feature of the i-th (i=1,2,3,4) layer, F DFEM (·) represents the features enhanced by the deep feature enhancement module. α i and β i It is the adaptive weight coefficient learned during the network optimization process.

[0045] Then, F iThe features are input into the multimodal cross fusion module (MCFM) for multi-channel and multi-level feature fusion. In the multimodal cross fusion module, the receptive field of each layer of features is gradually expanded and spliced ​​to the previous layer of features through upsampling. Then the output fusion features are captured by the fusion refinement module (FRM) guided by semantic position information to obtain more spatial fine-grained information. Finally, 3×3 convolution is used to combine the output features to obtain the refined prediction map S.

[0046] Among them, DFEM is a deep feature enhancement module, and its structure is as follows Figure 2 As shown in the figure, the module is composed of a channel attention mechanism and a spatial attention mechanism, in which each layer of deep features passes through a deep feature enhancement module to enhance the compatibility of deep features while retaining multi-level information. The DFEM module includes a channel attention mechanism and a spatial attention mechanism, and its formula is as follows:

[0047] F DFEM (F di )=S att (C att (F di ));

[0048] Among them, C att (·) represents channel attention, S att (·) represents spatial attention, and their specific expression formulas are as follows:

[0049]

[0050] Among them, MaxPool(·) represents the global maximum pooling feature of each feature map, AvgPool(·) represents the global average pooling feature of each feature map, MLP(·) represents multi-layer perceptron, σ represents the Sigmoid activation function, and F represents the input feature map.

[0051] The detailed process of obtaining the camouflaged target image of the present invention is as follows:

[0052] (1) Train the camouflaged target detection model as follows:

[0053] Obtain a disguised target image dataset, and divide the disguised target image dataset into a training set and a test set;

[0054] The experimental platform is carried out under the environment configuration of Cuda10.2, PyTorch1.12, Python3.8, and GPU is NVIDIAGeForce RTXA5000. All input images in the training set are resized to 352×352 and input into the disguised target detection model to train the disguised target detection model. In the training stage, the sample batch size of the input model is set to 16, the number of epochs is set to 200, and the initial learning rate is set to 10 -4 .

[0055] (2) The RGB image and depth image of the camouflaged target to be detected are input into the trained camouflaged target detection model, and the two weight-sharing backbone networks PVTv2 of the trained camouflaged target detection model are used to extract the multi-scale RGB features F containing the camouflaged target image. ri ,i∈{1,2,3,4} and multi-scale deep features F di ,i∈{1,2,3,4}.

[0056] (3) The deep features F extracted from the backbone network di ,i∈{1,2,3,4} are respectively input into the deep feature enhancement module DFEM for feature enhancement;

[0057] The implementation flow chart of the deep feature enhancement module DFEM described in the example of the present invention is as follows: Figure 2 As shown in the figure, the module is composed of a channel attention mechanism and a spatial attention mechanism, in which each layer of deep features passes through a deep feature enhancement module to enhance the compatibility of deep features while retaining multi-level information. The DFEM module contains a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism gives different weights to different channels. First, the input feature map is subjected to maximum pooling and average pooling based on width and height to obtain two 1×1×C feature maps. Then, the number of channels is adjusted through a shared multi-layer perceptron Share MLP, and then the output is added and the channel information is obtained after the Sigmoid activation function. Finally, the channel information is multiplied with the input feature map to obtain the output feature map of the channel attention mechanism. The spatial attention mechanism gives different weights to different spaces. First, the feature map is subjected to maximum pooling and average pooling based on the channel to obtain two H×W×1 feature maps, which are spliced ​​from the channel direction. Then, the feature map is reduced in dimension through a 7×7 convolution, and a weight parameter is generated through a Sigmoid function to obtain spatial information. Finally, the spatial information is multiplied by the output feature map of the channel attention mechanism to obtain the output feature map of the hybrid attention mechanism. The output after enhanced features can be expressed as:

[0058]

[0059] F DFEM (F di )=S att (C att (F di ));

[0060] Among them, MaxPool(·) represents the global maximum pooling feature of each feature map, AvgPool(·) represents the global average pooling feature of each feature map, MLP(·) represents multi-layer perceptron, σ represents the Sigmoid activation function, and F represents the input feature map.

[0061] (4) The RGB feature F ri ,i∈{1,2,3,4} and the depth feature F after depth enhancement DFEM (F di ), i∈{1,2,3,4} are assigned different weights α i ,β i , i∈{1,2,3,4} is added element by element, and its characteristic formula can be expressed as:

[0062] F i =α i F ri +β i F DFEM (F di );

[0063] Among them, F i represents the fusion feature of the i-th (i=1,2,3,4) layer, F ri represents the RGB features of the i-th (i=1,2,3,4) layer, F di represents the depth feature of the i-th (i=1,2,3,4) layer, F DFEM (·) represents the features enhanced by the deep feature enhancement module. Adaptive parameter α i ,β i ,i∈{1,2,3,4} is the adaptive weight coefficient learned during the network optimization process.

[0064] (5) The added feature F i ,i∈{1,2,3,4} is input to the multi-modal cross fusion module MCFM for multi-channel and multi-level feature fusion;

[0065] Figure 3 FIG. 1 is a flow chart of the implementation of the multi-modal cross fusion module MCFM described in the real-time example of the present invention, as shown in FIG. Figure 3As shown in the figure, in order to better capture more contextual information of each layer of features, a fusion unit module (RU) is used to expand the receptive field of the feature map of each level. In the fusion unit module, the receptive field of the feature map of each level is increased through convolution operations of different sizes, and then weighted fusion is performed through parameters. Finally, an upsampling operation is used to splice the sampled feature map with the feature map of the previous level. The fusion unit module consists of a 1×1 convolution, a 3×3 convolution, and two 5×5 convolutions. Taking into account the different requirements for contextual information of features at different levels, four learnable parameters are introduced, which are automatically learned during the network optimization process. Through the learnable parameters, the fusion ratio of feature maps at different levels can be controlled to ensure that the contribution of features at each level can be fully reflected in the final fused feature map. The formula can be expressed as:

[0066] F si =α i *Conv1(F i )+β i *Conv3(F i )+γ i *Conv5(F i )+δ i *Conv5(F i );

[0067] Among them, F si and F i They represent the output features and input features of the fusion unit of the i-th layer (i=1, 2, 3, 4), α i , β i , γ i and δ i denotes learnable parameters, Conv1(·), Conv3(·), and Conv5(·) denote 1×1, 3×3, and 5×5 convolution operations, respectively.

[0068] (6) The fusion feature F obtained in step 5 si , i∈{1,2,3,4} is input to the fusion refinement module for refinement to obtain a fine feature map F f ;

[0069] The fusion refinement module FRM described in the real-time example of the present invention is as follows Figure 4 As shown in Figure 1, first, three 1×1 convolutions are used to extract the query (Q), key (K), and value information (V) of the input feature map. The output channels of the first two convolutions are reduced to one eighth of the original channels, which can reduce the computational cost to a certain extent. By using the prior knowledge of the coarse saliency map, the converted probability map (S C) is multiplied with the query (Q), key (K), and value information (V) in sequence to obtain the enhanced query (Q), key (K), and value information (V), and the formula is as follows:

[0070] Q=S C ×Q,K=S C ×K,V=S C ×V;

[0071] Then, the enhanced three-dimensional query Q∈R C×H×W Reconstruct into a two-dimensional matrix q∈R C×N , where N = H × W, the formula for transposing q is as follows:

[0072] q = Transpose(Reshape(Q));

[0073] At the same time, the enhanced three-dimensional bond information K∈R C×H×W Reconstruct into a two-dimensional matrix k∈R C×N , the formula is as follows:

[0074] k = Reshape(K);

[0075] Perform matrix multiplication on q and k and use the Softmax function to get the mapping R representing the long-range dependency, as shown in the following formula:

[0076]

[0077] After obtaining the relationship mapping R, the three-dimensional value information V∈R C×H×W Reconstruct into a two-dimensional matrix v∈R C×N , then perform matrix multiplication on the two-dimensional matrix v and the mapping R to obtain the enhanced matrix, the formula is as follows:

[0078] v = Reshape(V);

[0079]

[0080] Reconstruct the enhanced matrix E to its original shape and size to obtain the enhanced features F e , in order to preserve the original information, and F s Add element by element to get the final output feature F f :

[0081] F e =Reshape(E);

[0082] F f =F e +F s .

[0083] (7) The enhanced feature map F obtained in step 6f Perform convolution to obtain the disguised target prediction map S, and perform deep supervision on it through the binary label map of the disguised target;

[0084] Among them, the F obtained above f The final disguised target prediction map S is obtained through 3×3 convolution, and deep supervision is performed through the binary label map GT of the disguised target to calculate its structured loss. The selected loss function is the weighted binary cross entropy loss, weighted IoU loss and dice loss combined to construct the loss function. The total loss function of the final model is:

[0085] L total =L BCE +L IoU +L dice ;

[0086] L BCE (P,G)=GlogP+(1-G)log(1-P);

[0087]

[0088] Among them, P represents the predicted situation, G represents the actual situation, and N represents the number of samples.

[0089] In order to verify the effectiveness of the present invention, the segmentation results of some images of six mainstream camouflaged target segmentation models, SINetV2, ZoomNet, DGNet, DTINet, SegMaR, and ICON, were tested on the camouflaged target datasets CAMO, CHAMELEON, COD10K, and NC4K, and compared with the camouflaged target prediction results of the present invention. Figure 5 As shown, Figure 5 (a) represents an RGB image, Figure 5 (b) represents the Depth image, Figure 5 (c) represents the GT disguised target saliency label, Figure 5 (d) represents the camouflaged target prediction result of Ours invention. Figure 5 (e)- Figure 5 (j) represents the segmentation results of some images of six mainstream camouflaged target segmentation models SINetV2, ZoomNet, DGNet, DTINet, SegMaR, and ICON in the camouflaged target datasets CAMO, CHAMELEON, COD10K, and NC4K. By comparison, it can be seen that the present invention can effectively detect camouflaged targets and improve the accuracy of camouflaged target detection. It has obvious advantages over other segmentation models in more complex situations.

[0090] The present invention combines the extraction of global features and local features, uses the PVTv2 network as the backbone network to extract features, capture global information, and obtain more comprehensive context information; the channel attention mechanism and the spatial attention mechanism can enhance the features of specific channels and spatial positions, and the global context can also be considered when processing local details; the multimodal cross-fusion module is used to fuse features at different levels, and local details and global information are combined; and the fusion refinement module is used to further combine global and local information to guide the final segmentation result, thereby improving the performance of camouflaged target detection. The present invention introduces a depth image and fuses the depth feature with the RGB feature to perform camouflaged target detection. Through the multimodal cross-fusion module, RGB and depth features can share and complement information at multiple levels. The fused features can not only retain the appearance information of the target, but also better perceive the spatial position and three-dimensional structure of the target through the depth feature, so that the model has stronger recognition ability in complex scenes. This fusion strategy can solve the problem that a single modality is easily affected by changes in illumination, color or texture, while enhancing the perception of the depth difference between the target and the background, avoiding the situation where the camouflaged target is fused with the background, thereby achieving higher accuracy and reliability in camouflaged target detection.

[0091] The present invention also provides a camouflaged target detection system, including an image acquisition module, a model construction module and a camouflaged target detection module. The image acquisition module is used to acquire an RGB image and a depth image of a camouflaged target to be detected; the model construction module is used to construct a camouflaged target detection model, and the camouflaged target detection model includes a backbone network PVTv2 network, multiple depth feature enhancement modules, a multimodal cross fusion module and a fusion refinement module; the camouflaged target detection module is used to simultaneously input the RGB image and the depth image into the camouflaged target detection model, perform multi-layer RGB feature extraction and multi-layer depth feature extraction in the backbone network PVTv2 with two weights shared, and use the depth feature enhancement module to perform feature enhancement processing on the depth features of each layer; use the multimodal cross fusion module to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the feature map after enhancement processing, and then use the fusion refinement module to enhance and refine the fused feature map to obtain the detected camouflaged target image.

[0092] The present invention also provides a computer device, comprising a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute a camouflaged target detection method.

[0093] The present invention also provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded by a processor to execute the camouflaged target detection method.

[0094] The embodiments described above are only preferred specific implementation modes of the present invention, and the protection scope of the present invention is not limited thereto. Any simple changes or equivalent replacements of the technical solutions that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention belong to the protection scope of the present invention.

Claims

1. A camouflaged target detection method, characterized in that: The steps include: Obtain the RGB image and depth image of the camouflaged target to be detected; Construct a disguised target detection model, which includes a backbone network PVTv2 network, a deep feature enhancement module, a multimodal cross fusion module, and a fusion refinement module; The RGB image and the depth image are simultaneously input into the camouflaged target detection model, and multi-layer RGB feature extraction and multi-layer depth feature extraction are performed through two weight-sharing backbone networks PVTv2. The depth feature enhancement module is used to enhance the depth features of each layer. The multi-modal cross-fusion module is used to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature maps, and the fusion refinement module is used to enhance and refine the fused feature maps to obtain the detected camouflaged target image.

2. The camouflaged target detection method according to claim 1, characterized in that: The deep feature enhancement module includes a channel attention mechanism and a spatial attention mechanism, and uses the channel attention mechanism to perform feature enhancement processing on the deep features, specifically: performing maximum pooling and average pooling based on width and height on the input deep features to obtain two 1×1×C feature maps, and then adjusting the number of channels through the shared multi-layer perceptron Share MLP, and adding the output feature maps, and then obtaining the channel information through the Sigmoid activation function, and finally multiplying the channel information with the input deep features to obtain the output feature map of the channel attention mechanism; The spatial attention mechanism is used to enhance the deep features. Specifically, the output feature map of the channel attention mechanism is subjected to channel-based maximum pooling and average pooling to obtain two H×W×1 feature maps, which are concatenated from the channel direction. The feature map is reduced in dimension through 7×7 convolution, and the weight parameters are generated through a Sigmoid function to obtain spatial information.

3. The camouflaged target detection method according to claim 2, characterized in that: After obtaining the spatial information, the method further includes multiplying the spatial information with the output feature map of the channel attention mechanism to obtain the output feature map of the hybrid attention mechanism.

4. The camouflaged target detection method according to claim 1, characterized in that: The multi-modal cross fusion module is used to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the feature map after enhancement processing. Specifically, the fusion unit RU of the multi-modal cross fusion module is used to perform convolution operations of different sizes, expand the receptive field of each layer of feature map, and then perform weighted fusion through parameters. Finally, an upsampling operation is used to splice the sampled feature map with the feature map of the previous level to obtain a fused feature map F. s .

5. The camouflaged target detection method according to claim 1, characterized in that: The method of using the fusion refinement module to enhance and refine the fused feature map includes the following steps: Use three 1×1 convolutions to extract the query information Q, key information K, and value information V of the input feature map respectively; The query information Q, key information K and value information V of the input feature map are enhanced respectively to obtain the three-dimensional query information Q∈R C×H×W , three-dimensional bond information K∈R C×H×W and three-dimensional value information V∈R C×H×W ; For the three-dimensional query information Q∈R C×H×W , three-dimensional bond information K∈R C×H×W and three-dimensional value information V∈R C×H×W Reconstruct and obtain the corresponding two-dimensional matrix q∈R C×N , k∈R C×N and v∈R C×N ; For a two-dimensional matrix q∈R C×N and k∈R C×N Perform matrix multiplication to obtain a mapping R representing the long-range dependency relationship. For the two-dimensional matrix v∈R C×N Perform matrix multiplication with the mapping R to obtain the enhanced matrix E; Reconstruct the enhanced matrix E to its original shape and size to obtain the enhanced features F e , and F e With F s Add element by element to get the final output feature F f .

6. The camouflaged target detection method according to claim 1, characterized in that: Before the RGB image and the depth image are respectively input into two weight-sharing backbone networks PVTv2 for multi-layer RGB feature extraction and multi-layer depth feature extraction, the camouflage target detection model is also trained. In the training stage, the sample batch size of the input model is set to 16, the number of iterations epoch is set to 200, and the initial learning rate is set to 10 -4 .

7. A camouflaged target detection system, characterized in that: include: An image acquisition module is used to acquire an RGB image and a depth image of a camouflaged target to be detected; A model building module is used to build a disguised target detection model. The disguised target detection model includes a backbone network PVTv2 network, multiple deep feature enhancement modules, a multi-modal cross fusion module, and a fusion refinement module; The camouflaged target detection module is used to input the RGB image and the depth image into the camouflaged target detection model at the same time, perform multi-layer RGB feature extraction and multi-layer depth feature extraction through two weight-sharing backbone networks PVTv2, and use the depth feature enhancement module to enhance the depth features of each layer; A multi-modal cross-fusion module is used to perform multi-channel and multi-level feature fusion on the multi-layer RGB features and the enhanced feature maps, and then a fusion refinement module is used to enhance and refine the fused feature maps to obtain the detected camouflaged target image.

8. A computer device, characterized in that: It comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the camouflaged target detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the camouflaged target detection method according to any one of claims 1 to 6.

Images