A camouflaged target detection method based on edge shrinkage and exposure

Abstract

The present invention discloses a camouflaged target detection method based on edge contraction and exposure, comprising: in the training stage, for each training image in the training set and the truth map of the training image, training a target detection model, using the coding features output by the coding unit in the model to process through a conditional random field, obtaining an intermediate process loss and a feature map, decoding processing based on the feature map and the coding features and the features output by the adjacent decoding units in the decoding unit corresponding to the coding unit based on the edge exposure mechanism, obtaining the intermediate process loss, traversing each decoding unit to obtain the final output truth map, adjusting the weight parameters of the target detection model according to the intermediate process loss and the loss of the final truth map, obtaining the trained target detection model, and then detecting the image to be processed to obtain the output truth map. The method of the present invention can locate precise edges, solving the problem that edges cannot be accurately distinguished in the prior art.

Description

Technical Field

[0001] The invention relates to computer vision technology, and in particular to a camouflage target detection method based on edge contraction and exposure. Background Art

[0002] Camouflaged object detection is a complex technology that aims to identify objects that are seamlessly integrated into their environment. Research on COD (camouflaged object detection) improves the understanding of complex visual differences while driving innovations in practical applications in various fields. COD technology has a wide range of applications in various fields, from hidden defect detection in manufacturing to pest monitoring in agriculture. In addition, it also helps in medical image segmentation, etc. Accurately locating the edges of objects disguised as backgrounds is an extremely complex task. Existing camouflaged object detection (COD) methods can generally accurately locate the position of the object, but there is still a certain amount of ambiguity at its edges.

[0003] Existing COD can be roughly divided into two categories: single-stage methods and multi-stage methods. Single-stage methods try to accurately locate the position of an object immediately in each iteration, while multi-stage methods need to gradually determine the position of an object in multiple stages.

[0004] The above-mentioned single-stage methods, such as Camouflaged Object Detection, directly locate and segment the target through a single process. However, it can only accurately locate the camouflaged target through one stage, but it cannot effectively determine the edge position. The edges of the final segmentation pattern are often blurred. This makes the edges of the obtained segmentation pattern inaccurate. In the above-mentioned multi-stage method, SegMar (segmentation, magnification and iteration) relies on bionic zoom to observe edge details. It simply magnifies the image of edge details in multiple stages. It is difficult to achieve the purpose of truly accurate segmentation by simply targeting edge details and directly using auxiliary networks to predict edges. The edges are often blurred. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a camouflaged target detection method based on edge shrinkage and exposure.

[0007] (II) Technical solution

[0008] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0009] In a first aspect, an embodiment of the present invention provides a camouflaged target detection method based on edge shrinkage and exposure, comprising:

[0010] In the training phase, a true value map of each training image is obtained; each training image is input into a pre-built target detection model to obtain an output training true value map;

[0011] Obtaining a first loss based on the truth map and the training truth map, adjusting a weight parameter of the target detection model according to the first loss and a second loss of some features and the truth map during the target detection model processing, to obtain a target detection model for detecting a target image;

[0012] The target detection model includes: an encoding component and a decoding component; the encoding unit of the encoding component and the decoding unit of the decoding component correspond to each other in sequence, and the encoding component outputs n encoding features based on a training image;

[0013] The decoding component includes: n decoding units, the current decoding unit receives the decoding features output by the previous decoding unit, the preprocessed encoding features, and the feature map corresponding to the previous decoding unit, processes them using edge tightening and exposure attention mechanisms, and outputs the decoding features of the current decoding unit; the encoding features are the features output by the encoding unit corresponding to the current decoding unit, and the feature map is the feature map obtained by processing the decoding features output by the previous decoding unit using a conditional random field;

[0014] The first decoding unit is used to use the features output by the corresponding encoding unit as decoding features;

[0015] The last decoding unit is used to receive the decoding features output by the previous decoding unit and the unpreprocessed encoding features and the feature map corresponding to the previous decoding unit for processing, and output the training truth map;

[0016] In the process of each decoding unit acquiring the feature map, the loss of the intermediate feature and the true value map is used as the second loss, and the target edge in the intermediate feature of the decoding unit is gradually refined.

[0017] Optionally, the encoding component includes: n-1 encoding units connected in series and FPN as the nth encoding unit; the encoding units of the encoding component are used to encode the texture features of the surface layer of the training image to the semantic features of the deep layer layer layer by layer to obtain multi-level encoding features; n is a natural number greater than or equal to 1;

[0018] The first encoding unit receives the training image and outputs a first encoding feature;

[0019] The i-th encoding unit receives the i-1th encoding feature output by the i-1th encoding unit, and outputs the i-th encoding feature, where i is a natural number from 2 to n-1;

[0020] The FPN receives the encoding features output by the 2nd to n-1th encoding units to extract semantic information, and outputs the nth encoding feature.

[0021] Optionally, obtaining a true value map of each training image includes:

[0022] Binarize the training image to obtain the true value image of the segmentation map of the camouflaged target;

[0023] The truth map is subjected to maximum pooling processing and minimum pooling processing to obtain multiple edge probability maps, each edge probability map corresponds to a decoding unit, and the target edge in the edge probability map is gradually refined toward the output end, and the last decoding unit has no corresponding edge probability map; each edge probability map is used as a reference in the branch prediction process of the corresponding decoding unit to obtain intermediate loss information, and all the intermediate loss information constitutes the second loss.

[0024] Optionally, binarization is performed on the training image to obtain a true value image of the segmentation image of the camouflaged target;

[0025] When n=5, the maximum pooling layers of 20*20, 14*14, 12*12, 8*8, and 4*4 are used to downsample the truth image, and then the minimum pooling layers of 8*8, 7*7, 6*6, 5*5, and 4*4 are used to downsample the truth image. Then, the truth value processed by the minimum pooling layer is XORed with 1 to achieve black and white flipping. Finally, the images that pass through the maximum pooling layer and the minimum pooling layer are superimposed and then upsampled to 352×352 to obtain the edge probability map of the camouflaged target from coarse to fine. p is the label of the marginal probability map;

[0026] And / or, each of the n-1 coding units connected in series is a pyramid vision TransformerPVTv2.

[0027] Optionally, the first decoding unit is used to use the feature output by the corresponding encoding unit as a decoding feature, including:

[0028] When the nth encoding unit corresponds to the first decoding unit, the encoding feature f output by the nth encoding unit is n ; The decoding feature of the first decoding unit is O1;

[0029] The feature map obtained by processing the decoding feature O1 output by the first decoding unit using the conditional random field include:

[0030]

[0031] Among them, p1 = gConv(s(O1)), gConv means group convolution, concat means concatenation in the feature dimension, S1 is O1 obtained by group convolution; edge is the identifier of the feature map; s(O1) means O1 is processed by 1*1 convolution layer s;

[0032] When the i-th encoding unit corresponds to the k-th decoding unit, the feature map obtained by processing the decoding features output by the k-th decoding unit using the conditional random field include:

[0033]

[0034] O k is the decoding feature output by the kth decoding unit, S k O k Obtained through 1*1 convolutional layer s;

[0035] p k =gConv(t(S k-1 ,O k )concat s(O k ));

[0036] S k-1 is the output feature O of the k-1th decoding unit k-1 It is obtained by processing with 1*1 convolution layer s; i represents from large to small, k represents from small to large, both are natural numbers from 2 to n-1; t represents 1*1 convolution layer, t(S k-1 ,O k ) indicates S k-1 and O k After concatenation in the feature dimension, convolution is performed through a 1*1 convolution layer. k ) is processed by 1*1 convolution layer s k .

[0037] Optionally, the current k+1th decoding unit receives the decoding feature O output by the previous decoding unit k , Encoded features after feature preprocessing Feature map corresponding to the previous decoding unit The edge contraction and exposure attention mechanism is used to process and output the decoding feature O of the current decoding unit. k+1 ,include:

[0038]

[0039]

[0040] The subscript s indicates the identifier of the encoding feature jump connection output by the encoding unit, gConv indicates group convolution, concat indicates concatenation in the feature dimension, and E-SA and E-CA indicate intermediate parameters;

[0041]

[0042]

[0043] Q, K, V are respectively 0 k A long vector obtained by the fully connected layer in the k+1th decoding unit, E is A long vector obtained by the fully connected layer in the k+1th decoding unit; Softmax is a known activation function, T is the transpose; the dimensions of Q, K, V, and E are the same; d m is the dimension of the long vector, m has no meaning;

[0044] Yes Convolution processing and upsampling to 352*352, as well as the predicted true value edge image obtained by reducing the feature dimension.

[0045] Optionally, the total loss L total for:

[0046] L total =L pre +L est +λL esa ;

[0047]

[0048]

[0049]

[0050] L pre =∑ k PPA(O k ,gt);

[0051] Among them, L est The output features of the first decoding unit are processed at each stage of the conditional random field in the branch prediction process to obtain the predicted edge probability map p k and p k-1 The loss between est Also includes: predicted marginal probability map p k And the marginal probability map obtained by preprocessing the true value map The loss between, PPA and BCE are functions for finding the loss;

[0052] λ is the control Lesa The weight parameter accounting for the total loss, w, h are the predicted edge probability map p k The width and height, p k and p k-1 is the edge probability map in the branch prediction in the k-th decoding unit and the predicted edge probability map in the branch prediction in the k-1-th decoding unit, p k and p k-1 Consistent size, is the edge probability map corresponding to the kth decoding unit obtained by preprocessing the truth map;

[0053] L esa The first term is p k and fine edge y k The second term is to predict fine edges. and fine edge y k As loss, fine edge y k is the true value of the manually processed fine edge of the corresponding training image; gt refers to the true value map of the segmentation map of the camouflaged target.

[0054] In a second aspect, an embodiment of the present invention further provides a camouflaged target detection method based on edge shrinkage and exposure, comprising:

[0055] The disguised target image / target image to be detected is input into the trained target detection model to obtain the output truth map, i.e., the saliency map;

[0056] The target detection model is trained by any of the methods described in the first aspect above, with no loss in calculation during use, and no need to adjust the weight parameters of the model. There is no need to obtain the true value map, fine edge and edge probability map of the disguised target image to be detected during use.

[0057] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, the processor executes the computer program stored in the memory, and executes the steps of the camouflaged target detection method based on edge shrinkage and exposure as described in any of the first aspect or the second aspect above.

[0058] In a fourth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the camouflaged target detection method based on edge shrinkage and exposure as described in any of the first aspect or the second aspect.

[0059] (III) Beneficial effects

[0060] The model trained in the method of the embodiment of the present invention is a multi-level refinement framework that can locate the precise edge of the camouflaged target. In the training stage, the method of "shrinking" and "exposing" edge details is integrated in each stage of decoding, and the model is well trained. In addition, using the trained target detection model to process the camouflaged target image to be tested can effectively solve the problem that the edges of the background and the detected target cannot be accurately distinguished in the camouflaged target detection.

[0061] The method of the embodiment of the present invention achieves the purpose of edge exposure and improves the processing of edge features. The exposure process of the decoding unit further improves the model's ability to extract edge parts, achieving a fine edge effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A diagram showing the main architecture of the target detection model provided by an embodiment of the present invention;

[0063] Figure 2 A flowchart of using a target detection model provided by an embodiment of the present invention;

[0064] Figure 3 Schematic diagram of the conditional random field processing process in an embodiment of the present invention;

[0065] Figure 4 A schematic diagram of part of the process flow of the decoding unit in the embodiment of the present invention;

[0066] Figure 5 A schematic diagram of the target detection model training process provided by an embodiment of the present invention;

[0067] Figures 6 to 8 They are respectively diagrams of experimental results verified by the method of the embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0069] Embodiment 1

[0070] like Figure 1 The main model architecture diagram shown in the figure, the embodiment of the present invention provides a camouflage target detection method based on edge tightening and exposure, and its specific implementation method includes the following steps:

[0071] In the training phase, a true value map of each training image is obtained; each training image is input into a pre-built target detection model to obtain an output training true value map;

[0072] Obtain a first loss based on the true value map and the training true value map, and obtain a second loss based on part of the feature map and the true value map during the target detection model processing;

[0073] Adjusting the weight parameters of the target detection model according to the first loss and the second loss to obtain a target detection model for detecting the disguised target image to be detected;

[0074] The target detection model includes: an encoding component and a decoding component; the encoding unit of the encoding component and the decoding unit of the decoding component correspond to each other in sequence, and the encoding component outputs n encoding features based on a training image;

[0075] The decoding component includes: n decoding units, the current decoding unit receives the decoding features output by the previous decoding unit, the preprocessed encoding features, and the feature map corresponding to the previous decoding unit, processes them using edge tightening and exposure attention mechanisms, and outputs the decoding features of the current decoding unit; the encoding features are the features output by the encoding unit corresponding to the current decoding unit, and the feature map is the feature map obtained by processing the decoding features output by the previous decoding unit using a conditional random field;

[0076] The first decoding unit is used to directly use the encoding feature output by the corresponding encoding unit as the decoding feature;

[0077] The last decoding unit is used to receive the decoding features output by the previous decoding unit and the unpreprocessed encoding features and the feature map corresponding to the previous decoding unit for processing, and output the training truth map;

[0078] In the process of each decoding unit obtaining a feature map, the loss of the intermediate feature map and the true value map is used as the second loss, and the target edge in the intermediate feature map of the decoding unit is gradually refined.

[0079] In this embodiment, the coding component may include: n-1 coding units connected in series and FPN as the nth coding unit; the coding unit of the coding component is used to encode the texture features of the surface layer of the training image to the semantic features of the deep layer layer layer by layer to obtain multi-level coding features; n is a natural number greater than or equal to 1;

[0080] The first encoding unit receives the training image and outputs a first encoding feature;

[0081] The i-th encoding unit receives the i-1th encoding feature output by the i-1th encoding unit, and outputs the i-th encoding feature, where i is a natural number from 2 to n-1;

[0082] The FPN receives the encoding features output by the 2nd to n-1th encoding units to extract semantic information, and outputs the nth encoding feature.

[0083] During the training phase, the ground truth map of each training image can be obtained before encoding the components, including:

[0084] Binarize the training image to obtain the true value image of the segmentation map of the camouflaged target;

[0085] The truth map is subjected to maximum pooling processing and minimum pooling processing to obtain multiple edge probability maps, each edge probability map corresponds to a decoding unit, and the target edge in the edge probability map is gradually refined toward the output end, and the last decoding unit has no corresponding edge probability map; each edge probability map is used as a reference in the branch prediction process of the corresponding decoding unit to obtain intermediate loss information, and all the intermediate loss information constitutes the second loss.

[0086] Alternatively, when n=5, the maximum pooling layers of 20*20, 14*14, 12*12, 8*8, and 4*4 are used to downsample the truth image, and then the minimum pooling layers of 8*8, 7*7, 6*6, 5*5, and 4*4 are used to downsample the truth image, and then the truth value processed by the minimum pooling layer is XORed with 1 to achieve black-and-white flipping. Finally, the images that pass through the maximum pooling layer and the minimum pooling layer are superimposed and then upsampled to 352×352 to obtain the edge probability map of the camouflaged target from coarse to fine. p is the identifier of the edge probability map, such as Figure 1 and Figure 5 shown.

[0087] In the encoding component of this embodiment, each of the n-1 encoding units connected in series is a pyramid visual Transformer PVTv2.

[0088] Reference Figure 1 The decoding component shown, the first decoding unit is used to use the features output by the corresponding encoding unit as decoding features, including:

[0089] When the nth encoding unit corresponds to the first decoding unit, the encoding feature f output by the nth encoding unit is n ; The decoding feature of the first decoding unit is O1;

[0090] The feature map obtained by processing the decoding feature O1 output by the first decoding unit using the conditional random field include:

[0091]

[0092] Among them, p1 = gConv(s(O1)), gConv represents group convolution, concat represents concatenation in the feature dimension, S1 is O1 obtained by group convolution; edge is the identifier of the feature map;

[0093] When the i-th encoding unit corresponds to the k-th decoding unit, the feature map obtained by processing the decoding features output by the k-th decoding unit using the conditional random field include:

[0094]

[0095] O k is the decoding feature output by the kth decoding unit, S k O k Obtained through 1*1 convolutional layer s;

[0096] p k =gConv(t(S k-1 ,O k )concat s(O k ));

[0097] S k-1 is the output feature O of the k-1th decoding unit k-1 It is obtained by processing with a 1*1 convolution layer s; t represents a 1*1 convolution layer, t(S k-1 ,O k ) indicates S k-1 and O k After concatenation on the feature dimension, convolution processing is performed through a 1*1 convolution layer; i represents from large to small, and k represents from small to large, both of which are natural numbers from 2 to n-1.

[0098] Specifically, the current k+1th decoding unit receives the decoding feature O output by the previous decoding unit k , Encoded features after feature preprocessing Feature map corresponding to the previous decoding unit The edge contraction and exposure attention mechanism is used to process and output the decoding feature O of the current decoding unit. k+1 ,include:

[0099]

[0100]

[0101] The subscript s indicates the identifier of the encoding feature jump connection output by the encoding unit, gConv indicates group convolution, concat indicates concatenation in the feature dimension, and E-SA and E-CA indicate intermediate parameters;

[0102]

[0103]

[0104] Q, K, V are respectively 0 k A long vector is obtained by the fully connected layer in the k+1th decoding unit, and the dimensions of Q, K, V and E are the same; d mis the dimension of the long vector, m has no meaning, and E is A long vector obtained by the fully connected layer in the k+1th decoding unit; Softmax is a known activation function, and T is the transpose; To predict the true value edge feature map;

[0105] Yes Convolution processing and upsampling to 352*352, as well as the predicted true value edge image obtained by reducing the feature dimension.

[0106] During the above training process, the total loss is:

[0107] L total For: L total =L pre +L est +λL esa ;

[0108]

[0109]

[0110] L pre =∑ k PPA(O k ,gt);

[0111] Among them, L est The output features of the first decoding unit are processed at each stage of the conditional random field in the branch prediction process to obtain the predicted edge probability map p k and p k-1 The loss between est Also includes: predicted marginal probability map p k And the marginal probability map obtained by preprocessing the true value map The loss between, PPA and BCE are functions for finding the loss;

[0112] That is, L est The first term is the predicted edge probability map p obtained at each stage of the conditional random field in the branch prediction of the decoding unit k and p k-1 The loss between them ensures that the edge probability map predicted in the next stage is compressed based on the previous stage. est The second term is the predicted marginal probability map p k And the edge probability map obtained by preprocessing PPA loss is made between k More accurate prediction of marginal probability maps.

[0113] λ is the control L esaThe weight parameter accounting for the total loss, w, h are the predicted edge probability map p k The width and height, p k and p k-1 is the edge probability map in the branch prediction in the k-th decoding unit and the predicted edge probability map in the branch prediction in the k-1-th decoding unit, p k and p k-1 The size is consistent, and it is a 352*352 predicted edge probability map with a feature dimension of 1. is the edge probability map corresponding to the kth decoding unit obtained by preprocessing the truth map;

[0114] L esa The first term is p k and fine edge y k As loss, it ensures that the predicted edge probability map is tightened to the correct fine edge; the second item is to predict the fine edge and fine edge y k As loss, ensure the accuracy of the predicted edge truth value; fine edge y k is the true value of the manually processed fine edge of the corresponding training image; gt refers to the true value map of the segmentation map of the camouflaged target.

[0115] The above-mentioned PPA and BCE are both existing known methods for calculating losses.

[0116] After the model is trained, it can be used in the usage phase, such as Figure 2 As shown, the target image to be detected can be processed based on the above-trained target detection model to obtain fine edge information, thereby effectively solving the problem that the edges of the background and the detected target cannot be accurately distinguished in camouflaged target detection.

[0117] Embodiment 2

[0118] The method of the present invention may include the following steps in the training phase: In this embodiment, 5 encoding units and 5 decoding units are used as an example for explanation. Figure 1 In practical applications, the number of encoding units and decoding units may not be limited to 5, and they may be adjusted and configured according to actual needs. Figures 1 to 5 As shown, the method of this embodiment may include the following steps.

[0119] Step 1: Get the training dataset and the test dataset.

[0120] For each training image in the training data set, the segmentation image of the training image, that is, the true value map gt (the segmentation map true value map of the camouflaged target) is obtained, and the edge probability map corresponding to the true value map is obtained.

[0121] The target edge in the edge probability map is gradually refined toward the output end. For example, the widest fine edge can be a camouflaged target edge portion segmented image with a width of 3 pixels.

[0122] In this embodiment, for each training image, a fine edge, that is, a fine edge of a segmented target, is obtained manually.

[0123] That is to say, in order to ensure that the training results of the target detection model can correctly camouflage the target segmentation, this embodiment selects 4040 camouflage training images to be segmented, true value maps, and edge probability maps from the COD-10K and CAMO datasets as the training set, and then selects 250, 206, and 4121 images from CAMO COD-10K and NC4K respectively as the test set.

[0124] In this embodiment, the true value map of the training image can be preprocessed to obtain the edge probability map. For example, the true value map is processed using maximum pooling and minimum pooling. For example, refer to Figure 1 In the initial stage of training, the five decoding units shown in the figure use the maximum pooling layers of 20*20, 14*14, 12*12, 8*8, and 4*4 to downsample the truth map, and then use the minimum pooling layers of 8*8, 7*7, 6*6, 5*5, and 4*4 to downsample the truth map. The truth value processed by the minimum pooling layer is then XORed with 1 to achieve black-and-white flipping. Finally, the images that pass through the maximum pooling layer and the minimum pooling layer are superimposed and then upsampled to 352×352 to obtain the edge probability map from coarse to fine. correspond

[0125] Edge probability map: that is, the range of fine edges of the camouflaged target to be segmented. The finer the edge probability map (the smaller the pooling layer used), the closer the coarseness is to the fine edge. This reflects the exposure and tightening of this embodiment.

[0126] Step 2: Input each training image into the encoding component to extract n=5 encoding features.

[0127] The encoding component consists of four PVTv2 Blocks (Pyramid Vision Transformer, i.e. encoding unit) and FPN (Feature Pyramid Network).

[0128] After the 352×352 training image is input into four serially connected PVTv2 Blocks, the feature maps of each encoding unit are 88×88 f1, 88×88 f2, 44×44 f3, 22×22 f4, and the number of channels is 64, 128, 320, and 512. Then, the 88×88 f2, 44×44 f3, and 22×22 f4 are used as the input of FPN, and the output of FPN is 44×44 f5, and the number of channels is 512.

[0129] The encoding component of this embodiment is pre-trained by the ImageNet data set to obtain prior information and improve the neural network's ability to extract features. The four encoding units encode the texture features of the image surface to the deep semantic features layer by layer so that the neural network can obtain multi-level information, and through FPN, strong semantic information is extracted from deeper features, and more robust information is obtained, thereby improving the ability to detect camouflaged targets. After FPN processing, the fifth encoding feature is generated.

[0130] These five sets of features (f1, f2, f3, f4, f5) are reduced to their feature layers through a 1×1 convolutional neural network as the five output encoding features.

[0131] Reducing the channel dimension in this step is to improve the feasibility of model training.

[0132] Step 3: The five encoded features output by the encoding unit are input into the decoding component for edge tightening and exposure processing.

[0133] Sub-step 30: In this embodiment, the architecture of the decoding component is similar to U-Net, and f1, f2, f3, and f4 in the encoding component are input to the corresponding decoding units in the decoding component in the form of strip connections.

[0134] Before inputting the decoding unit, a 1×1 convolutional layer is used to process the dimensions of f1, f2, f3, and f4 into 128 dimensions (to reduce the number of model parameters). f1 can be directly used as (Right now ) is input to the fifth decoding unit;

[0135] f2, f3, f4 and f5 are concatenated in the feature dimension. Jump connection to the decoding unit corresponding to each encoding unit.

[0136]

[0137] Right now: Input to the fourth decoding unit,

[0138] Input to the third decoding unit,

[0139] input to the second decoding unit;

[0140] f5 as Directly input to the first decoding unit.

[0141] Sub-step 31: Convert the FPN output Directly input into the first decoding unit for branch prediction, the output of each decoding unit in the decoding component needs to be branch predicted. The first decoding unit directly uses f5 as the state feature O1, that is, the first decoding feature.

[0142] The branch prediction of O1 output by the first decoding unit is divided into three stages.

[0143] Branch prediction for the first decoding unit: Stage 1: Directly use the truth map and O1 as loss as L pre (This is part of the branch prediction, especially the output of the last stage of the decoder also needs to use L pre Since the true value image is 352×352, bilinear interpolation is used to adjust O1 from 44×44 to 352×352.

[0144] The second stage: O1 is passed through a 1*1 convolutional layer s to obtain the observed feature S1, and then upsampled to the original image size of 352×352 to obtain p1 after being processed through a group convolutional layer; the convolutional layers in this embodiment are all lowercase s.

[0145] In this process, p1 and The loss (as part of the second loss) is used to constrain the predicted edge probability map.

[0146] It should be noted here that Figure 3 As shown in the figure, in the conditional random field of branch prediction, t and s are both 1*1 convolutional layers with shared parameters. The decoding unit outputs features as state features, which are processed by the convolutional layer s to generate observation features S1. Each observation feature S1 is implemented by t, so that the output features of each stage are refined to achieve better segmentation effect.

[0147] The third stage: use p1 and O1 to perform dot product, then concatenate with S1 and O1 in the dimension direction, and reduce the dimension to 1 to obtain the feature map

[0148] At this time, the size is 352×352 A loss is made with the fine edge (the fine edge map of the current training image obtained manually in the initial stage of training) (as part of the second loss) to ensure the accuracy of the predicted edge true value.

[0149]

[0150] p1=gConv(s(O1)).

[0151] In this embodiment, the true value map is used to directly supervise the features output from the two stages. Multi-stage supervision can achieve better segmentation capabilities. Supervision based on images from different stages can improve the model's segmentation capabilities for the main body of the disguised target and the edges of the disguised target.

[0152] Sub-step 32: Perform feature preprocessing on the output f4 of the fourth encoding unit and the feature map O1 is input to the second decoding unit, which processes it using edge contraction and exposure attention mechanism and outputs O2.

[0153] The feature map O2 is obtained by concatenating the attention matrix dimensions of the outputs of E-CA and E-SA and then concatenating them with the corresponding encoding unit input. Perform dot multiplication and then add the output.

[0154] In this embodiment, the intermediate parameters E-CA and E-SA can be obtained based on the following formulas: Figure 4 As shown;

[0155]

[0156]

[0157] Among them, Q, K, and V are respectively k A long vector obtained by the fully connected layer in the k+1th decoding unit, E is A long vector obtained by the fully connected layer in the k+1th decoding unit; the dimensions of Q, K, V and E are the same; d m is the dimension of the long vector, m has no meaning; Is After the convolution layer is processed and upsampled to a size of 352*352, the feature dimension is reduced to 1 dimension to obtain the predicted true value edge image.

[0158] In this embodiment The purpose of dot multiplication with K as a mask is to expose the edge area.

[0159] It is understandable that the backbone of the second to fifth decoding units is implemented by combining the edge exposure attention E-CA and E-SA, and the sliding window mechanism in the swin-Transformer. k+1 See the following formula:

[0160]

[0161] in,

[0162] That is, the current k+1th decoding unit receives the decoding feature O output by the previous decoding unit k , Encoded features after feature preprocessing Feature map corresponding to the previous decoding unit The edge contraction and exposure attention mechanism is used to process and output the decoding feature O of the current decoding unit. k+1 .

[0163] That is to say, the input of each decoding unit in this embodiment is processed by edge exposure attention (edge ​​exposure attention, self-attention and cross-attention are composed of group convolution) The output features are dot-producted with the features of the relative position encoding unit and then added. Through edge exposure attention, the features of branch prediction and the features of the previous stage are fully extracted and mixed through the self-attention mechanism. Figure 3 In the equation, f is the total input, f = f s gConv[E-SA,E-CA]+f s , which represents the characteristics of each stage of the input encoding component,

[0164] Sub-step 33: The branch prediction of O2 output by the second decoding unit is divided into three stages.

[0165] Branch prediction for the second decoding unit: Stage 1: Directly use the truth map and O2 as loss L pre Since the true image is 352×352, bilinear interpolation is used to adjust the dimension of O2 from 44×44 to 352×352.

[0166] The second stage: O2 is passed through a 1*1 convolutional layer s to obtain the observed feature S2, and then upsampled to the original image size of 352×352 to obtain p2 after processing through a group convolutional layer;

[0167] Compare p2 with Assume loss constraint p2 to approach

[0168] The third stage: use p2 and O2 to perform dot product, then concatenate with S2 and O2 in the dimension direction, and reduce the dimension to 1 to obtain the feature map (At this time k=2); see the formula as follows.

[0169]

[0170] at this time, (The output is then processed through a convolutional layer and up-sampled to a size of 352×352. Loss with fine edges.

[0171] in,

[0172] Prediction marginal probability map p k =gConv(t(S k-1 ,O k )concat s(O k ))

[0173] S k-1 is the output feature O of the k-1th decoding unit k-1 It is obtained by processing with 1*1 convolution layer s; i represents from large to small, k represents from small to large, both are natural numbers from 2 to n-1; t represents 1*1 convolution layer, t(S k-1 ,O k ) indicates S k-1 and O k After splicing in the feature dimension, convolution processing is performed through a 1*1 convolution layer.

[0174] It should be noted that the processing of each stage in the decoding component is basically the same, that is, the processing process of each decoding unit from the n-1th decoding unit to the 2nd decoding unit in the decoding component is basically the same, so this embodiment will not describe each process in detail.

[0175] In order to achieve the effect of edge compression in the above embodiment, the multi-stage processing of the decoding component can use a continuously compressed edge probability map, such as Figure 3 As shown, each decoding unit in the decoding component continuously improves the prediction accuracy of edge features.

[0176] The method of this embodiment achieves the purpose of edge exposure and improves the processing of edge features. The exposure process of the decoding unit further improves the model's ability to extract edge parts, achieving a fine edge effect.

[0177] In this embodiment, group convolution is used in each stage to fully integrate the fine features extracted from the edge with the target segmentation features of the previous stage, thereby deepening the recognition and segmentation capabilities of the feature map of the current stage for edge features. After multiple rounds of training, the model acquires the ability to detect and segment camouflaged targets.

[0178] Step 4: Loss Description

[0179] In order to ensure the correct execution of "shrinkage" and "exposure", each decoding unit of the decoding component in the model of this embodiment has three kinds of losses, for example, it may include edge state transition loss L est , edge state abnormal loss L esa , predicted loss L pre The purpose of the first two loss schemes is to penalize areas beyond the predicted edge, thereby achieving the effect of edge tightening.

[0180] In this embodiment, the weight parameters of the training model are adjusted based on the loss of each training image, and then the model is made to meet the end conditions through the training image to obtain the trained image.

[0181] The total loss L for each training image total It can be expressed as:

[0182] L total =L pre +L est +λL esa #

[0183] λ is a hyperparameter that controls the weight of Lesa in the total loss and the weight of edge abnormality loss.

[0184] L est It can be expressed as:

[0185]

[0186] L est The marginal prediction probability map P of the previous stage k-1 and the marginal prediction probability map p in the next stage k Difference is made and compared with 0 to get the minimum value. This can ensure that the range of the edge probability map predicted in the next stage falls within the edge probability map predicted in the previous stage. If the range of the edge prediction map in the next stage exceeds the edge prediction map in the previous stage, make p k-1 With p k The difference is negative, so L est The PPA is the loss between the prediction of the output edge probability map and the edge probability map. The role of PPA is to make the loss penalty heavier for the more difficult to distinguish part. w and h are the predicted edge probability map p k Width and height, p k and p k-1 is the predicted marginal probability map of stage k and stage k-1, which has the same size. is the marginal probability map of the ground truth map corresponding to the training images.

[0187] L esa It can be expressed as:

[0188]

[0189] L esa The basic principle of loss and L est Basically similar, but the object is changed to the predicted edge probability map and the edge probability map preprocessed from the true value to achieve the same penalty effect. BCE is the cross entropy loss, which uses it to lose the predicted fine edge and the fine edge true value. y k is the fine edge truth value for each training image during the training phase.

[0190] L pre =∑ k PPA(O k ,gt), gt refers to the segmentation map truth map of the disguised target. In this embodiment, PPA and BCE are existing known functions / methods for calculating loss.

[0191] Experimental example:

[0192] Compared with the neural network for camouflaged target recognition in the prior art, the method of this embodiment can achieve more accurate prediction of the edges of camouflaged targets, thereby achieving accurate detection and segmentation of camouflaged targets. On three widely used datasets CAMO, NC4K and COD10K, the camouflage dataset CAMO provides 1,250 different image sets. NC4K is a large-scale COD test dataset with a subset of 4,121 images. COD10K represents the largest benchmark to date, including 10,000 images. For training purposes, SAE is trained using 1,000 images from the camouflage dataset and 3,040 images from COD10K (i.e., COD10K-Train and CAMO-Train). The evaluation process involves a test set containing 2,026 images from COD10K, 4,121 images from NC4K, and 250 images of camouflage from NC4K.

[0193] The model detection method of this embodiment has achieved excellent results in the above-mentioned data sets. Figure 6 Comparison between the method of this embodiment and other neural networks for COD camouflage target recognition shows that the model processing method of this embodiment is ahead of other networks in terms of edge refinement, and thus achieves a very obvious effect in the accuracy of camouflage target detection.

[0194] In particular, the model decoding component can significantly improve edge refinement in multi-stage exposure: Figure 7 Shows visualization of multiple steps of exposure and contraction in the network. Figure 8 We show the visualization effects of multi-step exposure and tightening of images in a real dataset.

[0195] Embodiment 3

[0196] In addition, an embodiment of the present invention further provides an electronic device, characterized in that it includes a memory and a processor, the memory stores a computer program, the processor executes the computer program stored in the memory, and performs the steps of the method described in any of the above embodiments.

[0197] On the other hand, an embodiment of the present invention further provides a computer program product, comprising the steps of the methods described in the above embodiments.

[0198] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0199] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0200] In the present invention, unless otherwise clearly specified and limited, when a first feature is “on” or “below” a second feature, it may be that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above” or “above” a second feature, it may be that the first feature is directly above or obliquely above the second feature, or it may simply mean that the first feature is higher in level than the second feature. When a first feature is “below”, “below” or “below” a second feature, it may be that the first feature is directly below or obliquely below the second feature, or it may simply mean that the first feature is lower in level than the second feature.

[0201] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0202] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A camouflage target detection method based on edge shrinkage and exposure, characterized in that: include: In the training phase, the true value map of each training image is obtained; Input each training image into a pre-built object detection model to obtain the output training truth map; Obtaining a first loss based on the truth map and the training truth map, adjusting a weight parameter of the target detection model according to the first loss and a second loss of some features and the truth map during the target detection model processing, to obtain a target detection model for detecting a target image; The target detection model includes: an encoding component and a decoding component; the encoding unit of the encoding component and the decoding unit of the decoding component correspond to each other in sequence, and the encoding component outputs n encoding features based on a training image; The decoding component includes: n decoding units, the current decoding unit receives the decoding features output by the previous decoding unit, the preprocessed encoding features, and the feature map corresponding to the previous decoding unit, processes them using edge tightening and exposure attention mechanisms, and outputs the decoding features of the current decoding unit; the encoding features are the features output by the encoding unit corresponding to the current decoding unit, and the feature map is the feature map obtained by processing the decoding features output by the previous decoding unit using a conditional random field; The first decoding unit is used to use the features output by the corresponding encoding unit as decoding features; The last decoding unit is used to receive the decoding features output by the previous decoding unit and the unpreprocessed encoding features and the feature map corresponding to the previous decoding unit for processing, and output the training truth map; In the process of obtaining feature maps in each decoding unit, the loss of the intermediate features and the true value map is used as the second loss, and the target edge in the intermediate features of the decoding unit is gradually refined; Get the true value map of each training image, including: Binarize the training image to obtain the true value image of the segmentation map of the camouflaged target; Performing maximum pooling processing and minimum pooling processing on the truth map to obtain multiple edge probability maps, each edge probability map corresponds to a decoding unit, and the target edge in the edge probability map is gradually refined toward the output end, and the last decoding unit has no corresponding edge probability map; each edge probability map is used as a reference in the branch prediction process of the corresponding decoding unit to obtain intermediate loss information, and all the intermediate loss information constitutes the second loss; The first decoding unit is used to use the feature output by the corresponding encoding unit as a decoding feature, including: when the nth encoding unit corresponds to the first decoding unit, the encoding feature f output by the nth encoding unit n ; The decoding characteristic of the first decoding unit is O1; n is a natural number greater than or equal to 1; The feature map obtained by processing the decoding feature O1 output by the first decoding unit using the conditional random field include: Among them, p1 = gConv(s(O1)), gConv represents group convolution, concat represents concatenation in the feature dimension, S1 is O1 obtained by group convolution; edge is the identifier of the feature map, s(O1) represents O1 processed by 1*1 convolution layer s; When the i-th encoding unit corresponds to the k-th decoding unit, the feature map obtained by processing the decoding features output by the k-th decoding unit using the conditional random field include: O k is the decoding feature output by the kth decoding unit, S k O k It is obtained by processing with a 1*1 convolutional layer s; i represents from large to small, and k represents from small to large, both of which are natural numbers from 2 to n-1; p k =gConv(t(S k-1 ,O k )concat s(O k )); S k-1 is the output feature O of the k-1th decoding unit k-1 It is obtained by processing with 1*1 convolution layer s; i represents from large to small, k represents from small to large, both are natural numbers from 2 to n-1; t represents 1*1 convolution layer, t(S k-1 ,O k ) indicates S k-1 and O k After concatenation in the feature dimension, convolution is performed through a 1*1 convolution layer. k ) is processed by 1*1 convolution layer s k ; The current k+1th decoding unit receives the decoding feature O output by the previous decoding unit k , Encoded features after feature preprocessing Feature map corresponding to the previous decoding unit The edge contraction and exposure attention mechanism is used to process and output the decoding feature O of the current decoding unit. k+1 ,include: The subscript s indicates the identifier of the encoding feature jump connection output by the encoding unit, gConv indicates group convolution, concat indicates concatenation in the feature dimension, and E-SA and E-CA indicate intermediate parameters; Q, K, V are respectively 0 k A long vector obtained by the fully connected layer in the k+1th decoding unit, E is A long vector obtained by the fully connected layer in the k+1th decoding unit; Softmax is a known activation function, T is the transpose; the dimensions of Q, K, V, and E are the same; d m is the dimension of the long vector, m has no meaning; Yes Convolution is performed and the size is upsampled to a preset value, and the predicted true value edge image is obtained by reducing the feature dimension.

2. The method according to claim 1, characterized in that The coding component includes: n-1 coding units connected in series and FPN as the nth coding unit; the coding units of the coding component are used to encode the texture features of the surface layer of the training image to the semantic features of the deep layer layer layer by layer to obtain multi-level coding features; The first encoding unit receives the training image and outputs a first encoding feature; The i-th encoding unit receives the i-1th encoding feature output by the i-1th encoding unit, and outputs the i-th encoding feature; The FPN receives the encoding features output by the 2nd to n-1th encoding units to extract semantic information, and outputs the nth encoding feature.

3. The method according to any one of claims 1 to 2, characterized in that: Binarize the training image to obtain the true value image of the segmentation map of the camouflaged target; When n=5, the maximum pooling layers of 20*20, 14*14, 12*12, 8*8, and 4*4 are used to downsample the truth image, and then the minimum pooling layers of 8*8, 7*7, 6*6, 5*5, and 4*4 are used to downsample the truth image. Then, the truth value processed by the minimum pooling layer is XORed with 1 to achieve black and white flipping. Finally, the images that pass through the maximum pooling layer and the minimum pooling layer are superimposed and then upsampled to 352×352 to obtain the edge probability map of the camouflaged target from coarse to fine. p is the label of the marginal probability map; And / or, each of the n-1 coding units connected in series is a pyramid vision Transformer PVTv2.

4. The method according to any one of claims 1 to 2, characterized in that: Total loss L total for: THE total =L pre +L est +λL esa ; IT pre =∑ k PPA(O k ,gt); Among them, L est The output features of the first decoding unit are processed at each stage of the conditional random field in the branch prediction process to obtain the predicted edge probability map p k and p k-1 The loss between est Also includes: predicted marginal probability map p k And the marginal probability map obtained by preprocessing the true value map The loss between, PPA and BCE are functions for finding the loss; λ is the control L esa The weight parameter accounting for the total loss, w, h are the predicted edge probability map p k The width and height, p k and p k-1 is the edge probability map in the branch prediction in the k-th decoding unit and the predicted edge probability map in the branch prediction in the k-1-th decoding unit, p k and p k-1 Consistent size, is the edge probability map corresponding to the kth decoding unit obtained by preprocessing the truth map; L esa The first term is p k and fine edge y k The second term is to predict fine edges. and fine edge y k As loss, fine edge y k is the true value of the manually processed fine edge of the corresponding training image; gt refers to the true value map of the segmentation map of the camouflaged target.

5. A camouflage target detection method based on edge shrinkage and exposure, characterized in that: include: Input the camouflaged target image to be detected into the trained target detection model to obtain the output truth map; The target detection model is obtained by training using the method described in any one of claims 1 to 4 above.

6. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program stored in the memory and performs the steps of the camouflaged target detection method based on edge shrinkage and exposure as described in any one of claims 1 to 4.

7. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the camouflaged target detection method based on edge shrinkage and exposure as claimed in any one of claims 1 to 4.

Images