A hidden target detection method based on transformer and multi-feature aggregation

Abstract

This invention discloses a concealed target detection method based on transformer and multi-feature aggregation. First, the acquired detection image is input into a pre-trained transformer to obtain backbone features at different scales. Then, the highest-level features are input into a receptive field-based feature enhancement module to obtain enhanced features, while the lowest-level features are input into a coordinate-based attention module to capture long-distance dependencies and obtain coordinate features. Finally, the backbone features, enhanced features, and coordinate features are input into two back-guided multi-feature fusion modules. Through back-mining, group fusion, and location information supplementation, the detection results are obtained, and the results are further refined in a cascaded manner to obtain the final concealed target detection result. This invention is applied to general concealed target image detection tasks, solving the problem of blurred detection boundaries and effectively improving the accuracy and efficiency of concealed target detection tasks, thus possessing high engineering application value.

Description

Technical Field

[0001] This invention relates to a method for detecting hidden targets based on transformer and multi-feature aggregation, belonging to the field of computer vision technology. Background Technology

[0002] Covert target detection technology is a key focus in the field of target detection and identification, such as detecting power facilities in dimly lit environments in the power industry, and detecting obstacles underwater in turbid waters in the water conservancy industry. Because covert targets are highly similar to their background, they often exhibit blurred boundaries, making covert target detection technology even more challenging.

[0003] Accuracy in concealed target boundary detection is a key factor in evaluating the quality of concealed target detection technology. With the application of deep convolutional neural networks, many deep learning-based concealed object detection methods have emerged. These algorithms have shown good results in relatively simple scenarios, but most are only applicable to a single scenario and have low applicability in other scenarios. When the concealed target is small and the background is complex, the detection accuracy drops significantly. The reasons can be divided into two points: First, the feature enhancement modules in most models do not consider the contribution of feature information at different levels to the network and the limitations of skip-feature fusion methods, resulting in insufficient feature representation capabilities. Second, the feature fusion module contains many redundant operations, retaining a large amount of interference information, leading to blurred target boundaries and affecting detection accuracy. Summary of the Invention

[0004] This invention provides a method for detecting hidden targets based on transformer and multi-feature aggregation, which solves the problem of insufficient accuracy of existing detection results.

[0005] To achieve the above objectives, the technical solution adopted by this invention is as follows:

[0006] A method for detecting hidden targets based on transformer and multi-feature aggregation includes the following steps:

[0007] Step 1) Acquire the detection image and input it into the pyramid transformer with the last classification layer removed. Through the pre-trained transformer-based feature extraction module, obtain the backbone features at four different scales. {F i ,i= 1,2,3,4} , i It is the level of the feature;

[0008] Step 2) Apply the highest-level backbone features obtained in Step 1) F 4The input is fed into the receptive field-based feature enhancement module. Dilated convolution effectively expands the receptive field, thus... F 4 Parallel inputs are fed into different branches, and dilated convolutions with different dilation rates are used to expand the receptive field. The outputs of each branch are summed, and after activation, channel-wise dimensionality reduction is performed to obtain single-channel enhanced features. F glo ;

[0009] Step 3) Interpolate the enhanced features output from Step 2) F glo The size is adjusted to match the original image size to obtain the enhanced feature map. P glo ;

[0010] Step 4) Apply the lowest-level backbone features obtained in Step 1) F 1 The input is fed into a coordinate-based attention module, which uses two one-dimensional pooling kernels to encode long-range dependencies in both the horizontal and vertical directions, and then... F 1 Multiplication enhances the representational power of features, resulting in coordinate features. F pos ;

[0011] Step 5) Using one 1×1 Convolution will output the coordinate features from step 4). F pos The number of channels is reduced to 1, and the image size is adjusted through interpolation to obtain the coordinate feature map. P pos ;

[0012] Step 6) Apply the backbone features obtained in Step 1) F 3 The enhanced features obtained in step 2) F glo The coordinate features obtained in step 4) F pos The input is fed into the first reverse-guided multi-feature fusion module, and then processed through reverse mining. F glo Obtaining reverse guidance information, and using group fusion to... F 3 Integrating with reverse guidance information, F pos As a supplement to location information, it enables the effective fusion of multiple features to obtain primary target features. F 3 cam ;

[0013] Step 7) Interpolate the primary target features obtained in Step 6)F 3 cam The size is adjusted to be the same as the original image to obtain the primary hidden target detection map. P 3 cam ;

[0014] Step 8) Use the primary target features obtained in Step 6) F 3 cam Enhanced features obtained in step 2) F glo Adding them together yields the enhanced features of the fusion. F’ glo ;

[0015] Step 9) Apply the backbone features obtained in Step 1) F 2 The enhanced features of fusion obtained in step 8) F’ glo The coordinate features obtained in step 4) F pos The input is fed into a second multi-feature fusion module based on reverse guidance, and then processed through reverse mining. F’ glo The reverse guidance information is obtained, and it is then fused with the reverse guidance information using group fusion. F pos As a supplement to location information, it enables the effective fusion of multiple features to obtain the final target features. F 2 cam ;

[0016] Step 10) Interpolate the final target features obtained in Step 9) F 2 cam The size was adjusted to match the original image to obtain the final hidden target detection map. P 2 cam .

[0017] Furthermore, in step 2), the highest-level backbone features obtained in step 1) are... F 4 The input has 5 parallel branches, specifically including:

[0018] Step 21) Shortcut branch utilizes a 1×1 Convolution reduces the number of channels to 64, thus reducing computational cost;

[0019] Step 22) The first cascading branch utilizes a 1×1 Convolution reduces the number of channels and a pair 1×3 , 3×1Asymmetric convolution speeds up inference;

[0020] Step 23) The second cascade branch first passes through a 1×1 Convolution is used to reduce the channel dimension, and then the result is added to the output of step 22), followed by a pair of... 1×3 , 3×1 Asymmetric convolution increases inference speed, and finally passes through a convolution with an inflation rate of 3. 3× 3 Dilated convolution expands the receptive field;

[0021] Step 24) The third cascade branch first goes through a 1×1 Convolution is used to reduce the channel dimension, and then the result is added to the output of step 23), followed by a pair of... 1×5 , 5×1 Asymmetric convolution increases inference speed, and finally passes through a dilation coefficient of 5. 5× 5 Dilated convolution modules expand the receptive field;

[0022] Step 25) The fourth cascade branch first passes through a 1×1 Convolution is used to reduce the channel dimension, and then the result is added to the output of step 24), followed by a pair of... 1×7 , 7×1 Asymmetric convolution increases inference speed, and finally passes through a dilation coefficient of 7. 7× 7 The convolutional dilation module expands the receptive field;

[0023] Step 26) Connect the outputs of steps 22), 23), 24), and 25), and then pass through one... 3×3 The convolution adjusts the size of the feature map, then adds it to the output of step 21), and after ReLU activation, it uses one... 1×1 Enhanced features are obtained by dimensionality reduction of convolutional channels. F glo .

[0024] Furthermore, in step 4), the lowest-level backbone features obtained in step 1) are... F 1 Inputting a coordinate-based attention module involves the following steps:

[0025] Step 41) To avoid the problem of difficulty in preserving location information when compressing global spatial information to the channel descriptor, utilize... and Pair the two one-dimensional pooling kernels with the lowest-level backbone features obtained in step 1). F 1 Each channel is average pooled along both the horizontal and vertical coordinates to encode the corresponding orientation-aware feature map. zh and z w ;

[0026] Step 42) Connect the two outputs from step 41) via a connection operation. z h and z w By fusing, an intermediate feature map is obtained. :

[0027] f= RELU ( BN ( con ([ z h , z w ])))

[0028] in, This indicates a connection operation along a spatial dimension. con express 1×1 convolution, BN It is a normalization operation. RELU It is a ReLU activation function. r It is the residual rate that controls the block size, which is set to 8 based on experience;

[0029] Step 43) Decompose the output f of step 42) into two separate tensors along the spatial dimension. and

[0030] :

[0031] ;

[0032] in σ Represents the sigmoid function;

[0033] Step 44) Combine the two outputs from step 43) with the main features of the output from step 1). F 1 Multiply to obtain coordinate features F pos .

[0034] Furthermore, in step 6), the backbone features obtained in step 1) are... F 3 The enhanced features obtained in step 2) F glo The coordinate features obtained in step 4) F pos The input to the first reverse-guided multi-feature fusion module includes the following steps:

[0035] Step 61) Use interpolation to enhance the features output in step 2). F glo Size and trunk features obtained in step 1) F 3 Adjust to be consistent:

[0036] F ’ 3 = I ( F glo )

[0037] in Indicates the interpolation operation;

[0038] Step 62) Reverse mining of the output of step 61) F ’ 3 Receive reverse guidance information R 3 :

[0039] R 3 =!( σ ( F ’ 3 ))

[0040] in It's the reverse operation, using the sigmoid function to transform the eigenvalues ​​to the range [0,1], and then using the identity matrix... E Subtracting the input from the middle yields the reverse guidance information;

[0041] Step 63) Apply the backbone features obtained in Step 1) F 3 Divided into n Groups with the same number of candidate features F n 3 , n Based on experience, it is set to 32;

[0042] Step 64) Periodically output in step 63) F n 3 Insert the output of step 62) R 3 To obtain the characteristics of the connection F cat 3 :

[0043]

[0044] Step 65) Using a3×3 Convolution reduces the number of channels from C 3 +n Reduce to C 3 Then, compare it with the backbone features output in step 1). F 3 Adding them together yields the group connection features. for:

[0045]

[0046] in, Con (∙) indicates 3×3 convolution;

[0047] Step 66) To fully utilize coordinate-aware details, interpolation is used to transform the output of step 4). F pos Adjusted to match the main characteristics F 3 The same size, and then the group connection feature output in step 65). The connection, through a 3×3 Convolution adjusts the number of channels, and finally, a ReLU function and a... 3×3 Convolution transforms feature maps into primary target features. F 3 cam :

[0048]

[0049] in Cat (∙) indicates a serial operation.

[0050] Furthermore, in step 9), the backbone features obtained in step 1) are... F 2 The enhanced features of fusion obtained in step 8) F’ glo The coordinate features obtained in step 4) F pos Inputting the second multi-feature fusion module based on reverse guidance includes the following steps:

[0051] Step 91) Use interpolation to fuse the enhanced features output from step 8). F’ glo Size and trunk features obtained in step 1) F 2 Adjust to be consistent:

[0052] ;

[0053] in I(∙) indicates an interpolation operation;

[0054] Step 92) Reverse mining the output of step 91) F’ 2 Receive reverse guidance information R 2 :

[0055] R 2 =!( σ ( F’ 2 ))

[0056] Here, !(∙) represents the reverse operation, using the sigmoid function to transform the eigenvalues ​​to the range [0,1], and then using the identity matrix... E Subtracting the input from the middle yields the reverse guidance information;

[0057] Step 93) Apply the backbone features obtained in Step 1) F 2 Divided into n Groups with the same number of candidate features F n 2 , n Based on experience, it is set to 32;

[0058] Step 94) Periodically output in step 93) F n 2 Insert the output of step 92) R 2 To obtain the characteristics of the connection F cat 2 :

[0059]

[0060] Step 95) Using a 3×3 Convolution reduces the number of channels from C 2 +n Reduce to C 2 Then, compare it with the backbone features output in step 1). F 2 Adding them together yields the group connection features. for:

[0061] ;

[0062] in, Con (∙) indicates 3×3 convolution;

[0063] Step 96) To fully utilize coordinate-aware details, interpolation is used to transform the output of step 4). F pos Adjusted to match the main characteristics F 2 The same size, and then the group connection feature output in step 95). The connection, through a 3×3 Convolution adjusts the number of channels, and finally, a ReLU function and a... 3×3 Convolution transforms feature maps into the final target features. F 2 cam :

[0064]

[0065] in Cat (∙) indicates a serial operation.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] This invention utilizes a pyramid transformer to extract image features, making it suitable for pixel-level detection tasks and generating feature maps at more scales. The receptive field-based feature enhancement module expands the receptive field of high-level backbone features using dilated convolutions with different dilation rates, which helps improve the accuracy of feature semantic information. The reverse-guided multi-feature fusion module fuses the backbone features extracted by the transformer, the enhanced features after expanding the receptive field, and the features processed by the attention mechanism, which eliminates redundant operations and speeds up processing while ensuring accuracy. Attached Figure Description

[0068] Figure 1 This is a flowchart of the method of the present invention;

[0069] Figure 2 This is a structural diagram of the feature enhancement module based on the receptive field provided by the present invention;

[0070] Figure 3 A diagram of the coordinate-based attention module structure provided by this invention;

[0071] Figure 4 The following is a structural diagram of the multi-feature fusion module based on reverse guidance provided by the present invention;

[0072] Figure 5 The image to be detected in the embodiment;

[0073] Figure 6 This is a result diagram based on the receptive field feature enhancement module in the embodiment;

[0074] Figure 7This is a result diagram based on the coordinate attention module in the embodiment;

[0075] Figure 8 This is a preliminary prediction image from the multi-feature fusion module based on reverse guidance in the embodiment.

[0076] Figure 9 This is a secondary prediction map based on a multi-feature fusion module guided by reverse direction;

[0077] Figure 10 This is the ground truth image of the image to be detected in the embodiment;

[0078] Figure 11 This is a comparison chart of the algorithm performance in Example 1;

[0079] Figure 12 This is a comparison chart of the algorithm performance in Example 2;

[0080] Figure 13 This is a comparison chart of the algorithm performance in Example 3;

[0081] Figure 11-13 In the diagram, Figure a represents the image to be detected in this embodiment, Figure b represents the ground truth map of the hidden target, i.e., the ground truth map, Figure c represents the detection result obtained using the hidden target detection algorithm proposed in this paper, and Figures d, e, f, g, and h represent the results obtained using five models: ZoomNet, OCENet, BGNet, ERRNet, and MGL, respectively. Detailed Implementation

[0082] The present invention will be further described below in conjunction with the accompanying drawings and examples. The following examples are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Example

[0083] like Figure 1 As shown, the present invention proposes a hidden target detection method based on transformer and multi-feature aggregation, the specific process of which is as follows:

[0084] Step 1) Collect data as follows Figure 5 The detected image shown is input into a pyramid transformer with the last classification layer removed. Through a pre-trained transformer-based feature extraction module, four backbone features at different scales are obtained. {F i ,i=1,2,3,4} , i It is the level of the feature;

[0085] Step 2) As Figure 2 As shown, the highest-level backbone features obtained in step 1) are... F 4The input is fed into the receptive field-based feature enhancement module. Based on the principle that dilated convolution can effectively expand the receptive field, the input is... F 4 Parallel inputs are fed into different branches, and dilated convolutions with different dilation rates are used to expand the receptive field. The outputs of each branch are summed, and after activation, channel-wise dimensionality reduction is performed to obtain single-channel enhanced features. F glo The specific process is as follows:

[0086] Step 21) Shortcut branch utilizes a 1×1 Convolution reduces the number of channels to 64, thus reducing computational cost;

[0087] Step 22) The first cascading branch utilizes a 1×1 Convolution reduces the number of channels and a pair 1×3 , 3×1 Asymmetric convolution speeds up inference;

[0088] Step 23) The second cascade branch first passes through a 1×1 Convolution is used to reduce the channel dimension, and then the result is added to the output of step 22), followed by a pair of... 1×3 , 3×1 Asymmetric convolution increases inference speed, and finally passes through a convolution with an inflation rate of 3. 3× 3 Dilated convolution expands the receptive field;

[0089] Step 24) The third cascade branch first goes through a 1×1 Convolution is used to reduce the channel dimension, and then the result is added to the output of step 23), followed by a pair of... 1×5 , 5×1 Asymmetric convolution increases inference speed, and finally passes through a dilation coefficient of 5. 5× 5 Dilated convolution modules expand the receptive field;

[0090] Step 25) The fourth cascade branch first passes through a 1×1 Convolution is used to reduce the channel dimension, and then the result is added to the output of step 24), followed by a pair of... 1×7 , 7×1 Asymmetric convolution increases inference speed, and finally passes through a dilation coefficient of 7. 7× 7 The convolutional dilation module expands the receptive field;

[0091] Step 26) Connect the outputs of steps 22), 23), 24), and 25), and then pass through one... 3×3 The convolution adjusts the size of the feature map, then adds it to the output of step 21), and after ReLU activation, it uses one... 1×1 Enhanced features are obtained by dimensionality reduction of convolutional channels. Fglo ;

[0092] Step 3) Interpolate the enhanced features output from step 2) F glo The size is adjusted to match the original image size to obtain the enhanced feature map. P glo ,like Figure 6 As shown;

[0093] Step 4) As Figure 3 As shown, the lowest-level backbone features obtained in step 1) are... F 1 The input is fed into a coordinate-based attention module, which uses two one-dimensional pooling kernels to encode long-range dependencies in both the horizontal and vertical directions, and then... F 1 Multiplication enhances the representational power of features, resulting in coordinate features. F pos The specific process is as follows:

[0094] Step 41) To avoid the problem of difficulty in preserving location information when compressing global spatial information to the channel descriptor, utilize... and Two one-dimensional pooling kernels perform average pooling on each channel along the horizontal and vertical coordinates respectively, encoding the corresponding orientation-aware feature maps. z h and z w ;

[0095] Step 42) Connect the two outputs from step 41) via a connection operation. z h and z w By fusing, an intermediate feature map is obtained. :

[0096] f= RELU ( BN ( con ([ z h , z w ])))

[0097] Where [·,∙] denotes a connection operation along a spatial dimension. con express 1×1 convolution, BN It is a normalization operation. RELU It is a ReLU activation function. r It is the residual rate that controls the block size, which is set to 8 based on experience;

[0098] Step 43) Decompose the output f of step 42) into two separate tensors along the spatial dimension. and :

[0099] ;

[0100] in σ Represents the sigmoid function;

[0101] Step 44) Combine the two outputs from step 43) with the main features of the output from step 1). F 1 Multiply to obtain coordinate features F pos ;

[0102] Step 5) Using one 1×1 Convolution will output the coordinate features of 4) F pos The number of channels is reduced to 1, and the image size is adjusted through interpolation to obtain the coordinate feature map. P pos ,like Figure 7 As shown;

[0103] Step 6) As Figure 4 As shown, the backbone features obtained in step 1) F 3 The enhanced features obtained in step 2) F glo The coordinate features obtained in step 4) F pos The input is fed into the first reverse-guided multi-feature fusion module, and then processed through reverse mining. F glo Obtaining reverse guidance information, and using group fusion to... F 3 Integrating with reverse guidance information, F pos As a supplement to location information, it enables the effective fusion of multiple features to obtain primary target features. F 3 cam The specific process is as follows:

[0104] Step 61) Use interpolation to enhance the features output in step 2). F glo Size and trunk features obtained in step 1) F 3 Adjust to be consistent:

[0105]

[0106] in Indicates the interpolation operation;

[0107] Step 62) Reverse mining of the output of step 61) F ’ 3 Receive reverse guidance information R 3 :

[0108] R 3 =!( σ ( F ’ 3 ))

[0109] in It's the reverse operation, using the sigmoid function to transform the eigenvalues ​​to the range [0,1], and then using the identity matrix... E Subtracting the input from the middle yields the reverse guidance information;

[0110] Step 63) Apply the backbone features obtained in Step 1) F 3 Divided into n Groups with the same number of candidate features F n 3 , n Based on experience, it is set to 32;

[0111] Step 64) Periodically output in step 63) F n 3 Insert the output of step 62) R 3 To obtain the characteristics of the connection F cat 3 :

[0112]

[0113] Step 65) Using a 3×3 Convolution reduces the number of channels from C 3 +n Reduce to C 3 Then, compare it with the backbone features output in step 1). F 3 Adding them together yields the group connection features. for:

[0114]

[0115] in, Con (∙) indicates 3×3convolution;

[0116] Step 66) To fully utilize coordinate-aware details, interpolation is used to transform the output of step 4). F pos Adjusted to match the main characteristics F 3 The same size, and then the group connection feature output in step 65). The connection, through a 3×3 Convolution adjusts the number of channels, and finally, a ReLU function and a... 3×3 Convolution transforms feature maps into primary target features. F 3 cam :

[0117]

[0118] in Cat (∙) indicates a chain operation;

[0119] Step 7) Obtain the primary target features by interpolation operation using the data obtained in Step 6). F 3 ca The size is adjusted to be the same as the original image to obtain the primary hidden target detection map. P 3 cam ,like Figure 8 As shown;

[0120] Step 8) Use the primary target features obtained in Step 6) F 3 ca Enhanced features obtained in step 2) F glo Adding them together yields the enhanced features of the fusion. F ’ glo ;

[0121] Step 9) Apply the backbone features obtained in Step 1) F 2 The fusion enhancement features obtained in step 8) F ’ glo The coordinate features obtained in step 4) F pos The input is fed into a second multi-feature fusion module based on reverse guidance, and then processed through reverse mining. F' glo Obtaining reverse guidance information, and using group fusion to... F 2 Integrating with reverse guidance information, F posAs a supplement to location information, it enables the effective fusion of multiple features to obtain the final target features. F 2 cam The specific process is as follows:

[0122] Step 91) Use interpolation to fuse and enhance the features output from step 8). F ’ glo Size and trunk features obtained in step 1) F 2 Adjust to be consistent:

[0123]

[0124] in I (∙) indicates an interpolation operation;

[0125] Step 92) Reverse mining the output of step 91) F ’ 2 Receive reverse guidance information R 2 :

[0126] R 2 =!( σ ( F ’ 2 ))

[0127] Here, !(∙) represents the reverse operation, using the sigmoid function to transform the eigenvalues ​​to the range [0,1], and then using the identity matrix... E Subtracting the input from the middle yields the reverse guidance information;

[0128] Step 93) Apply the backbone features obtained in Step 1) F 2 Divided into n Groups with the same number of candidate features F n 2 , n Based on experience, it is set to 32;

[0129] Step 94) Periodically output in step 93) Fn 2 Insert the output of step 92) R 2 To obtain the characteristics of the connection F cat 2 :

[0130]

[0131] Step 95) Using a 3×3 Convolution reduces the number of channels from C 2 +n Reduce to C 2 Then, compare it with the backbone features output in step 1). F 2 Adding them together yields the group connection features. for:

[0132]

[0133] in, Con (∙) indicates 3×3 convolution;

[0134] Step 96) To fully utilize coordinate-aware details, interpolation is used to transform the output of step 4). F pos Adjusted to match the main characteristics F 2 The same size, and then the group connection feature output in step 95). The connection, through a 3×3 Convolution adjusts the number of channels, and finally, a ReLU function and a... 3×3 Convolution transforms feature maps into the final target features. F 2 cam :

[0135]

[0136] in Cat (∙) indicates a chain operation;

[0137] Step 10) Interpolate the final target features obtained in step 96) F 2 cam The size was adjusted to match the original image to obtain the final hidden target detection map. P 2 cam ,like Figure 9 ;

[0138] Steps 2), 6), and 9 above all utilize a joint supervision strategy to jointly train multiple branches. These branches can obtain information from multimodal data (based on pixel, object, and image ground truth). The loss function in the joint supervision strategy can be expressed as:

[0139] ;

[0140] This includes two parts:

[0141] For hidden edge supervision, in order to reduce the imbalance between positive and negative samples, only pixel-level loss is used. ;

[0142] For supervision of the hidden mask, considering the importance of local and global features, a loss function is introduced. L hybrid To guide the network in learning pixel, target, and image-level features, it can be defined as:

[0143]

[0144] in, Lw BCE It is a weighted binary cross-entropy loss. L ξ Represents enhanced alignment loss. Lw IOU It is a weighted IOU loss. To balance the numerical values, we set λ1=λ2=λ3=1.

[0145] In this embodiment 1, the detection results obtained using the above method were compared with five models: ZoomNet, OCENet, BGNet, ERRNet, and MGL. The comparison results are as follows: Figure 11 As shown.

[0146] This application conducted experiments on four hidden target datasets—CHAMELEON, CAMO, COD10K, and NC4K—based on the method described in Embodiment 1 above, and compared them with five models: ZoomNet, OCENet, BGNet, ERRNet, and MGL. The comparative experimental results are shown in Table 1. Due to space limitations and the large number of comparison images, only the other two sets of comparison images besides those in Embodiment 1 are provided here. Figure 12 , 13 As shown, these are embodiments 2 and 3 of this application, respectively.

[0147] Table 1

[0148]

[0149] Considering that the importance of errors varies depending on different dependencies or location constraints, this method uses a weighted F-metric to evaluate the accuracy of concealed target detection.

[0150] First, define G Let be the binarized truth matrix of a certain image. D This represents the detection result matrix obtained by the detection algorithm for the image. TP Represents the absolute error of the test result. E=|G - D| Therefore, commonly used basic quantities TP (True positive) FP(False positive) TN (True Negative) and FN (False negative) is defined in this embodiment as follows:

[0151] TP = ( 1 - E ) ∙ G;

[0152] TN = ( 1 - E ) ∙ ( 1 - G );

[0153] FP = E ∙ ( 1 - G );

[0154] FN = E • G；

[0155] For commonly used basic quantities TP , FP , TN and FN We obtain the formula by weighting. TP ω , FP ω , TN ω and FN ω The specific weighting process is as follows:

[0156] Introducing a pixel dependency matrix This represents the dependency values ​​between hidden target pixels, where the weights of each pixel pair are quantized using Gaussian weights:

[0157]

[0158] in, d ( i , j (pixels) i and pixels j The Euclidean distance between them σ 2 Controlling the influence of pixels that are farther away, σ 2 A larger value indicates a greater influence from distant pixels. To make better use of neighboring pixels, this application defaults to... σ 2 = 5;

[0159] Introducing an importance difference matrix False positive pixels are assigned importance weights, where the weight of each pixel is related to its distance from the hidden target, specifically defined as follows:

[0160]

[0161] in Δ ( i ) = min d ( i , j The constant α determines the decay rate, and is used in this application as follows:

[0162] ;

[0163] Obtain the matrix and Then, using the weighted formula Calculate the weighted absolute error matrix E ω , and by E ω Replace the basic quantity definition formula E To calculate the weighted average TP ω , FP ω , TN ω and FN ω Basic quantity.

[0164] Calculate using the weighted index Fω β As the final evaluation indicator, to address the shortcomings of basic quantitative indicators such as discrepancy, dependency, and equal importance, the specific expression is as follows:

[0165]

[0166] in, Precision ω This represents the prediction accuracy of the hidden target pixels. Recall ω This represents the recall rate of the hidden target pixels. β Express Precision ω and Recall ω The degree of preference. It is not difficult to find that, Fω β The larger the value, the closer the hidden target detection effect is to the ground truth map.

[0167] The data in Table 1 shows that the method proposed in this patent... Fω β The proposed method outperforms other models in all metrics, showing a significant advantage, particularly on the CAMO and COD10K datasets, indicating that it has better instance segmentation capabilities. Furthermore, it demonstrates superior performance compared to the same model on different datasets. Fω βThe indicators show that this method has smaller variance fluctuations, indicating that it has better robustness and a wider range of applications.

[0168] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting hidden targets based on transformer and multi-feature aggregation, comprising the following steps: Step 1) Acquire the detection image and input it into the pyramid transformer with the last classification layer removed. Through the pre-trained transformer-based feature extraction module, obtain the backbone features at four different scales. {F i ,i=1,2, 3,4} , i It is the level of the feature; Step 2) Apply the highest-level backbone features obtained in Step 1) F 4 The input is a feature enhancement module based on the receptive field. Dilated convolution can effectively expand the receptive field. F 4 Parallel inputs are fed into different branches, and dilated convolutions with different dilation rates are used to expand the receptive field. The outputs of each branch are summed, and after activation, channel-wise dimensionality reduction is performed to obtain single-channel enhanced features. F glo ; Step 3) Interpolate the enhanced features output from Step 2) F glo The size is adjusted to match the original image size to obtain the enhanced feature map. P glo ; Step 4) Apply the lowest-level backbone features obtained in Step 1) F 1 The input is a coordinate-based attention module, which uses two one-dimensional pooling kernels to encode long-range dependencies in both the horizontal and vertical directions. F 1 Multiplication enhances the representational power of features, resulting in coordinate features. F pos ; Step 5) Using one 1×1 Convolution will output the coordinate features from step 4). F pos The number of channels is reduced to 1, and the image size is adjusted through interpolation to obtain the coordinate feature map. P pos ; Step 6) Apply the backbone features obtained in Step 1) F 3 The enhanced features obtained in step 2) F glo The coordinate features obtained in step 4) F pos The input is fed into the first reverse-guided multi-feature fusion module, and then processed through reverse mining. F glo Obtaining reverse guidance information, and using group fusion to... F 3 Integrating with reverse guidance information, F pos As a supplement to location information, it enables the effective fusion of multiple features to obtain primary target features. F 3 cam ; Step 7) Interpolate the primary target features obtained in Step 6) F 3 cam The size is adjusted to be the same as the original image to obtain the primary hidden target detection map. P 3 cam ; Step 8) Use the primary target features obtained in Step 6) F 3 cam Enhanced features obtained in step 2) F glo Adding them together yields the enhanced features of the fusion. F’ glo ; Step 9) Apply the backbone features obtained in Step 1) F 2 The enhanced features of fusion obtained in step 8) F’ glo The coordinate features obtained in step 4) F pos The input is fed into a second multi-feature fusion module based on reverse guidance, and then processed through reverse mining. F’ glo Obtaining reverse guidance information, and using group fusion to... F 2 Integrating with reverse guidance information, F pos As a supplement to location information, it enables the effective fusion of multiple features to obtain the final target features. F 2 cam ; Step 10) Interpolate the final target features obtained in Step 9) F 2 cam The size is adjusted to match the original image to obtain the final hidden target detection map. P 2 cam .

2. The method for detecting concealed targets based on transformer and multi-feature aggregation according to claim 1, characterized in that, In step 2), the highest-level backbone features obtained in step 1) are... F 4 The input has 5 parallel branches, specifically including: Step 21) Shortcut branch utilizes a 1×1 Convolution reduces the number of channels to 64, thus reducing computational cost; Step 22) The first cascading branch utilizes a 1×1 Convolution reduces the number of channels and a pair 1×3 , 3×1 Asymmetric convolution speeds up inference; Step 23) The second cascade branch first passes through a 1×1 Convolution is used to reduce the channel dimension, and then the result is added to the output of step 22), followed by a pair of... 1×3 , 3×1 Asymmetric convolution increases inference speed, and finally passes through a convolution with an inflation rate of 3. 3×3 Dilated convolution expands the receptive field; Step 24) The third cascade branch first goes through a... 1×1 Convolution is used to reduce the channel dimension, and then the result is added to the output of step 23), followed by a pair of... 1×5 , 5×1 Asymmetric convolution increases inference speed, and finally passes through a dilation coefficient of 5. 5×5 Dilated convolution modules expand the receptive field; Step 25) The fourth cascade branch first passes through a 1×1 Convolution is used to reduce the channel dimension, and then the result is added to the output of step 24), followed by a pair of... 1×7 , 7×1 Asymmetric convolution increases inference speed, and finally passes through a dilation coefficient of 7. 7×7 The convolutional dilation module expands the receptive field; Step 26) Connect the outputs of steps 22), 23), 24), and 25), and then pass through one... 3×3 The convolution adjusts the size of the feature map, then adds it to the output of step 21), and after ReLU activation, it uses one... 1×1 Enhanced features are obtained by dimensionality reduction of convolutional channels. F glo .

3. The method for detecting concealed targets based on transformer and multi-feature aggregation according to claim 1, characterized in that, In step 4), the lowest-level backbone features obtained in step 1) are... F 1 Inputting a coordinate-based attention module involves the following steps: Step 41) To avoid the problem of difficulty in preserving location information when compressing global spatial information to the channel descriptor, utilize... and Pair the two one-dimensional pooling kernels with the lowest-level backbone features obtained in step 1). F 1 Each channel is average pooled along both the horizontal and vertical coordinates to encode the corresponding orientation-aware feature map. z h and z w ; Step 42) Connect the two outputs from step 41) via a connection operation. z h and z w By fusing, an intermediate feature map is obtained. : f= RELU ( BN ( con ([ z h , z w ]))) in, This indicates a connection operation along a spatial dimension. con express 1×1 convolution, BN It is a normalization operation. RELU It is a ReLU activation function. r This is the residual rate controlling the block size, which is set to 8 based on experience; Step 43) Decompose the output f of step 42) into two separate tensors along the spatial dimension. and : ； in σ Represents the sigmoid function; Step 44) Combine the two outputs from step 43) with the main features of the output from step 1). F 1 Multiply to obtain coordinate features F pos。 4. The method for detecting concealed targets based on transformer and multi-feature aggregation according to claim 1, characterized in that, In step 6), the backbone features obtained in step 1) are... F 3 The enhanced features obtained in step 2) F glo The coordinate features obtained in step 4) F pos The input to the first reverse-guided multi-feature fusion module includes the following steps: Step 61) Use interpolation to enhance the features output in step 2). F glo Size and trunk features obtained in step 1) F 3 Adjust to be consistent: ； in Indicates the interpolation operation; Step 62) Reverse mining of the output of step 61) F ’ 3 Receive reverse guidance information R 3 : R 3 =!( σ ( F ’ 3 )) in It's the reverse operation, using the sigmoid function to transform the eigenvalues ​​to the range [0,1], and then using the identity matrix... E Subtracting the input from the middle yields the reverse guidance information; Step 63) Apply the backbone features obtained in Step 1) F 3 Divided into n Groups with the same number of candidate features F n 3 , n Based on experience, it is set to 32; Step 64) Periodically output in step 63) F n 3 Insert the output of step 62) R 3 To obtain the characteristics of the connection F cat 3 : ； Step 65) Using a 3×3 Convolution reduces the number of channels from C 3 +n Reduce to C 3 Then, compare it with the backbone features output in step 1). F 3 Adding them together yields the group connection features. for: ； in, express 3×3 convolution; Step 66) To fully utilize coordinate-aware details, interpolation is used to transform the output of step 4). F pos Adjusted to match the main characteristics F 3 The same size, and then the group connection feature output in step 65). The connection, through a 3×3 Convolution adjusts the number of channels, and finally, a ReLU function and a... 3×3 Convolution transforms feature maps into primary target features. F 3 cam : ； in, This indicates a serial operation.

5. The method for detecting concealed targets based on transformer and multi-feature aggregation according to claim 1, characterized in that, In step 9), the backbone features obtained in step 1) are... F 2 The enhanced features of fusion obtained in step 8) F’ glo The coordinate features obtained in step 4) F pos The input to the second back-guided multi-feature fusion module includes the following steps: Step 91) Using interpolation operations to integrate the fused enhanced features output from Step 8). F’ glo Size and trunk features obtained in step 1) F 2 Adjust to be consistent: ； in Indicates the interpolation operation; Step 92) Reverse mining the output of step 91) F ’ 2 Receive reverse guidance information R 2 : R 2 =!( σ ( F ’ 2 ))； in It's the reverse operation, using the sigmoid function to transform the eigenvalues ​​to the range [0,1], and then using the identity matrix... E Subtracting the input from the middle yields the reverse guidance information; Step 93) Apply the backbone features obtained in Step 1) F 2 Divided into n Groups with the same number of candidate features F n 2 , n Based on experience, it is set to 32; Step 94) Periodically output in step 93) F n 2 Insert the output of step 92) R 2 To obtain the characteristics of the connection F cat 2 : ； Step 95) Using a 3×3 Convolution reduces the number of channels from C 2 +n Reduce to C 2 Then, compare it with the backbone features output in step 1). F 2 Adding them together yields the group connection features. for: ； in, express 3×3 convolution; Step 96) To fully utilize coordinate-aware details, interpolation is used to transform the output of step 4). F pos Adjusted to match the main characteristics F 2 The same size, and then the group connection feature output in step 95). The connection, through a 3×3 Convolution adjusts the number of channels, and finally, a ReLU function and a... 3×3 Convolution transforms feature maps into the final target features. F 2 cam : ； in This indicates a serial operation.

Images