Camouflage object segmentation method based on grouping attention fusion
By using the Transformer framework and group attention mechanism, the accuracy problem of camouflaged object segmentation in complex scenes is solved, achieving efficient segmentation of camouflaged objects and improving the network's global perception and high-level semantic feature extraction capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIAN UNIV OF TECH
- Filing Date
- 2023-05-12
- Publication Date
- 2026-04-24
AI Technical Summary
Existing methods for segmenting camouflaged objects struggle to accurately segment camouflaged targets in complex scenes, lacking global perception capabilities and advanced semantic information extraction capabilities.
The Transformer framework is used as the backbone network, and self-attention and group attention mechanisms are combined. Through image feature extraction, group attention module and multi-scale feature fusion module, the network's global perception and high-level semantic feature extraction capabilities are enhanced.
It improves the network's ability to distinguish between foreground and background in images, achieves accurate segmentation of camouflaged objects, reduces computational complexity, and is suitable for dense prediction tasks.
Smart Images

Figure CN116630620B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision and artificial intelligence technology, and relates to a method for segmenting camouflaged objects based on group attention fusion. Background Technology
[0002] Camouflaged object segmentation refers to identifying objects in an image that are highly similar to the background.
[0003] In recent years, camouflaged object segmentation has attracted increasing attention from researchers in the field of object segmentation. Camouflage, derived from biology, is a powerful and ingenious way to hide oneself. Camouflaged objects typically blend into their surroundings through color changes, environmental mimicry, or partial occlusion; many animals in nature use this method to evade predators or hunt prey. Due to its ability to identify camouflaged objects, camouflaged object segmentation algorithms have broad application prospects in many fields, such as species detection, polyp segmentation, lung infection segmentation, and disaster search and rescue.
[0004] However, the high similarity between the target and the background also presents a greater challenge to the task of camouflaged object segmentation. Unlike general target segmentation algorithms, camouflaged target segmentation requires the network to have greater global perception capabilities and capture more high-level semantic information in order to accurately separate the target from the background. Most existing methods are based on CNN networks to extract features and use boundary or texture cues to enhance the model's perception capabilities. However, in some complex scenes, they cannot accurately segment camouflaged targets.
[0005] We use the Transformer framework as the backbone network for feature extraction and establish global perception capability with the help of self-attention mechanism. Then we designed a group fusion module to further mine high-level semantic features with the help of group attention mechanism, which enhances the network's ability to distinguish between foreground and background. Finally, we construct a multi-scale feature fusion module to recover high-resolution predicted images step by step from coarse to fine. Summary of the Invention
[0006] The purpose of this invention is to provide a camouflaged object segmentation method based on group attention fusion, which can accurately segment camouflaged objects; it greatly improves the network's ability to distinguish between foreground and background in images, and solves the problem that existing methods are difficult to segment camouflaged objects in complex scenes.
[0007] The technical solution adopted in this invention is a camouflaged object segmentation method based on group attention fusion, which is implemented according to the following steps:
[0008] Step 1: Extract features from the input image using the image feature extraction module;
[0009] Step 2, construct the group attention module;
[0010] Step 3: Construct a multi-scale feature fusion module.
[0011] The invention is further characterized by:
[0012] Specifically, step 1 involves the image feature extraction module using a Transformer module with a pyramid structure to extract features from the input image;
[0013] Step 1 is implemented in the following steps:
[0014] Step 1.1: The input image first passes through the first Transformer module to process the image to be detected and output the first feature map;
[0015] Step 1.2: The first feature map passes through the second Transformer module to process the features, and the operation is the same as above, outputting the second feature map F2;
[0016] Step 1.3: The second feature map passes through the third Transformer module to process the features, and the operation is the same as above, outputting the third feature map F3;
[0017] Step 1.4: The fourth feature map passes through the fourth Transformer module to process the features, and the operation is the same as above, outputting the fourth feature map F4;
[0018] Step 1.1 is implemented as follows:
[0019] Step 1.1.1: The input image first passes through the PatchEmbedding module to divide the image into H×W×3 segments. Where H and W are the width and height of the image, P i and C i These are hyperparameters, representing the number of blocks and the number of output channels, respectively. The division here uses overlapping and cross-division to enhance the interaction between blocks and enable the network to capture more global information.
[0020] Step 1.1.2: The processed features are processed through a self-attention mechanism module with pyramid pooling, as shown in the following formula:
[0021]
[0022] In the formula, These represent the quary, the pooled key, and the pooled value in the self-attention mechanism, respectively. head Represents the number of multi-head attention; Softmax() refers to the activation function.
[0023] Step 1.1.3: After self-attention, the features are processed by a feedforward neural network. LaryNorm is used for layer normalization. Then, a linear mapping is used to amplify the features by a factor of four, and then a linear mapping is used to shrink the features by a factor of four. Finally, the first feature map F1 is output.
[0024] Step 2, constructing the grouped attention module, is implemented as follows:
[0025] Step 2.1: Perform dimensionality reduction operations on the four layers of features extracted from the backbone network.
[0026] Step 2.2: Perform cross-fusion on the four sets of features after dimensionality reduction;
[0027] Step 2.3: Denote the four sets of features after cross-fusion as f′1, f′2, f′3, and f′4, and perform group attention fusion on each set.
[0028] Step 2.1 is implemented as follows:
[0029] Step 2.1.1: Perform convolution operations with 3x3 kernels on the four layers of features respectively, set the number of output channels to 64, use BatchNorm for normalization, and finally use ReLU operation for activation processing;
[0030] Step 2.1.2: Perform a 3x3 convolution operation on the four processed features respectively, with the input and output channels set to the same value of 64, denoted as f1, f2, f2, f4;
[0031] Step 2.2 is implemented as follows:
[0032] Step 2.2.1: Since the fourth layer features have no upper layer features, only two 3x3 convolution operations are performed on them, keeping the number of channels the same.
[0033] Step 2.2.2: Split the processed fourth group of features into N groups according to channels, split the unprocessed third group of features into N groups according to the number of channels, upsample the fourth group and perform cross-fusion, and then perform fusion through a 3x3 convolution module. The number of input channels is 128 and the number of output channels is 64.
[0034] Step 2.2.3: Split the processed third group of features into N groups according to channels, split the unprocessed second group of features into N groups according to the number of channels, upsample the third group and perform cross-fusion, and then perform fusion through a 3x3 convolution module. The number of input channels is 128 and the number of output channels is 64.
[0035] Step 2.2.4: Split the processed second group of features into N groups according to channels, split the unprocessed first group of features into N groups according to the number of channels, upsample the second group and perform cross-fusion, and then perform fusion through a 3x3 convolution module. The number of input channels is 128 and the number of output channels is 64.
[0036] Step 2.3 is implemented in the following steps:
[0037] Step 2.3.1: For the fourth set of features after cross-fusion, first split it into M groups according to the number of channels, denoted as G. i For i∈{1,...,M}, first perform a dimension increase operation on G1, using a 3x3 convolution to expand its channel count to 3 times the original, denoted as G. 1j , j∈{1,2,3};
[0038] Step 2.3.2, G 1j Divide into 3 groups based on the number of channels, and take the first group G. 11 Perform a Cat operation with G2; then G2 also expands the number of channels to obtain G. 2j , j∈{1,2,3}; The operation for G3 is the same as above, and a total of M groups of features are generated. Each group of features is then split into three groups according to the number of channels.
[0039] Step 2.3.3, select the first group G from each of the M groups of features. i1 After catching the cat, perform a convolution operation, and set the output channel to 64;
[0040] Step 2.3.4, select the third group G from each of the M groups of features. i3 After catching the cat, perform a convolution operation and set the output channel to 1;
[0041] Step 2.3.5: Finally, the feature with 1 channel is multiplied element-wise with the feature with 64 channels, and then added element-wise to the initially cross-fused feature f1′. A residual connection is then performed to obtain the final output f. i ", i∈{1,2,3,4}; the entire group attention mechanism can be expressed by the following formula:
[0042] f i "=f i ′+GA(CBR(chunk(f i ′, k))), k∈{1,...,M} (2)
[0043] In the formula, chunk() represents the channel splitting operation, CBR() represents the convolution, BatchNorm and ReLU operations, and GA() represents the group attention operation;
[0044] Step 3 is implemented in the following steps:
[0045] Step 3.1: Perform multi-scale fusion on the four-layer features obtained from the grouped attention module;
[0046] Step 3.2: Calculate the loss between the predicted image and the ground truth labeled image, and adjust the network parameters through backpropagation;
[0047] Step 3.1 is implemented in the following steps:
[0048] Step 3.1.1: For the fourth-layer feature f″4 output by the group attention module, since there are no upper-layer features, we directly upsample it, then perform a 3x3 convolution operation, and then perform element-wise multiplication with the third-layer features, denoted as... The formula is expressed as follows:
[0049]
[0050] Step 3.1.2: For the third-layer feature f″3 output by the attention module, upsample it and then perform a 3x3 convolution operation. Then, the f3 obtained in step 3.1.1 is... * Similarly, upsampling and convolution are performed, and these are then multiplied element-wise with the upper-layer feature f2″ to obtain f2. * The formula is expressed as follows:
[0051]
[0052] Step 3.1.3: For the second-layer feature f″2 output by the attention module, upsample it and then perform a 3x3 convolution operation. Then, the f2 obtained in step 3.1.2 is... * Similarly, upsampling and convolution are performed, and then element-wise multiplication is performed together with the upper-layer feature f1″ to obtain f1. * The formula is expressed as follows:
[0053]
[0054] In step 3.1.4, since f4″ has no upper-layer features, it is denoted as p4. After upsampling and convolving p4, it is then combined with f3. * Performing the Cat operation yields p3; p2 is also generated by the same operation, as shown in the following formula:
[0055] p i =Cat(f i * ,CBR(U(p i+1 ))), i∈[1,2,3] (6)
[0056] Step 3.1.5, finally yields four sets of features p. iFor i∈{1,2,3,4}, p1 is processed through two 3x3 convolution operations. The first layer keeps the number of channels undisclosed, and the second layer reduces the number of channels to 1 to obtain the final prediction image P.
[0057] Step 3.2 specifically involves:
[0058] Step 3.2.1: Apply weighted binary cross-entropy loss and weighted intersection-over-union (IoU) loss to the predicted image P and the ground truth labeled image G, respectively. The formulas are as follows:
[0059]
[0060] Step 3.2.2: Calculate the uncertainty-perceived loss by comparing the predicted image P with the ground truth labeled image G. The formula is as follows:
[0061] L U =λ×(1-|2p-1| 2 (8)
[0062] In the formula, p is the pixel value of each point in the prediction image, and λ is the hyperparameter;
[0063] Step 3.2.3: Add the two loss calculations together to get the final loss value.
[0064] The beneficial effects of this invention are:
[0065] The camouflaged object segmentation method based on group attention fusion of this invention does not use convolutional neural networks, but is built on the Transformer architecture. It leverages the self-attention mechanism to better establish global dependencies. Using a Transformer module with a pyramid structure significantly reduces computational complexity, making the network better suited for dense prediction segmentation tasks. The proposed group attention fusion module employs a group interaction mode to better explore the weight relationships between channels. Through learning and adjustment, it obtains the optimal weight ratio, further mining high-level semantic information of the image and enhancing the network's ability to distinguish between foreground and background. The proposed multi-scale feature fusion module adopts a coarse-to-fine fusion strategy, effectively fusing multi-level features to generate accurate high-resolution prediction maps while maximizing the preservation of the network's global perception capabilities. Attached Figure Description
[0066] Figure 1 This is a structural diagram of the group attention module used in step 2 of the camouflaged object segmentation method based on group attention fusion of the present invention;
[0067] Figure 2 This is a visualization of the image processing result of the group attention fusion module used in step 2 of the camouflaged object segmentation method based on group attention fusion of the present invention.
[0068] Figure 3 This is a structural diagram of the multi-scale feature fusion module used in step 3 of the camouflaged object segmentation method based on group attention fusion of the present invention;
[0069] Figure 4 This is a schematic diagram of the overall model framework of the camouflaged object segmentation method based on group attention fusion of the present invention.
[0070] Figure 5 This refers to the original image and the ground truth labeled image of COD10K-CAM-1-Aquatic-9-GhostPipefish-326 in the COD10K dataset in step 1 of the embodiment of the camouflaged object segmentation method based on group attention fusion of the present invention;
[0071] Figure 6 The image segmentation result of the camouflaged object segmentation method based on group attention fusion of the present invention is shown on the COD10K-CAM-1-Aquatic-9-GhostPipefish-326 image in the public dataset COD10K.
[0072] Figure 7 The results show the comparison of the Em evaluation index of the camouflaged object segmentation method based on group attention fusion of the present invention with 16 existing methods on four public datasets. The highest curve is that of the method of the present application.
[0073] Figure 8 The results show the comparison of the Fm evaluation index of the camouflaged object segmentation method based on group attention fusion of the present invention with 16 existing methods on four public datasets. The highest curve is that of the method of the present application.
[0074] Figure 9 The results show the comparison of the PR evaluation metrics of the camouflaged object segmentation method based on group attention fusion of the present invention with those of 16 existing methods on four public datasets, with the highest curve being the method of this application. Detailed Implementation
[0075] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.
[0076] This invention provides a method for camouflaged object segmentation based on group attention fusion, such as... Figure 4 As shown, please follow these steps:
[0077] Step 1: Extract features from the input image using the image feature extraction module;
[0078] Step 2, construct the group attention module;
[0079] Step 3: Construct a multi-scale feature fusion module.
[0080] Example 1
[0081] In the camouflaged object segmentation method based on group attention fusion in this application, step 1 specifically involves the image feature extraction module using a Transformer module with a pyramid structure to extract features from the input image, which is implemented in the following steps:
[0082] Step 1.1: The input image first passes through the first Transformer module to process the image to be detected, outputting the first feature map:
[0083] Step 1.1.1: The input image first passes through the PatchEmbedding module to divide the image into H×W×3 segments. Where H and W are the width and height of the image, P i and C i These are hyperparameters, representing the number of blocks and the number of output channels, respectively. The division here uses overlapping and cross-division to enhance the interaction between blocks and enable the network to capture more global information.
[0084] Step 1.1.2: The processed features are processed through a self-attention mechanism module with pyramid pooling, as shown in the following formula:
[0085]
[0086] In the formula, These represent the quary, the pooled key, and the pooled value in the self-attention mechanism, respectively. head The number of multi-head attention is represented by Softmax(), which refers to the activation function. The self-attention mechanism with pyramid pooling is used to reduce computational complexity and make the network more suitable for dense prediction tasks.
[0087] Step 1.1.3: After self-attention, the features are processed by a feedforward neural network. LaryNorm is used for layer normalization. Then, a linear mapping is used to amplify the features by a factor of four, and then a linear mapping is used to shrink the features by a factor of four. Finally, the first feature map F1 is output.
[0088] Step 1.2: The first feature map passes through the second Transformer module to process the features, and the operation is the same as above, outputting the second feature map F2;
[0089] Step 1.3: The second feature map passes through the third Transformer module to process the features, and the operation is the same as above, outputting the third feature map F3;
[0090] Step 1.4: The fourth feature map passes through the fourth Transformer module to process the features, and the operation is the same as above, outputting the fourth feature map F4;
[0091] Example 2
[0092] The second step of the camouflaged object segmentation method based on group attention fusion of the present invention, which involves constructing a group attention module, is specifically implemented according to the following steps, such as... Figure 1 and Figure 2 As shown:
[0093] Step 2.1: Perform dimensionality reduction operations on the four layers of features extracted from the backbone network:
[0094] Step 2.1.1: Perform convolution operations with 3x3 kernels on the four layers of features respectively, set the number of output channels to 64, use BatchNorm for normalization, and finally use ReLU operation for activation processing;
[0095] Step 2.1.2: Perform a 3x3 convolution operation on the four processed features respectively, with the input and output channels set to the same value of 64, denoted as f1, f2, f3, and f4.
[0096] Step 2.2: Perform cross-fusion on the four sets of features after dimensionality reduction:
[0097] Step 2.2.1: Since the fourth layer features have no upper layer features, only two 3x3 convolution operations are performed on them, keeping the number of channels the same.
[0098] Step 2.2.2: Split the processed fourth group of features into N groups according to channels, split the unprocessed third group of features into N groups according to the number of channels, upsample the fourth group and perform cross-fusion, and then perform fusion through a 3x3 convolution module. The number of input channels is 128 and the number of output channels is 64.
[0099] Step 2.2.3: Split the processed third group of features into N groups according to channels, split the unprocessed second group of features into N groups according to the number of channels, upsample the third group and perform cross-fusion, and then perform fusion through a 3x3 convolution module. The number of input channels is 128 and the number of output channels is 64.
[0100] Step 2.2.4: Split the processed second group of features into N groups according to channels, split the unprocessed first group of features into N groups according to the number of channels, upsample the second group and perform cross-fusion, and then perform fusion through a 3x3 convolution module. The number of input channels is 128 and the number of output channels is 64.
[0101] Step 2.3: Denote the four sets of features after cross-fusion as f1′, f2′, f3′, and f4′, and perform group attention fusion on each set:
[0102] Step 2.3.1: For the fourth set of features after cross-fusion, first split it into M groups according to the number of channels, denoted as G. i For i∈{1,...,M}, first perform a dimension increase operation on G1, using a 3x3 convolution to expand its channel count to 3 times the original, denoted as G. 1j , j∈{1,2,3};
[0103] Step 2.3.2, G 1j Divide into 3 groups based on the number of channels, and take the first group G. 11 Perform a Cat operation with G2; then G2 also expands the number of channels to obtain G. 2j , j∈{1,2,3}; The operation for G3 is the same as above, and a total of M groups of features are generated. Each group of features is then split into three groups according to the number of channels.
[0104] Step 2.3.3, select the first group G from each of the M groups of features. i1 After catching the cat, perform a convolution operation, and set the output channel to 64;
[0105] Step 2.3.4, select the third group G from each of the M groups of features. i3 After catching the cat, perform a convolution operation and set the output channel to 1;
[0106] Step 2.3.5: Finally, the feature with 1 channel is multiplied element-wise with the feature with 64 channels, and then added element-wise to the initially cross-fused feature f1′. A residual connection is then performed to obtain the final output f. i ", i∈{1,2,3,4}; the entire group attention mechanism is expressed by the following formula:
[0107] f″ i =f′ i +GA(CBR(chunk(f i ′, k))), k∈{1,...,M} (2)
[0108] In the formula, chunk() represents the channel splitting operation, CBR() represents the convolution, BatchNorm and ReLU operations, and CA() represents the group attention operation.
[0109] Example 3
[0110] Step 3 of the camouflaged object segmentation method based on group attention fusion of the present invention is specifically implemented according to the following steps, such as... Figure 3 As shown:
[0111] Step 3.1: Perform multi-scale fusion on the four-layer features obtained from the grouped attention module:
[0112] Step 3.1.1: For the fourth-layer feature f″4 output by the group attention module, since there are no upper-layer features, we directly upsample it, then perform a 3x3 convolution operation, and then perform element-wise multiplication with the third-layer feature, denoted as f3. * The formula is expressed as follows:
[0113]
[0114] Step 3.1.2, for the third-layer feature f output by the attention module a ", then upsample it and perform a 3x3 convolution operation, then the f3 obtained in step 3.1.1 * Similarly, upsampling and convolution are performed, and these are then multiplied element-wise with the upper-layer feature f2″ to obtain f2. * The formula is expressed as follows:
[0115]
[0116] Step 3.1.3: For the second-layer feature f2″ output by the attention module, upsample it and then perform a 3x3 convolution operation. Then, the f2 obtained in step 3.1.2 is... * Similarly, upsampling and convolution are performed, and then element-wise multiplication is performed together with the upper-layer feature f1″ to obtain f1. * The formula is expressed as follows:
[0117]
[0118] In step 3.1.4, since f4″ has no upper-layer features, it is denoted as p4. After upsampling and convolving p4, it is then combined with f3. * Performing the Cat operation yields p3; p2 is also generated by the same operation, as shown in the following formula:
[0119] p i =Cat(f i * , CBR(U(pi+1))), i∈[1, 2, 3] (6)
[0120] Step 3.1.5, finally yields four sets of features p. i For i∈{1,2,3,4}, p1 is processed through two 3x3 convolution operations. The first layer keeps the number of channels undisclosed, and the second layer reduces the number of channels to 1 to obtain the final prediction image P.
[0121] Step 3.2: Calculate the loss between the predicted image and the ground truth labeled image, and adjust the network parameters through backpropagation.
[0122] Step 3.2.1: Apply weighted binary cross-entropy loss and weighted intersection-over-union (IoU) loss to the predicted image P and the ground truth labeled image G, respectively. The formulas are as follows:
[0123]
[0124] Step 3.2.2: Calculate the uncertainty-perceived loss by comparing the predicted image P with the ground truth labeled image G. The formula is as follows:
[0125] L U =λ×(1-|qp-1| 2 (8)
[0126] In the formula, p is the pixel value of each point in the prediction image, and λ is the hyperparameter;
[0127] Step 3.2.3: Add the two loss calculations together to get the final loss value.
[0128] like Figure 5 The image shown is the original image and the ground truth labeled image of COD10K-CAM-1-Aquatic-9-GhostPipefish-326 from the public dataset COD10K;
[0129] like Figure 6 The image shown is the segmentation result of the COD10K-CAM-1-Aquatic-9-GhostPipefish-326 image from the public dataset COD10K.
[0130] like Figure 7 , Figure 8 , Figure 9 The figure shows the E-values of this invention on four public datasets compared to 16 existing methods. m F m The comparison results of PR evaluation metrics show that the highest curve is our method, demonstrating that our method exhibits leading performance across all metrics.
Claims
1. A camouflaged object segmentation method based on group attention fusion, characterized in that, The specific steps are as follows: Step 1: Extract features from the input image using the image feature extraction module; Step 2, construct the group attention module; Step 3: Construct a multi-scale feature fusion module; Step 2, constructing the grouped attention module, is implemented according to the following steps: Step 2.1: Perform dimensionality reduction operations on the four layers of features extracted from the backbone network. Step 2.2: Perform cross-fusion on the four sets of features after dimensionality reduction; Step 2.3, denote the four sets of features after cross-fusion as follows: Attention fusion was performed in groups separately; Step 2.1 is implemented in the following steps: Step 2.1.1: Perform convolution operations with 3x3 kernels on the four layers of features respectively, set the number of output channels to 64, use BatchNorm for normalization, and finally use ReLU operation for activation processing; Step 2.1.2: Perform a 3x3 convolution operation on each of the four processed feature layers, with the input and output channels set to the same value of 64. This is denoted as... ; Step 2.2 is implemented in the following steps: Step 2.2.1: Since the fourth layer features have no upper layer features, only two 3x3 convolution operations are performed on them, keeping the number of channels the same. Step 2.2.2: Split the processed fourth group of features into N groups according to channels, split the unprocessed third group of features into N groups according to the number of channels, upsample the fourth group and perform cross-fusion, and then perform fusion through a 3x3 convolution module. The number of input channels is 128 and the number of output channels is 64. Step 2.2.3: Split the processed third group of features into N groups according to channels, split the unprocessed second group of features into N groups according to the number of channels, upsample the third group and perform cross-fusion, and then perform fusion through a 3x3 convolution module. The number of input channels is 128 and the number of output channels is 64. Step 2.2.4: Split the processed second group of features into N groups according to channels, split the unprocessed first group of features into N groups according to the number of channels, upsample the second group and perform cross-fusion, and then perform fusion through a 3x3 convolution module. The number of input channels is 128 and the number of output channels is 64. Step 2.3 is implemented in the following steps: Step 2.3.1: For the fourth set of features after cross-fusion, first split it into M groups according to the number of channels, denoted as... First of all To perform a dimensionality increase operation, a 3x3 convolution is used to triple the number of channels, denoted as . ; Step 2.3.2, will Divide into 3 groups based on the number of channels, and take the first group. and Perform the Cat operation; then Similarly, by expanding the number of channels, we obtain... ;for The operation is the same as above, and a total of M groups of features are generated. Each group of features is then split into three groups according to the number of channels. Step 2.3.3: Select the first group from each of the M groups of features. After catching the cat, perform a convolution operation, and set the output channel to 64; Step 2.3.4, select the third group from each of the M groups of features. After catching the cat, perform a convolution operation and set the output channel to 1; Step 2.3.5: Finally, the feature with 1 channel is multiplied element-wise with the feature with 64 channels, and then multiplied with the initially cross-fused feature. Add elements one by one, perform residual joins, and obtain the final output. The entire group attention mechanism can be represented by the following formula: (2) In the formula, This indicates a channel-based split operation. This represents the convolution, BatchNorm, and ReLU operations. This indicates a grouped attention operation.
2. The camouflaged object segmentation method based on group attention fusion according to claim 1, characterized in that, Step 1 specifically involves the image feature extraction module using a Transformer module with a pyramid structure to extract features from the input image.
3. The camouflaged object segmentation method based on group attention fusion according to claim 2, characterized in that, Step 1 is implemented in the following steps: Step 1.1: The input image first passes through the first Transformer module to process the image to be detected and output the first feature map; Step 1.2: The first feature map passes through the second Transformer module, where the features are processed in the same way as above, and the second feature map is output. ; Step 1.3: The second feature map passes through the third Transformer module, where the features are processed in the same way as above, and the third feature map is output. ; Step 1.4: The fourth feature map passes through the fourth Transformer module, where the features are processed in the same way as above, and the fourth feature map is output. .
4. The camouflaged object segmentation method based on group attention fusion according to claim 3, characterized in that, Step 1.1 is implemented in the following steps: Step 1.1.1: The input image first passes through the PatchEmbedding module to segment the image, thus dividing the input image... After being divided into Where H and W are the width and height of the image, and These are hyperparameters, representing the number of blocks and the number of output channels, respectively. The division here uses overlapping and cross-division to enhance the interaction between blocks and enable the network to capture more global information. Step 1.1.2: The processed features are processed through a self-attention mechanism module with pyramid pooling, as shown in the following formula: (1) In the formula, These represent the quary, the key after pooling, and the value after pooling in the self-attention mechanism, respectively. The amount of attention represented by bullish sentiment. Activation function; Step 1.1.3: After self-attention, the features are processed by a feedforward neural network. LaryNorm is used for layer normalization, then a linear mapping is used to amplify the features by a factor of four, and then a linear mapping is used to shrink the features by a factor of four, finally outputting the first feature map. .
5. The camouflaged object segmentation method based on group attention fusion according to claim 1, characterized in that, Step 3 is implemented in the following steps: Step 3.1: Perform multi-scale fusion on the four-layer features obtained from the grouped attention module; Step 3.2: Calculate the loss between the predicted image and the ground truth labeled image, and adjust the network parameters through backpropagation.
6. The camouflaged object segmentation method based on group attention fusion according to claim 5, characterized in that, Step 3.1 is implemented in the following steps: Step 3.1.1, for the fourth layer features output by the group attention module Since there are no upper-layer features, we directly upsample them, then perform a 3x3 convolution, and finally multiply them element-wise with the third-layer features, denoted as... The formula is expressed as follows: (3) Step 3.1.2, for the third layer features output by the attention module After upsampling, a 3x3 convolution operation is performed, and then the result obtained in step 3.1.1 is... It also performs upsampling and convolution, which are then combined with the upper-layer features. We perform element-wise multiplication together and get... The formula is expressed as follows: (4) Step 3.1.3, for the second layer features output by the attention module After upsampling, a 3x3 convolution operation is performed, and then the result obtained in step 3.1.2 is... It also performs upsampling and convolution, which are then combined with the upper-layer features. We perform element-wise multiplication together and get... The formula is expressed as follows: (5) Step 3.1.4, Since it lacks upper-level features, it is denoted as ,Will After upsampling and convolution, and with Perform a Cat operation to get ; It is also generated by the same operation, and the formula is expressed as follows: (6) Step 3.1.5 ultimately yields four sets of features. ,Will After two layers of 3x3 convolution operations, the first layer keeps the number of channels undisclosed, and the second layer reduces the number of channels to 1, resulting in the final prediction image P; Step 3.2 specifically involves: Step 3.2.1: Apply weighted binary cross-entropy loss and weighted intersection-over-union (IoU) loss to the predicted image P and the ground truth labeled image G, respectively. The formulas are as follows: (7) Step 3.2.2: Calculate the uncertainty-perceived loss by comparing the predicted image P with the ground truth labeled image G. The formula is as follows: (8) In the formula, p is the pixel value of each point in the predicted image. Hyperparameters; Step 3.2.3: Add the two loss calculations together to get the final loss value.
Citation Information
Patent Citations
Image classification method based on high-precision and lightweight feature extraction network model
CN115311489A