Prostate image segmentation method with multi-granularity self-attention mechanism
By introducing a multi-granularity self-attention mechanism into the U-Net structure, the problems of insufficient local information and limited global perception capability of the standard U-Net in prostate image segmentation are solved, improving the segmentation accuracy and robustness of the model, and making it particularly suitable for prostate MRI images with complex structures or blurred boundaries.
Patent Information
- Application Number
- CN202511307100.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-13
- Publication Date
- 2026-02-10
AI Technical Summary
When processing prostate images with complex structures or blurred boundaries, the standard U-Net structure suffers from insufficient local information or limited global perception capabilities, affecting the accuracy and robustness of the segmentation results.
A multi-granularity self-attention mechanism is introduced after the convolutional module of U-Net. By splitting the convolutional features into two parts, local point-to-point relationships and regional global dependencies are modeled separately, thereby enhancing the feature representation capability.
The model's ability to perceive detailed boundaries and overall morphology in prostate images has been improved, resulting in more stable segmentation performance. It balances computational efficiency and accuracy and is suitable for medical image segmentation tasks with complex structures or blurred boundaries.
Smart Images
Figure CN121504797A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of medical image processing, more particularly, it relates to a prostate image segmentation method with a multi-granularity self-attention mechanism. BACKGROUND
[0002] Prostate cancer is one of the most common malignant tumors in men, and accurate diagnosis and treatment at an early stage are of great significance for patient prognosis. Magnetic resonance imaging (MRI) is an important imaging method for evaluating prostate structure and lesion distribution, and is widely used in the clinical screening and diagnosis of prostate diseases due to its high soft tissue contrast. In order to further improve the efficiency and accuracy of clinical diagnosis, automatic segmentation methods based on deep learning have been extensively studied and applied.
[0003] Among them, the U-Net structure has excellent feature extraction and reconstruction ability, and performs well in medical image segmentation tasks. However, the standard U-Net structure still faces problems of insufficient local information or limited global perception ability when dealing with prostate images with complex structures or fuzzy boundaries, affecting the accuracy and robustness of the segmentation results. Therefore, how to enhance the model's ability to model details and global information while maintaining computational efficiency has become a key problem in prostate segmentation research. SUMMARY
[0004] The purpose of the present application is to provide a prostate image segmentation method with a multi-granularity self-attention mechanism, which improves the classic U-Net structure to improve the model's segmentation accuracy of the prostate region, especially in images with complex structures or fuzzy boundaries. Specifically, a multi-granularity self-attention mechanism is introduced after the convolution module of U-Net to enhance the feature expression ability.
[0005] The above technical purpose of the present application is achieved by the following technical solution: a prostate image segmentation method with a multi-granularity self-attention mechanism, which introduces a multi-granularity self-attention mechanism after the convolution module of U-Net to establish a prostate image segmentation model to enhance the feature expression ability. This method processes each group of convolutional feature maps by branching, dividing them into two parts to reduce the computational burden while capturing context information at different scales; the first branch focuses on fine-grained feature modeling, using a self-attention mechanism to calculate the relationship between each feature point in the graph, emphasizing the structural details within the local region, and reconstructing the original features accordingly; the second branch focuses on a lower-granularity but more globally-aware modeling approach, capturing long-range dependency information through the relationship between feature points and image regions, and improving the model's understanding of the overall structure; finally, the features reconstructed at different granularities are fused to effectively combine local details and global semantic information, thereby enhancing the prostate image model's performance in complex prostate images.
[0006] The present invention is further configured such that the method specifically includes the following steps:
[0007] S1 Data Input: Read 3D images and labels, perform data augmentation, and convert them into 2D image format, which is then input into the prostate image segmentation model for calculation;
[0008] The S2 prostate image segmentation model processes and analyzes images: it inputs a 2D image, encodes and decodes it using the prostate image segmentation model, then uses a segmentation head to make predictions, and finally compares the predictions with the true labels.
[0009] S3 multi-granularity self-attention module analysis and processing: The convolutional features are divided into two parts, one part is used to calculate the relationship between each feature point and the other part is used to calculate the relationship between each feature point and each region.
[0010] The present invention is further configured such that step S1 specifically includes the following steps:
[0011] S11 Input data, input image data x of training set sample i. (i) and corresponding label y (i) ;
[0012] S12 data augmentation, for x (i) Perform spatial-based data augmentation; y (i) The same spatial transformation is also performed accordingly;
[0013] S13 image sampling: Since the prostate image segmentation model in this method uses a 2D model, the input size requirement is: channel * length * width. Therefore, in x... (i) Randomly select a consecutive number of images, say c images, as 2D input, i.e., c × w × d, c channels; the corresponding y (i) Do the same treatment;
[0014] S14 will use the 2D image x from S13 (i) As input to the segmentation model, y (i) Then it is used as a label for loss calculation.
[0015] The present invention is further configured such that step S2 specifically includes the following steps:
[0016] S21 Initial Convolution: For the input image, let the size be c×w×d. Use the convolutional network to extract features from it, output features with size c0×w×d, and input the output features into the subsequent encoding network.
[0017] S22 encoding network, the segmentation model comprises L encoding layers, each encoding layer is composed of a downsampling network, a convolution network and a multi-granularity self-attention module; the output feature of the encoding layer is input into the next encoding layer and the corresponding decoding layer;
[0018] S23 intermediate layer, the output feature of the Lth encoding layer is input into the intermediate layer, the intermediate layer has the same network result as the encoding layer, that is, first downsampling, then convolution, and finally attention calculation, and the output is input into the decoding network;
[0019] S24 decoding network, the decoding network has L decoding layers, and the network structure is similar to that of the encoding layer, that is, upsampling, convolution and attention;
[0020] S25 segmentation head, the first decoding feature is upsampled and spliced with the output feature of the initial convolution in the channel dimension to obtain a feature with a size of (c+c0) x w x d; the segmentation head, that is, the convolution network, is used for calculation to obtain the final prediction result The size is c x w x d.
[0021] S26 loss calculation, the predicted result and the real label y (i) are compared, the loss is calculated, and the parameters of the segmentation model are updated according to the loss.
[0022] The application further provides that the S22 specifically comprises the following steps:
[0023] S221 downsampling, for the input feature F, assuming that the feature size is c j x w j x d j , first pass through the downsampling network, the downsampling network is a maximum pooling, average pooling or convolution network, to obtain a feature with a size reduced by half, that is, denoted as c j x w j+1 x d j+1 ;
[0024] S222 convolution, the output feature of S221 is calculated by using the convolution network to extract features with higher orders and larger receptive fields in the feature F in S221;
[0025] S223 attention, the feature of S222 is input into the multi-granularity attention module, different fine-grained features are calculated, and the different fine-grained features are fused, and finally the output feature is output, which is input into the next encoding layer and the corresponding decoding layer.
[0026] The application further provides that the S24 specifically comprises the following steps:
[0027] S241 upsampling: For the input feature F, let the feature size be c. j+1 ×w j+1 ×d j+1 First, upsample the image to gradually restore its size, resulting in a size of c. j+1 ×2w j+1 ×2d j+1 The characteristic, denoted as c j+1 ×w j ×d j ;
[0028] S242 feature concatenation involves concatenating the features from S241 with the corresponding size features from the coding layer. At this point, it should have c j ×w j ×d j The encoded features are concatenated along the channel dimension, meaning the size of the concatenated feature is (c j +c j+1 )×w j ×d j ;
[0029] The S243 decoding computation, based on the features of S242, first utilizes a convolutional network, then employs multi-granularity attention to perform computation, yielding the final decoded features, with size c. j ×w j ×d j This decoding feature will be input into the next decoding layer.
[0030] The present invention is further configured such that step S3 specifically includes the following steps:
[0031] S31 convolution: For the input feature F, let the size be c×w×d. First, use the convolutional network to calculate the output feature, with a size of k×w×d, where k is an even number.
[0032] Feature S32 partitioning divides the features obtained in S31 into two along the channel dimension, resulting in feature F1 with a size of [missing information]. And feature F2, the size is also
[0033] S33 fine-grained attention: For feature F1, three convolutional networks are first used to compute Q, K, and V respectively; then, matrix multiplication is performed using Q and K to obtain the relationship between each feature point; finally, using the relationship between feature points, matrix multiplication is performed with V to reconstruct the feature and output F'1, with a size of [missing information].
[0034] S34 is a coarse-grained attention feature. Feature F2 is computed using a convolutional network to obtain Q, with a size of [missing information]. Then, average pooling is used to calculate feature F2, resulting in K1 and V1, with dimensions of... K1 = V1. Based on Q and K1, calculate the relationship between each feature point in Q and each p×p region in feature F2 (i.e., K1). Finally, using the relationship between feature points and regions, perform matrix multiplication with V1 to reconstruct the features and output feature O1. Then, continue using average pooling based on K1 and V1 to obtain K2 and V2, with dimensions of... Here, K2 = V2, and the output feature O2 is calculated using the same method, ultimately yielding attention features of different fine granularities;
[0035] S35 feature concatenation: The features F'1 of S33 and O1 and O2 of S34 are concatenated along the channel dimension. The concatenated features are then computed using a convolutional network to obtain the final output feature F', with dimensions c×w×d.
[0036] In summary, the present invention has the following beneficial effects:
[0037] 1. Introducing a multi-granularity self-attention mechanism to balance local details and global structure: This method innovatively introduces a multi-granularity self-attention mechanism based on the traditional U-Net structure. By dividing the convolutional features into two branches of different granularities, it models local point-to-point relationships and region-level global dependencies respectively, effectively improving the model's ability to perceive detail boundaries and overall shape in the image.
[0038] 2. Feature branch design improves efficiency and enriches representation: Unlike directly using the entire feature to perform different attention calculations sequentially, this scheme divides the convolutional feature into two parts, used for fine-grained and coarse-grained attention modeling respectively. This design not only avoids computational redundancy but also achieves complementary enhancement of feature representation, improving model performance without significantly increasing the computational burden.
[0039] 3. Integrating contextual information at different scales to achieve more stable segmentation results: By fusing features reconstructed at two granularities, the model can more comprehensively understand the morphological features of the prostate, enhancing its adaptability in complex structures or with blurred boundaries, and making the segmentation results more stable and accurate.
[0040] 4. Designed for medical scenarios and tailored to clinical needs: This method fully considers the dual requirements of accuracy and efficiency in medical image segmentation tasks. The technical design emphasizes the balance between lightweight and expressive power, and has good practicality and prospects for promotion. It is particularly suitable for image processing tasks with high requirements for boundary awareness, such as prostate MRI. Attached Figure Description
[0041] Fig. 1 This is the U-Net network structure in this embodiment of the invention;
[0042] Fig. 2 This is the multi-granularity attention module in this embodiment of the invention. Detailed Implementation
[0043] The following is in conjunction with the appendix Figs. 1-2 The present invention will be described in further detail below.
[0044] Example: A prostate image segmentation method with a multi-granularity self-attention mechanism
[0045] The data for this method contains N samples. Each sample consists of a pair of three-dimensional images, i.e., the original image x (i) and tag y (i) All data were divided into training, validation, and test sets in an 8:1:1 ratio. The training set was used to train the model, the validation set was used to select the best-performing model from the trained models, and the test set was used to evaluate the model's performance.
[0046] The prostate image segmentation method introduced a multi-granularity self-attention mechanism after the convolutional module of U-Net to establish a prostate image segmentation model, thereby enhancing feature representation capabilities. This method branches each convolutional feature map in two to reduce computational burden while capturing contextual information at different scales. The first branch focuses on fine-grained feature modeling, using the self-attention mechanism to calculate the relationship between each feature point in the image, emphasizing structural details within local regions, and reconstructing the original features accordingly. The second branch focuses on a lower-granularity but more globally perceptive modeling approach, capturing long-distance dependency information through the relationship between feature points and image regions, improving the model's understanding of the overall structure. Finally, the features reconstructed from the two different granularities are fused, effectively combining local details and global semantic information, thereby enhancing the prostate image model's expressiveness in complex prostate images.
[0047] The method specifically includes the following steps:
[0048] S1 Data Input: Read 3D images and labels, perform data augmentation, and convert them into 2D image format, which is then input into the prostate image segmentation model for calculation;
[0049] S11 Input data, input image data x of training set sample i. (i) and corresponding label y (i) ;
[0050] S12 data augmentation, for x (i) Perform spatial-based data augmentation, such as rotation, translation, and elastic transformation, and intensity-based spatial augmentation, such as noise addition; (i) The same spatial transformation is also performed accordingly;
[0051] S13 image sampling: Since the prostate image segmentation model in this method uses a 2D model, the input size requirement is: channel * length * width. Therefore, in x... (i) Randomly select a consecutive number of images, say c images, as 2D input, i.e., c × w × d, c channels; the corresponding y (i) Do the same treatment;
[0052] S14 will use the 2D image x from S13 (i) As input to the segmentation model, y (i) Then it is used as a label for loss calculation.
[0053] The S2 prostate image segmentation model processes and analyzes images: it inputs a 2D image, encodes and decodes it using the prostate image segmentation model, then uses a segmentation head to make predictions, and finally compares the predictions with the true labels.
[0054] S21 Initial Convolution: For the input image, let the size be c×w×d. Use the convolutional network to extract features from it, output features with size c0×w×d, and input the output features into the subsequent encoding network.
[0055] The S22 encoding network, a segmentation model, contains L encoding layers. Each encoding layer consists of a downsampling network, a convolutional network, and a multi-granularity self-attention module. The output features of the encoding layer are input to the next encoding layer and the corresponding decoding layer (skip connection).
[0056] S221 downsampling: For the input feature F, let the feature size be c. j ×w j ×d j First, the feature is processed through a downsampling network, which can be a max pooling, average pooling, or convolutional network, resulting in a feature size that is halved. Let it be c j ×w j+1 ×d j+1 This process represents a 2×2 region with a single feature point in order to compute features with a larger receptive field while also reducing the number of features.
[0057] S222 convolution uses a convolutional network to calculate the output features of S221 and extract features with higher order and larger receptive field than feature F in feature S221.
[0058] The S223 attention module takes the features from S222 and feeds them into a multi-granularity attention module. It calculates features at different fine-grained levels, fuses these features, and finally outputs a feature that is fed into the next encoding layer and the corresponding decoding layer. Here, different granularities represent different receptive fields. For example, the highest level of granularity focuses on the relationships between each feature point, achieving the highest level of detail or fine-grainedness. Conversely, reducing granularity focuses on the relationships between each feature point and each region. Reducing granularity increases the receptive field, resulting in better global information (see the attention calculation process description below). This process optimizes the output features of S222, increasing both detailed and global information.
[0059] S23 intermediate layer: The output features of the Lth coding layer are input into the intermediate layer. The intermediate layer has the same network result as the coding layer, that is, it first downsamples, then convolves, and finally calculates attention and outputs to the decoding network.
[0060] The S24 decoding network has L decoding layers. Its network structure is similar to that of the coding layer, namely upsampling, convolution and attention.
[0061] S241 upsampling: For the input feature F, let the feature size be c. j+1 ×w j+1 ×d j+1 First, upsample the image to gradually restore its size, resulting in a size of c. j+1 ×2w j+1 ×2d j+1 The characteristic, denoted as c j+1 ×w j ×d j ;
[0062] S242 feature concatenation involves concatenating the features from S241 with the corresponding size features from the coding layer. At this point, it should have c j ×w j ×d j The encoded features are concatenated along the channel dimension, meaning the size of the concatenated feature is (c j +c j+1 )×w j ×d j ;
[0063] The S243 decoding computation, based on the features of S242, first utilizes a convolutional network, then employs multi-granularity attention to perform computation, yielding the final decoded features, with size c. j ×w j ×d j This decoding feature will be input into the next decoding layer.
[0064] The S25 segmentation head upsamples the first decoded feature and concatenates it with the output feature of the initial convolution along the channel dimension to obtain a feature of size (c+c0)×w×d. The segmentation head, i.e., the convolutional network, then performs calculations on this feature to obtain the final prediction result. The dimensions are c×w×d;
[0065] S26 loss calculation, and real label y (i) The comparison is performed, the loss is calculated, and the parameters of the segmentation model are updated based on the loss.
[0066] S3 multi-granularity self-attention module analysis and processing: The convolutional features are divided into two parts, one part is used to calculate the relationship between each feature point and the other part is used to calculate the relationship between each feature point and each region.
[0067] S31 convolution: For the input feature F, let the size be c×w×d. First, use the convolutional network to calculate the output feature, with a size of k×w×d, where k is an even number.
[0068] Feature S32 partitioning divides the features obtained in S31 into two along the channel dimension, resulting in feature F1 with a size of [missing information]. And feature F2, the size is also The purpose of splitting the features into two parts is twofold: first, different features are used to calculate attention with different fine-grained values; second, calculating them separately instead of using the whole feature to calculate features of different granularities sequentially (for example, directly using the features obtained from S31 to calculate fine-grained attention first, and then calculating coarse-grained attention) will significantly reduce the amount of computation.
[0069] S33 fine-grained attention: For feature F1, three convolutional networks are first used to compute it separately, obtaining Q, K, and V; then matrix multiplication is performed using Q and K to obtain the relationship between each feature point. If the size of Q is also... There are a total of w*d feature points here, and the dimension of each feature point is... Similar to natural images, there are w*d pixels, each with 3 dimensions (RGB). Finally, using the relationships between feature points, a matrix multiplication is performed with V to reconstruct the features, outputting F'1, with dimensions [missing information]. Here, attention calculates the relationship between pairs of feature points, so it has the highest level of detail (pixel level).
[0070] S34 is a coarse-grained attention feature. Feature F2 is computed using a convolutional network to obtain Q, with a size of [missing information]. Then, average pooling is used to calculate feature F2, resulting in K1 and V1, with dimensions of... Here, average pooling is used so that subsequent attention calculations compute the relationship between each feature point and each p×p region, resulting in lower granularity compared to S33, meaning a larger receptive field. Note that K1 = V1 here, meaning they represent the same feature, further reducing computation. Next, based on Q and K1, the relationship between each feature point in Q and each p×p region in feature F2 (i.e., K1) is calculated. Finally, using the relationship between feature points and regions, a matrix multiplication is performed with V1 to reconstruct the feature, outputting feature O1. Specifically, this reconstruction involves re-representing the feature points. Since attention calculates the relationship between a feature point and all regions, this feature point is re-represented using features from all regions. To further increase the receptive field, average pooling is applied to K1 and V1 to obtain K2 and V2, with dimensions of [missing information]. Here, K2 = V2. The K2 and V2 regions are larger. Therefore, a similar method is used to calculate the output feature O2. Thus, attention features of different fine-grained levels are finally obtained.
[0071] S35 feature concatenation: The features F'1 of S33 and O1 and O2 of S34 are concatenated along the channel dimension. The concatenated features are then computed using a convolutional network to obtain the final output feature F', with dimensions c×w×d.
[0072] This specific embodiment is merely an explanation of the present invention and is not intended to limit the invention. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of the claims of the present invention.
Claims
1. A prostate image segmentation method with a multi-granularity self-attention mechanism, characterized in that: The prostate image segmentation method introduces a multi-granularity self-attention mechanism after the convolution module of U-Net to establish a prostate image segmentation model to enhance feature expression capabilities. This method performs branch processing on each set of convolutional feature maps, splitting them into two to reduce computational burden, while capturing contextual information at different scales. The first branch focuses on fine-grained feature modeling, using a self-attention mechanism to calculate the relationship between each feature point in the graph, emphasizing the structural details in local regions, and reconstructing the original features accordingly. The second branch focuses on a modeling approach with lower granularity but greater global awareness. By capturing long-distance dependency information through the relationship between feature points and image regions, it improves the model's understanding of the overall structure. Finally, the features reconstructed at two different granularities are fused to effectively combine local details with global semantic information, thereby enhancing the expressiveness of the prostate image model in complex prostate images.
2. The prostate image segmentation method with a multi-granularity self-attention mechanism according to claim 1, characterized in that: The method specifically includes the following steps: S1 Data Input: Read 3D images and labels, perform data augmentation, and convert them into 2D image format, which is then input into the prostate image segmentation model for calculation; The S2 prostate image segmentation model processes and analyzes images: it inputs a 2D image, encodes and decodes it using the prostate image segmentation model, then uses a segmentation head to make predictions, and finally compares the predictions with the true labels. S3 multi-granularity self-attention module analysis and processing: The convolutional features are divided into two parts, one part is used to calculate the relationship between each feature point and the other part is used to calculate the relationship between each feature point and each region.
3. The prostate image segmentation method with a multi-granularity self-attention mechanism according to claim 2, characterized in that: S1 specifically includes the following steps: S11 Input data, input image data x of training set sample i. (i) and corresponding label y (i) ; S12 data augmentation, for x (i) Perform spatial-based data augmentation; y (i) The same spatial transformation is also performed accordingly; S13 image sampling: Since the prostate image segmentation model in this method uses a 2D model, the input size requirement is: channel * length * width. Therefore, in x... (i) Randomly select a consecutive number of images, say c images, as 2D input, i.e., c × w × d, c channels; the corresponding y (i) Do the same treatment; S14 will use the 2D image x from S13 (i) As input to the segmentation model, y (i) Then it is used as a label for loss calculation.
4. The prostate image segmentation method with a multi-granularity self-attention mechanism according to claim 2, characterized in that: S2 specifically includes the following steps: S21 Initial Convolution: For the input image, let the size be c×w×d. Use the convolutional network to extract features from it, output features with size c0×w×d, and input the output features into the subsequent encoding network. The S22 encoding network, a segmentation model, contains L encoding layers. Each encoding layer consists of a downsampling network, a convolutional network, and a multi-granularity self-attention module. The output features of the encoding layer are input into the next encoding layer and the corresponding decoding layer. S23 intermediate layer: The output features of the Lth coding layer are input into the intermediate layer. The intermediate layer has the same network result as the coding layer, that is, it first downsamples, then convolves, and finally calculates attention and outputs to the decoding network. The S24 decoding network has L decoding layers. Its network structure is similar to that of the coding layer, namely upsampling, convolution and attention. The S25 segmentation head upsamples the first decoded feature and concatenates it with the output feature of the initial convolution along the channel dimension to obtain a feature of size (c+c0)×w×d. The segmentation head, i.e., the convolutional network, then performs calculations on this feature to obtain the final prediction result. The dimensions are c×w×d; S26 loss calculation, and real label y (i) The comparison is performed, the loss is calculated, and the parameters of the segmentation model are updated based on the loss.
5. The prostate image segmentation method with a multi-granularity self-attention mechanism according to claim 4, characterized in that: S22 specifically includes the following steps: S221 downsampling: For the input feature F, let the feature size be c. j ×w j ×d j First, the feature is processed through a downsampling network, which can be a max pooling, average pooling, or convolutional network, resulting in a feature size that is halved. Let it be c j ×w j+1 ×d j+1 ; S222 convolution uses a convolutional network to calculate the output features of S221 and extract features with higher order and larger receptive field than feature F in feature S221. S223 attention takes the features from S222 and inputs them into the multi-granularity attention module to calculate features of different fine granularities. It then fuses these features and outputs the final features, which are then input into the next encoding layer and the corresponding decoding layer.
6. The prostate image segmentation method with a multi-granularity self-attention mechanism according to claim 4, characterized in that: S24 specifically includes the following steps: S241 upsampling: For the input feature F, let the feature size be c. j+1 ×w j+1 ×d j+1 First, upsample the image to gradually restore its size, resulting in a size of c. j+1 ×2w j+1 ×2d j+1 The characteristic, denoted as c j+1 ×w j ×d j ; S242 feature concatenation involves concatenating the features from S241 with the corresponding size features from the coding layer. At this point, it should have c j ×w j ×d j The encoded features are concatenated along the channel dimension, meaning the size of the concatenated feature is (c j +c j+1 )×w j ×d j ; The S243 decoding computation, based on the features of S242, first utilizes a convolutional network, then employs multi-granularity attention to perform computation, yielding the final decoded features, with size c. j ×w j ×d j This decoding feature will be input into the next decoding layer.
7. A prostate image segmentation method with a multi-granularity self-attention mechanism according to claim 2, characterized in that: S3 specifically includes the following steps: S31 convolution: For the input feature F, let the size be c×w×d. First, use the convolutional network to calculate the output feature, with a size of k×w×d, where k is an even number. Feature S32 partitioning divides the features obtained in S31 into two along the channel dimension, resulting in feature F1 with a size of [missing information]. And feature F2, the size is also S33 fine-grained attention: For feature F1, three convolutional networks are first used to compute Q, K, and V respectively; then, matrix multiplication is performed using Q and K to obtain the relationship between each feature point; finally, using the relationship between feature points, matrix multiplication is performed with V to reconstruct the feature and output F'1, with a size of [missing information]. S34 is a coarse-grained attention feature. Feature F2 is computed using a convolutional network to obtain Q, with a size of [missing information]. Then, average pooling is used to calculate feature F2, resulting in K1 and V1, with dimensions of... K1 = V1. Based on Q and K1, calculate the relationship between each feature point in Q and each p×p region in feature F2 (i.e., K1). Finally, using the relationship between feature points and regions, perform matrix multiplication with V1 to reconstruct the features and output feature O1. Then, continue using average pooling based on K1 and V1 to obtain K2 and V2, with dimensions of... Here, K2 = V2, and the output feature O2 is calculated using the same method, ultimately yielding attention features of different fine granularities; S35 feature concatenation: The features F'1 of S33 and O1 and O2 of S34 are concatenated along the channel dimension. The concatenated features are then computed using a convolutional network to obtain the final output feature F', with dimensions c×w×d.