Handheld instrument fine-grained positioning method based on attention mechanism and self-distillation

Abstract

The invention discloses a handheld instrument fine-grained positioning method based on an attention mechanism and self-distillation, and the method solves the problems of insufficient multi-scale feature fusion, fuzzy instrument boundary and inaccurate interference area recognition in the prior art by introducing a feature fusion module, a dual attention module and a self-distillation mechanism. Specifically, the feature fusion module effectively fuses feature maps of different scales to obtain more accurate global and local information; the dual attention module further enhances the capturing ability of instrument features in a complex operation scene by combining channel attention and position attention; the precision of the positioning model is improved through cross-layer knowledge propagation by the self-distillation mechanism, and particularly, the positioning performance is remarkably optimized on the premise of keeping a lightweight model structure. The method not only improves the understanding ability of the model for the complex semantic relationship, but also makes significant progress in the aspects of detail positioning of the instrument and accurate identification of the interference area.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and relates to a fine-grained positioning method for a handheld device, and specifically relates to a fine-grained positioning method for a handheld device based on an attention mechanism and self-distillation. Background Art

[0002] Traditional device positioning methods usually rely on the grayscale features, edge features or regional features of the image to achieve the positioning of the device. However, with the development of deep learning and attention mechanisms, more and more studies have begun to combine visual information with contextual semantics to improve the accuracy of device positioning in complex environments. Although the existing fine-grained positioning technology based on deep learning and attention mechanisms has made some progress in capturing local visual features and global semantic features in complex scenes, due to the simplified design of the fusion method of semantic information and feature information, it is often easy to ignore some detailed features, affecting the positioning accuracy of the device. Therefore, how to effectively fuse multi-scale features and dual attention mechanisms under the premise of lightweight models to improve positioning accuracy in complex scenes is still a challenge that needs to be solved urgently. Summary of the invention

[0003] In order to address the problems of insufficient feature fusion and insufficient positioning accuracy in the existing fine-grained positioning technology of instruments in complex environments due to complex scenes, the present invention provides a fine-grained positioning method for handheld instruments based on attention mechanism and self-distillation. This method significantly improves the accuracy of fine-grained positioning through feature map fusion module, dual attention module and self-distillation module, solves the problems of incomplete fusion of semantic information and detail information and high computational overhead in the existing technology, can accurately locate targets in complex scenes, and demonstrates excellent performance on handheld instrument positioning datasets.

[0004] The objective of the present invention is achieved through the following technical solutions:

[0005] A fine-grained localization method for handheld devices based on attention mechanism and self-distillation includes the following steps:

[0006] Step 1: Feature extraction and preliminary processing of input images containing handheld devices:

[0007] Given an input image I∈R (H×W×3) , the encoder extracts features from the input image and generates feature maps E1, E2, E3, E4, and E5 of different sizes, with resolutions of H / 2×W / 2, H / 4×W / 4, H / 8×W / 8, H / 16×W / 16, and H / 32×W / 32, respectively, and the number of channels is 16, 24, 32, 96, and 1280, respectively, where H and W represent the height and width of the input image, respectively;

[0008] Step 2: Fusion extraction of semantic and detail information in feature fusion module:

[0009] The feature fusion module uses mid-level features as an intermediary to fuse the semantic information of high-level features with the detailed information of low-level features to generate unified output features. The specific steps are as follows:

[0010] Step 2.1, feature extraction:

[0011] For different levels of feature maps from the encoder, the low-level feature map is defined as f i-1 , the middle-level feature map is f i , the high-level feature map is f i+1 ;

[0012] Step 2.2: Fusion of low-level and high-level features:

[0013] Step 2.2.1, low-level feature fusion:

[0014] For the low-level feature map f i-1 Use a convolution operation with a kernel size of 2 and a stride of 2 to reduce its resolution to half of the original resolution to match the resolution of the mid-level feature map. Adjust the number of channels through a 1×1 convolution and multiply it element-by-element with the mid-level feature map to generate a fused low-level feature map.

[0015] Step 2.2.2, high-level feature fusion:

[0016] For the high-level feature map f i+1 Use bilinear interpolation upsampling to double the resolution, reduce the number of channels through 1×1 convolution, and multiply the elements with the mid-level feature map to generate a fused high-level feature map;

[0017] Step 2.3: Channel fusion and output of feature map:

[0018] The fused high-level features and low-level features are concatenated in the channel dimension to generate a fused feature map, and the number of channels of the fused feature map is adjusted to 48 through a 1×1 convolution;

[0019] Step 3: Feature optimization of dual attention module:

[0020] Step 3.1. Design a dual attention module:

[0021] The dual attention module consists of a position attention module, a channel attention module, and a fusion calibration module;

[0022] Step 3.2: Implementation of the position attention module:

[0023] The position attention module receives a feature map of size C×H×W and outputs a position attention map of size 1×H×W. The position attention module approximates the low-rank bilinear pooling method for feature extraction. After the fused features are added and pooled, the number of channels becomes 1, becoming a position attention map of size 1×H×W. A 1×1 convolution is performed on the position attention map to enhance the feature representation capability. The specific calculation process is:

[0024]

[0025] Among them, A p Represents the position attention output; Sum represents sum pooling, which is used to control the output size; X and Y represent the input feature map; Conv 1×1 represents 1×1 convolution; Softmax represents the Softmax activation function;

[0026] Step 3.3, implementation of channel attention module:

[0027] The channel attention module receives a feature map of size C×H×W and outputs a channel attention vector of size C×1×1. The channel attention module uses the global average pooling (GAP) operation to aggregate global information into an attention vector. After global average pooling, two 1×1 convolutions with batch normalization are used to reduce and then increase the number of channels of the channel attention vector to refine the semantic relationship. The specific calculation process is:

[0028] A c =Conv 1×1 (Conv1×1(GAP(X)))

[0029] Among them, A c represents channel attention output; GAP represents global average pooling;

[0030] Step 3.4: Implementation of the fusion calibration module:

[0031] The Hadamard product with broadcasting is used to fuse the position attention map with the channel attention vector. After the fusion is completed, the feature map is calibrated. The original input and the attention feature are used to perform the Hadamard product. The product result is residually connected with the original input feature map to obtain the optimized feature map of the dual attention module. The specific calculation process is:

[0032]

[0033] Among them, f DAM represents the dual attention module output; ⊕ represents element-wise addition; Represents the calibrated attention feature map; stands for Hadamard product; A srepresents the attention feature map; ⊙ represents the Hadamard product with broadcasting; k represents the feature map dimension, k=1,2,...,C; h represents the feature map height, h=1,2,...,H; w represents the feature map width, w=1,2,...,W;

[0034] Step 4: Self-distillation module improves positioning effect:

[0035] The self-distillation module uses the output of the deeper layer as the target and the output of the shallower layer as the input. It generates a unified probability map by adjusting the resolution and number of channels of the feature map, evaluates the similarity between the two, and achieves efficient use of features and significant improvement in positioning performance by optimizing the loss during training. The specific steps are as follows:

[0036] Step 4.1, probability map generation:

[0037] The output of the dual attention module and the decoder is adjusted to the number of positioning categories through 1×1 convolution, and the Softmax activation function is used to generate a probability map with the number of channels equal to the number of positioning categories. The specific calculation process is:

[0038] Ψ(A n )=softmax(AvgPool(Conv 1×1 (A n )))

[0039] Φ(A n )=Softmax(Conv 1×1 (A n ))

[0040] Among them, Ψ(A n ) is the distillation target, Φ(A n ) is the distillation input; A n Represents the nth layer involved in distillation; Softmax represents the Softmax activation function; AvgPool represents average pooling;

[0041] Step 4.2, implementation of distillation operation:

[0042] In a single distillation, the output of the deeper layer is selected as the distillation target, and the output of the shallower layer is used as the distillation input. Average pooling is used to reduce the resolution of the probability map of the deeper layer to the same level as the shallower layer. Figure 1 Consistently, a loss function is used to evaluate the similarity between two probability maps, and the distillation loss is minimized during training;

[0043] Step 5: Positioning model optimization:

[0044] The localization model consists of a feature map fusion module, a dual attention module, and a self-distillation module. The localization model optimization is achieved by calculating the localization loss and the distillation loss. The localization loss Lseg Using focus loss, distillation loss L distill Using L2 loss, the total loss function is:

[0045] L=λ seg L seg +λ distill L distill

[0046] Among them, λ seg and λ distill is the weight.

[0047] Compared with the prior art, the present invention has the following advantages:

[0048] 1. The present invention realizes accurate positioning of instruments in complex environments through the collaborative work of feature map fusion module, dual attention module and self-distillation module, and significantly improves the model's ability to capture detail information and context information. The feature map fusion module fuses the semantic information of high-level feature maps with the detail information of low-level feature maps, effectively solving the problem of inconsistent information between high-level and low-level features, and enhancing the ability to locate complex boundaries; the dual attention module combines position attention and channel attention to help the model focus on more meaningful features; the self-distillation module transfers knowledge between different layers of the model, without adding additional modules, to improve positioning accuracy, thereby improving model performance without increasing computational overhead.

[0049] 2. The present invention has significant advantages in terms of computational overhead and parameter quantity. Compared with previous fine-grained positioning methods, it has taken the lead in multiple indicators and maintained extremely low computing resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flowchart of the fine-grained positioning method of handheld devices based on attention mechanism and self-distillation of the present invention;

[0051] Figure 2 It is a structural diagram of the feature fusion module of the present invention;

[0052] Figure 3 It is a structural diagram of the dual attention module of the present invention;

[0053] Figure 4 This is a structural diagram of the self-distillation module of the present invention. DETAILED DESCRIPTION

[0054] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0055] The present invention provides a method for fine-grained positioning of handheld instruments based on an attention mechanism and self-distillation. The method solves the problems of insufficient multi-scale feature fusion, blurred instrument boundaries, and inaccurate identification of interference areas in the prior art by introducing a feature fusion module, a dual attention module, and a self-distillation mechanism. Specifically, the feature fusion module effectively fuses feature maps of different scales to obtain more accurate global and local information; the dual attention module further enhances the ability to capture instrument features in complex surgical scenes by combining channel attention with position attention; the self-distillation mechanism improves the accuracy of the positioning model through cross-layer knowledge propagation, especially while maintaining a lightweight model structure, significantly optimizing the positioning performance. Through this overall design, the present invention not only improves the model's ability to understand complex semantic relationships, but also makes significant progress in the detailed positioning of instruments and the accurate identification of interference areas. Figure 1 As shown, the specific steps include:

[0056] Step 1: Feature extraction and preliminary processing of input images containing handheld devices:

[0057] Given an input image I∈R (H×W×3) , the encoder extracts features from the input image and generates feature maps E1, E2, E3, E4, and E5 of different sizes, with resolutions of H / 2×W / 2, H / 4×W / 4, H / 8×W / 8, H / 16×W / 16, and H / 32×W / 32, respectively, and the number of channels is 16, 24, 32, 96, and 1280, respectively, where H and W represent the height and width of the input image, respectively;

[0058] Step 2: Fusion extraction of semantic and detail information in feature fusion module:

[0059] The feature fusion module uses mid-level features as an intermediary to fuse the semantic information of high-level features with the detailed information of low-level features to generate unified output features. By adjusting the resolution and number of channels of features at different levels, they are kept consistent during the fusion process, and by splicing and integrating features, the advantages of features at each level are fully utilized, thereby improving feature expression capabilities and positioning effects. The specific steps are as follows:

[0060] Step 2.1: Feature extraction of feature fusion module:

[0061] For different levels of feature maps from the encoder, the low-level feature map is defined as f i-1 , the middle-level feature map is f i , the high-level feature map is f i+1Since the high-level features and low-level features are far apart in the encoder output position, the scales of the feature maps are inconsistent, and the semantic information and detail information they contain are very different, direct feature fusion may cause conflicts and cause the information contained in the two to interfere with each other, affecting the positioning effect. The middle-level feature map is used as the intermediary in the feature fusion stage, and the high-level feature map and the low-level feature map are fused with the middle-level feature map respectively. i Use 1×1 convolution to adjust the number of channels to 48 to ensure that the number of channels remains consistent during subsequent fusion.

[0062] Step 2.2: Fusion of low-level and high-level features:

[0063] Step 2.2.1, low-level feature fusion:

[0064] Low-level feature map f i-1 Compared with the mid-level feature map f i The scale is larger and the number of channels is smaller, such as Figure 2 As shown in the figure, a convolution operation with a convolution kernel size of 2 and a stride of 2 is used to reduce its resolution to half of the original resolution, which is consistent with the resolution of the mid-level feature map. The number of channels is adjusted through 1×1 convolution, and the fused low-level feature map is generated by element-by-element multiplication with the mid-level feature map.

[0065] Step 2.2.2, high-level feature fusion:

[0066] High-level feature map f i+1 Compared with the mid-level feature map f i The scale is smaller and the number of channels is larger, such as Figure 2 As shown in the figure, bilinear interpolation upsampling is used to double the resolution, 1×1 convolution is used to reduce the number of channels, and the fused high-level feature map is generated by element-by-element multiplication with the mid-level feature map.

[0067] Step 2.3: Channel fusion and output of feature map:

[0068] like Figure 2 As shown in the figure, the fused high-level features and low-level features are concatenated in the channel dimension to generate a fused feature map. The channels of the feature map are adjusted using 1×1 convolution to a uniform value of 48 as the output of the feature fusion module.

[0069] Step 3: Feature optimization of dual attention module:

[0070] Step 3.1. Design a dual attention module:

[0071] The present invention designs a dual attention module to screen the features in the fused feature map, introduces an attention mechanism, and adjusts the weights of different features to allow the model to focus on more meaningful features and improve the positioning effect. Figure 3 As shown, the dual attention module consists of a position attention module, a channel attention module and a fusion calibration module, wherein:

[0072] The position attention module assigns weights to different positions in the feature map, allowing the model to focus on key areas and improve positioning accuracy. The input feature map size is C×H×W, and the output is a position attention map of 1×H×W. The module uses low-rank bilinear pooling to extract global semantic information, effectively reducing computational overhead and modeling semantic relationships between pixels.

[0073] The channel attention module captures the semantic dependencies between channels, strengthens effective features, and suppresses useless features. The input feature map size is C×H×W, and the output is a channel attention vector of C×1×1. The module uses global average pooling to aggregate global information, and refines the channel relationship through two 1×1 convolutions to enhance the feature expression capability.

[0074] The fusion calibration module combines the position attention map and the channel attention vector to optimize the feature map. The position and channel attention are fused through the Hadamard product, and then calibrated and residually connected with the input feature map to avoid gradient problems and improve model performance, and finally output the optimized feature map.

[0075] Step 3.2: Implementation of the position attention module:

[0076] The position attention module assigns weights to different positions in the feature map, helping the model pay more attention to the meaningful parts of the image, thereby improving the accuracy of positioning. The position attention module receives a feature map of size C×H×W and outputs a position attention map of size 1×H×W. The position attention module approximates the low-rank bilinear pooling method for feature extraction, introduces matrix decomposition and Hadamard product to avoid the extremely computationally expensive matrix multiplication, significantly reducing the computational cost. After the fused features are added and pooled, the number of channels becomes 1, becoming a position attention map of size 1×H×W, which can retain global feature information while reducing the size of the feature map. The position attention map contains the semantic relationship between pixels and can better model complex semantic situations. Finally, a 1×1 convolution is performed on the position attention map to enhance the feature representation capability. The specific calculation process is:

[0077]

[0078] Among them, A pRepresents the position attention output; Sum represents sum pooling, which is used to control the output size; X and Y represent the input feature map; Conv 1×1 represents 1×1 convolution; Softmax represents the Softmax activation function.

[0079] Step 3.3, implementation of channel attention module:

[0080] By utilizing the semantic dependencies between channels, the model can strengthen effective semantic features and suppress useless semantic features. The channel attention module receives a feature map of size C×H×W and outputs a channel attention vector of size C×1×1. The channel attention module uses a global average pooling (GAP) operation to aggregate global information into an attention vector. Each element in the attention vector contains the global information of the channel, which helps to expand the receptive field. After global average pooling, two 1×1 convolutions with batch normalization are used to reduce and then increase the number of channels of the channel attention vector to refine the semantic relationship. The specific calculation process is:

[0081] A c =Conv 1×1 (Conv 1×1 (GAP(X)))

[0082] Among them, A c represents channel attention output; GAP represents global average pooling; X represents the original feature map of the input.

[0083] Step 3.4: Implementation of the fusion calibration module:

[0084] The fusion calibration module fuses and calibrates the position attention map of size 1×H×W and the channel attention vector of size X×1×1. The Hadamard product with broadcasting is used to fuse the position attention map with the channel attention vector. After the fusion is completed, the feature map is calibrated, and the Hadamard product is used with the original input and the attention feature. The product result is residually connected with the original input feature map to avoid the gradient vanishing and gradient exploding problems, accelerate training and improve the performance of the model, and obtain the optimized feature map of the dual attention module. The specific calculation process is:

[0085]

[0086] Among them, f DAM represents the dual attention module output; ⊕ represents element-wise addition; Represents the calibrated attention feature map; stands for Hadamard product; A srepresents the attention feature map; ⊙ represents the Hadamard product with broadcasting; k represents the feature map dimension, k=1,2,...,C; h represents the feature map height, h=1,2,...,H; w represents the feature map width, w=1,2,...,W; X represents the original feature map of the input.

[0087] Step 4: Self-distillation module improves positioning effect:

[0088] The present invention designs a self-distillation module for making full use of the features extracted by the network to improve the performance of the network without adding additional modules. The feature map of the current layer is extracted and passed as knowledge to the next layer for learning, and the performance of the model is improved through multiple knowledge transfers. Specifically, the module uses the output of the deeper layer as the target and the output of the shallower layer as the input, and generates a unified probability map by adjusting the resolution and number of channels of the feature map to evaluate the similarity between the two. By optimizing the loss during the training process, efficient use of features and significant improvement in positioning performance are achieved without adding additional modules.

[0089] Step 4.1, probability map generation:

[0090] The outputs of the dual attention module and the decoder are distilled. The deep output of the model contains richer semantic information. The output of the deeper layer is used as the distillation target, and the output of the shallower layer is used as the distillation input. Since the resolution of the deep probability map is higher than that of the shallow probability map, average pooling is used before distillation to adjust the resolution of the two to be consistent. The number of channels is adjusted to the number of positioning categories through 1×1 convolution, and the Softmax activation function is used to generate a probability map with the number of channels equal to the number of positioning categories. The specific calculation process is:

[0091] Ψ(A n )=Softmax(AvgPool(Conv 1×1 (A n )))

[0092] Φ(A n )=Softmax(Conv 1×1 (A n ))

[0093] Among them, Ψ(A n ) is the distillation target, Φ(A n ) is the distillation input; A n Represents the nth layer involved in distillation; Softmax represents the Softmax activation function; AvgPool represents average pooling.

[0094] Step 4.2, implementation of distillation operation:

[0095] In a single distillation, the output of the deeper layer is selected as the distillation target, and the output of the shallower layer is selected as the distillation input. Average pooling is used to reduce the resolution of the probability map of the deeper layer to the same level as the shallower layer. Figure 1 The loss function is used to evaluate the similarity between two probability maps, and the distillation loss is minimized during training.

[0096] Step 5: Positioning model optimization:

[0097] The localization model consists of a feature map fusion module, a dual attention module, and a self-distillation module. The optimization of the localization model is achieved by calculating the localization loss and the distillation loss. The localization loss L seg Using focus loss, distillation loss L distill Using L2 loss, the total loss function is:

[0098] L=λ seg L seg +λ distill L distill

[0099] Among them, λ seg and λ distill For weights:

[0100] Example:

[0101] This embodiment performs performance testing on a handheld device positioning dataset. The present invention is implemented in Pytorch and runs on hardware equipped with an Intel(R) 10700k CPU and an Nvidia 3090 GPU. The model input is uniformly adjusted to 640×384. The entire model is trained for a total of 100 epochs using the ADAM optimizer with an initial learning rate of 3×10 -4 , every 5 iterations, the learning rate is multiplied by 0.98, and the weighted coefficients of the loss function are set to λ seg =1 and λ distill =0.02.

[0102] In the performance testing phase, the present invention was tested on the handheld device positioning dataset and compared with other methods. Table 1 shows the performance comparison between the present invention and the existing positioning methods. The present invention surpasses other methods in terms of success rate indicators and can achieve the best performance with the least number of parameters and less computational overhead. The mIoU reached 0.612, mPa reached 0.665, and mDice reached 0.705. Compared with the second-ranked model, the three indicators were improved by 0.056, 0.042, and 0.030 respectively.

[0103] Table 1 Comparison of various indicators of different models

[0104]

[0105] In summary, the experimental results of the present invention on the handheld device positioning dataset show that it is superior to the existing technology in terms of positioning effect, fully demonstrating the superior performance and wide applicability of the attention mechanism and self-distillation method.

Claims

1. A fine-grained localization method for handheld devices based on attention mechanism and self-distillation, characterized in that The method comprises the following steps: Step 1: Feature extraction and preliminary processing of input images containing handheld devices: Given an input image I∈R (H×W×3) , the encoder extracts features from the input image and generates feature maps E1, E2, E3, E4, and E5 of different sizes, with resolutions of H / 2×W / 2, H / 4×W / 4, H / 8×W / 8, H / 16×W / 16, and H / 32×W / 32, respectively, and the number of channels is 16, 24, 32, 96, and 1280, respectively, where H and W represent the height and width of the input image, respectively; Step 2: Fusion extraction of semantic and detail information in feature fusion module: The feature fusion module uses mid-level features as an intermediary to fuse the semantic information of high-level features with the detailed information of low-level features to generate a unified output feature; Step 3: Feature optimization of dual attention module: Step 3.

1. Design a dual attention module: The dual attention module consists of a position attention module, a channel attention module, and a fusion calibration module; Step 3.2: Implementation of the position attention module: The position attention module receives a feature map of size C×H×W and outputs a position attention map of size 1×H×W. The position attention module approximates the low-rank bilinear pooling method for feature extraction. After the fused features are added and pooled, the number of channels becomes 1, becoming a position attention map of size 1×H×W. A 1×1 convolution is performed on the position attention map to enhance the feature representation capability. The specific calculation process is: Among them, A p Represents the position attention output; Sum represents sum pooling, which is used to control the output size; X and Y represent the input feature map; Conv 1×1 represents 1×1 convolution; Softmax represents the Softmax activation function; Step 3.3, implementation of channel attention module: The channel attention module receives a feature map of size C×H×W and outputs a channel attention vector of size C×1×1. The channel attention module uses a global average pooling operation to aggregate global information into an attention vector. After global average pooling, two 1×1 convolutions with batch normalization are used to reduce and then increase the number of channels of the channel attention vector to refine the semantic relationship. The specific calculation process is: A c =Conv 1×1 (Conv 1×1 (GAP(X))) Among them, A c represents channel attention output; GAP represents global average pooling; Step 3.4: Implementation of the fusion calibration module: The Hadamard product with broadcasting is used to fuse the position attention map with the channel attention vector. After the fusion is completed, the feature map is calibrated. The original input and the attention feature are used to perform the Hadamard product. The product result is residually connected with the original input feature map to obtain the optimized feature map of the dual attention module. The specific calculation process is: Among them, f DAM Represents the dual attention module output; stands for element-wise addition; Represents the calibrated attention feature map; stands for Hadamard product; A s represents the attention feature map; ⊙ represents the Hadamard product with broadcasting; k represents the feature map dimension, k=1,2,...,X; h represents the feature map height, h=1,2,...,H; w represents the feature map width, w=1,2,...,W; Step 4: Self-distillation module improves positioning effect: The self-distillation module uses the output of the deeper layer as the target and the output of the shallower layer as the input. It generates a unified probability map by adjusting the resolution and number of channels of the feature map, evaluates the similarity between the two, and achieves efficient use of features and significant improvement in positioning performance by optimizing the loss during training. Step 5: Positioning model optimization: The localization model consists of a feature map fusion module, a dual attention module, and a self-distillation module. The localization model optimization is achieved by calculating the localization loss and the distillation loss. The localization loss L seg Using focus loss, distillation loss L distill Using L2 loss, the total loss function is: L=λ seg L seg +λ distill L distill Among them, λ seg and λ distill is the weight.

2. The method for fine-grained localization of handheld devices based on attention mechanism and self-distillation according to claim 1 is characterized in that The specific steps of step 2 are as follows: Step 2.1, feature extraction: for Feature maps from different levels of the encoder, defining the low-level feature map as f i-1 , the middle-level feature map is f i , the high-level feature map is f i+1 ; Step 2.2: Fusion of low-level and high-level features: Step 2.2.1, low-level feature fusion: For the low-level feature map f i-1 Use a convolution operation with a kernel size of 2 and a stride of 2 to reduce its resolution to half of the original resolution to match the resolution of the mid-level feature map. Adjust the number of channels through a 1×1 convolution and multiply it element-by-element with the mid-level feature map to generate a fused low-level feature map. Step 2.2.2, high-level feature fusion: For the high-level feature map f i+1 Use bilinear interpolation upsampling to double the resolution, reduce the number of channels through 1×1 convolution, and multiply the elements with the mid-level feature map to generate a fused high-level feature map; Step 2.3: Channel fusion and output of feature map: The fused high-level features and low-level features are concatenated in the channel dimension to generate a fused feature map, and the number of channels of the fused feature map is adjusted to 48 through a 1×1 convolution.

3. The handheld device fine-grained localization method based on attention mechanism and self-distillation according to claim 1 is characterized in that The specific steps of step 4 are as follows: Step 4.1, probability map generation: The output of the dual attention module and the decoder is adjusted to the number of positioning categories through 1×1 convolution, and the Softmax activation function is used to generate a probability map with the number of channels equal to the number of positioning categories. The specific calculation process is: Ψ(A n )=Softmax(AvgPool(Conv 1×1 (A n ))) Φ(A n )=Softmax(Conv 1×1 (A n )) Among them, Ψ(A n ) is the distillation target, Φ(A n ) is the distillation input; A n Represents the nth layer involved in distillation; Softmax represents the Softmax activation function; AvgPool represents average pooling; Step 4.2, implementation of distillation operation: In a single distillation, the output of the deeper layer is selected as the distillation target, and the output of the shallower layer is selected as the distillation input. Average pooling is used to reduce the resolution of the probability map of the deeper layer to be consistent with the shallower probability map. The loss function is used to evaluate the similarity between the two probability maps, and the distillation loss is minimized during training.

Images