Metric-based medical endoscopic image classification algorithm incorporating subspace attention

Abstract

The application discloses a kind of medical endoscope image classification algorithm based on metric combined with subspace attention, for solving the problem that a large amount of noise exists in endoscope image in endoscope imaging process due to various complex factors, the noise signal contained in the image features extracted by convolutional neural network disturbs the spatial distribution of effective semantic features, weakens the representation ability of model to key semantic features, leading to the decline of classification accuracy. Including: 1) training model using metric-based few-shot learning method;2) the features extracted from the high layer of network structure are divided into multiple feature subspaces;3) the attention distribution and global correlation are calculated for each feature subspace respectively, to suppress the noise in the subspace and highlight the effective semantic area expressed. The image classification method proposed in the application can make the model pay attention to different key semantic parts in the endoscope image, suppress the interfering features, enhance the expression ability of effective semantic features, thereby improve the classification accuracy of the model.

Description

Technical Field

[0001] This invention relates to the field of computer-aided diagnosis based on medical images, and more specifically to a metric-based medical endoscopic image classification algorithm that integrates subspace attention. Background Technology

[0002] Early detection and diagnosis are crucial for controlling gastrointestinal diseases and improving survival rates. Medical endoscopic screening is an effective means of detecting early lesions in the digestive tract, allowing for precise identification of the lesion site. During digestive endoscopy, machine learning-based assisted diagnostic systems must accurately extract endoscopic image information, analyze and identify this information, and provide strong diagnostic support to doctors.

[0003] During microscopic imaging, switching light sources causes uneven image brightness and abnormal color changes; low imaging resolution and endoscope movement cause varying degrees of blurring; digestive tract mucus and residue, as well as strong light reflection, contribute to significant noise in the resulting endoscopic images. Machine learning methods based on deep neural networks typically use convolutional neural networks to extract image features for classification; therefore, the extracted endoscopic image features also contain considerable noise. The presence of these interfering signals inevitably disrupts the spatial distribution of semantic features, making it difficult for the model to accurately capture key semantic regions in the image, thus weakening the model's semantic representation ability and affecting classification performance. Summary of the Invention

[0004] Purpose of the invention: To address the above-mentioned problems, this invention proposes a metric-based medical endoscopic image classification algorithm that incorporates a subspace attention mechanism.

[0005] Technical Solution: A subspace attention mechanism is introduced in the last stage of the high-level ResNet network structure to highlight the key semantic feature regions expressed by the subspace. This allows the model to focus on effective feature information, suppress interfering features, and acquire more expressive semantic information. The implementation steps are as follows:

[0006] Step 1: Build a large-scale C-way K-shot few-shot classification task by upsampling from a medical endoscopic image set. Specifically, each time, C classes are randomly selected from the endoscopic image set, and K images are randomly sampled from each selected class to form the support set I. s Then, randomly sample any number of images from the remaining images in these C classes to form a query set I. q It supports combining the set and query set of images into a few-shot classification task T. i ={(I s ,I q ),(y s ,y q )}.

[0007] Step 2: Use the ResNet network structure with the last fully connected layer removed as the feature extraction network f θ The C-way K-shot task T created in step 1 i The input feature extraction network has a lot of noise in the lower layers of the neural network, while the higher layers have stronger semantic and positional information. Therefore, a subspace attention mechanism is used in the last stage of the high-level feature extraction network structure to enhance the model's semantic feature learning ability.

[0008] Specifically, the feature F output by the last BatchNorm layer in each basic block or bottleneck of the last stage of the feature extraction network structure is divided into g subspaces along the channel dimension [F]. 1 ,…,F n ,…,F g ], for each feature subspace F n Calculate the two attention matrices:

[0009] ①attn(F n )=σ(bn(gap(F n )⊙F n )),

[0010] Where σ represents the sigmoid function, bn represents the batch normalization layer, gap represents the global average pooling function, ⊙ represents the dot product operation, and attn(F n () represents an attention matrix generated using the similarity between global and local feature descriptors of a subspace, and a new feature subspace to which the attention mechanism is applied. The spatial distribution of internal features was enhanced, interfering features were suppressed, and key semantic feature regions within the subspace were highlighted, thereby improving the semantic feature learning ability of the feature subspace.

[0011] ②attn(F n )=σ(pconv(mp(dconv(F n )))),

[0012] Where σ represents the sigmoid function, dconv represents a depthwise convolution operation with a 1×1 kernel, mp represents a max pooling function with a 3×3 window size and a stride of 1, pconv represents a dotted convolution operation with a 1×1 kernel, and ⊙ represents a dotted product operation. This represents pixel-wise addition, with the attention matrix attn(F) nThis represents a new feature subspace that incorporates global relevance captured across channels, integrating semantic and spatial information, and for which an attention mechanism has been applied. The distribution of internal features has been improved, highlighting local semantic regions.

[0013] Step 3: Merge the 2g new feature subspaces obtained in Step 2:

[0014]

[0015]

[0016] F new =conv1*1(concat(F new_1 ,F new_2 ))

[0017] Here, `concat` represents concatenation by channel, and `conv1*1` represents a 1×1 convolution operation used to fuse data from different channels and reduce dimensionality. Each basic block or bottleneck ultimately outputs a feature F enhanced by a subspace attention mechanism. new .

[0018] Step 4: Extract features from the feature extraction network f that incorporates a subspace attention mechanism. θ The support set features Z are separated from the final output image features Z. s and query set features Z q Calculate the prototype of the feature class of the support set:

[0019]

[0020] Step 5: Measure the distance between the query set features and each class prototype, and predict the probability of the lesion category to which the query set image belongs based on the distance metric:

[0021]

[0022] Where d() represents the Euclidean distance.

[0023] Step 6: Calculate the cross-entropy loss between the predicted query set class and its actual class:

[0024]

[0025] Optimize the model network parameters through backpropagation.

[0026] The beneficial effects of this invention are as follows: ① Dividing the high-level convolutional features of the model network structure into multiple feature subspaces and generating an attention matrix separately for each subspace allows the model to focus on multiple different key semantic regions in the image; ② By learning cross-channel information and the similarity between global and local features, two attention matrices are generated for each feature subspace to improve the spatial distribution of features, suppress interfering features, and highlight key semantic feature regions; ③ The proposed attention mechanism can enhance the semantic expressive ability of the model without adding additional parameters and with low computational cost. It can be embedded into other feature extraction networks to improve feature learning and increase classification accuracy. Attached Figure Description

[0027] Figure 1 The flowchart and subspace attention mechanism are shown in the embodiments of the present invention; Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] This invention proposes a metric-based medical endoscopic image classification algorithm that incorporates a subspace attention mechanism, embedding a subspace attention module into the feature extraction network. A detailed explanation with specific implementation examples follows:

[0030] This invention employs a ResNet network structure with the last fully connected layer removed as the feature extraction network to project images into the embedding space and obtain image features. The ResNet network structure is a deep residual network, divided into 5 stages from the bottom to the top. Except for stage 0, the last 4 stages are composed of basic blocks or bottlenecks, and their structures are quite similar.

[0031] Figure 1 The diagram shows a flowchart and subspace attention module illustrating a specific implementation example of the present invention using the ResNet-50 network structure as an example. Stages 1 to 4 of ResNet-50 contain 3, 4, 6, and 3 bottlenecks, respectively.

[0032] Step 1: Create a C-way K-shot task T i Input feature extraction network;

[0033] Step 2: Divide the feature F output by the last BatchNorm layer in each bottleneck of the highest stage 4 into g subspaces along the channel dimension [F 1 ,…,F n ,…,F g ], for each feature subspace F n Calculate the two attention matrices:

[0034] ①attn(F n )=σ(bn(gap(F n )⊙F n )),

[0035] Where σ represents the sigmoid function, bn represents the batch normalization layer, gap represents the global average pooling function, ⊙ represents the dot product operation, and attn(F n ) represents the first attention matrix generated, resulting in a new feature subspace with the attention mechanism applied.

[0036] ②attn(F n )=σ(pconv(mp(dconv(F n )))),

[0037] Where σ represents the sigmoid function, dconv represents a depthwise convolution operation with a 1×1 kernel, mp represents a max pooling function with a 3×3 window size and a stride of 1, pconv represents a dotted convolution operation with a 1×1 kernel, and ⊙ represents a dotted product operation. This represents pixel-by-pixel addition, attn(F) n ) represents the generated second attention matrix, resulting in a new feature subspace with the attention mechanism applied.

[0038] Step 3: Merge the 2g new feature subspaces obtained in Step 2:

[0039]

[0040]

[0041] F new =conv1*1(concat(F new_1 ,F new_2 ))

[0042] Here, `concat` represents concatenation by channel, and `conv1*1` represents a 1×1 convolution operation used to fuse data from different channels and reduce dimensionality. Each basic block or bottleneck ultimately outputs a feature F enhanced by a subspace attention mechanism. new .

[0043] Step 4: Extract features from the feature extraction network f that incorporates a subspace attention mechanism. θ The support set features Z are separated from the final output image features Z. s and query set features Z q Calculate the prototype of the feature class of the support set:

[0044]

[0045] Step 5: Measure the distance between the query set features and each class prototype, and predict the probability of the lesion category to which the query set image belongs based on the distance metric:

[0046]

[0047] Where d() represents the Euclidean distance.

[0048] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A metric-based medical endoscopic image classification algorithm incorporating subspace attention, characterized in that, Includes the following steps: Step 1: Upsample a large number of C-way K-shots from a medical endoscopic image set to establish a few-shot classification task; Step 2: Input one of the C-way K-shot tasks from Step 1 into the feature extraction network f θ In the last stage of the high-level feature extraction network structure, a subspace attention mechanism is used to enhance the model's semantic feature learning ability. Step 3: Fuse the 2g new feature subspaces obtained in Step 2 to obtain semantically enhanced features; Step 4: Extract features from the feature extraction network f that incorporates a subspace attention mechanism. θ The support set features Z are separated from the final output image features Z. s and query set features Z q Calculate the prototype of the feature class of the support set: Step 5: Measure the distance between the query set features and each class prototype, and predict the probability of the lesion category to which the query set image belongs based on the distance metric: Where d() represents the Euclidean distance; Step 6: Calculate the cross-entropy loss between the predicted query set class and its actual class: Optimize the model network parameters through backpropagation.

2. The method for establishing a few-sample classification task according to step 1 of claim 1, characterized in that, Each few-shot classification task includes support set images and query set images: T i ={(I s ,I q ),(y s ,y q )}.

3. The method for generating subspace attention in step 2 according to claim 1, characterized in that, The feature F output by the last BatchNorm layer in each basic block or bottleneck of the last stage of the feature extraction network structure is divided into g subspaces according to the channel dimension [F]. 1 ,…,F n ,…,F g ], for each feature subspace F n Calculate the two attention matrices: ①attn(F n )=σ(bn(gap(F n )⊙F n )), Where σ represents the sigmoid function, bn represents the batch normalization layer, gap represents the global average pooling function, ⊙ represents the dot product operation, and attn(F n ) represents the first attention matrix generated. This represents a new feature subspace to which an attention mechanism has been applied; 2attn(F n )=σ(pconv(mp(dconv(F n )))), Where σ represents the sigmoid function, dconv represents a depthwise convolution operation with a 1×1 kernel, mp represents a max pooling function with a 3×3 window size and a stride of 1, pconv represents a dotted convolution operation with a 1×1 kernel, and ⊙ represents a dotted product operation. This represents pixel-by-pixel addition, attn(F) n ) represents the generated second attention matrix. This represents a new feature subspace to which an attention mechanism has been applied.

4. The method of fusing the new feature subspace in step 3 according to claim 1, characterized in that, All new feature subspaces with two attention mechanisms applied respectively are concatenated by channel, and then cross-channel features are fused and dimensionality reduced by 1×1 convolution operation; F new =conv1*1(concat(F new_1 ,F new_2 )) Where conv1*1 represents a 1×1 convolution operation, concat represents concatenation by channel, and F new This represents the semantically enhanced features obtained.

Images