An image classification method and related apparatus, device, and storage medium

Abstract

The application discloses an image classification method and related devices, equipment and storage media. The image classification method uses a feature extraction backbone network containing a variability convolution module and a scale-in feature interaction layer module to perform multi-scale feature extraction, and then uses a multi-scale feature fusion module to selectively fuse complementary information of different scales by using a cross-scale multi-head self-attention mechanism. On the basis of the first fused feature map, a series of feature enhancement and interaction operations such as spatial feature decoupling and lesion pixel feature learning, extraction and remodeling of long-distance dependence features, and strengthening of small target sample features are continued, and finally the fused features are input into a classifier for classification processing, and the classification probability of the medical image is output. The above method can effectively integrate shallow details and deep semantic information, enhance the feature expression ability under complex interference, and improve the classification accuracy of the lesion area of the medical image, thereby providing an efficient and accurate solution for automatic clinical diagnosis.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to an image classification method and related devices, equipment and storage media. Background Technology

[0002] Gastrointestinal diseases are susceptible to various conditions, potentially causing symptoms such as abdominal pain, bloating, changes in bowel habits, difficulty swallowing, gastrointestinal bleeding, and even cancer, severely impacting people's quality of life. Early diagnosis of serious gastrointestinal diseases can significantly control the risk of complications. Gastrointestinal endoscopy, as the only method for directly observing the mucus lining of the digestive tract, has become an irreplaceable diagnostic tool in gastrointestinal examinations. However, manual interpretation of endoscopic images is time-consuming and error-prone due to limited field of view and overlapping conditions. On the other hand, deep learning has achieved success in computer vision tasks, with accuracy comparable to human experts. Therefore, the demand for automated classification of gastrointestinal endoscopic images using computer-aided diagnostic systems is growing.

[0003] Traditional diagnostic methods for gastrointestinal diseases rely on machine learning models, such as logistic regression, support vector machines, and feature fusion techniques, to manually extract color, texture, and morphological features from images. However, these methods depend on expert experience, struggle to adapt to lesion polymorphism, and lack robustness under complex imaging conditions (such as low-contrast WCE images). For example, Yuan et al. achieved an efficiency of 88.61% using K-means clustering analysis of WCE images; Sen Wang et al. developed the Second Glance paradigm based on 1,504 WCE images, achieving an ROC-AUC of 0.9235. Nevertheless, because lesions in the digestive tract may occupy a very small area in endoscopic images or spread along a certain direction, resulting in significant differences in the size and contour of diseases within a class, these natural attributes of lesion areas can lead to a loss of key semantic and detailed contextual information in general classification algorithms. This makes it difficult to extract features from endoscopic images, thus hindering these methods from capturing the complex nonlinear features of diseased tissues.

[0004] In recent years, deep learning-based algorithms have gradually become mainstream, such as Convolutional Neural Networks (CNNs), attention mechanism models, and transfer learning frameworks. These algorithms automatically extract deep semantic features through end-to-end learning, significantly improving classification performance. For example, Asdasivan et al. achieved an AUC of 98.65% using strip-level CNN classification; Sharif et al. combined Grad-CAM++ and SegNet to achieve 98% accuracy and 99% ROC-AUC on the KID dataset. However, interference factors such as soapy water, turbidity, and debris generated by the movement of the capsule camera during endoscopic imaging reduce the detection rate of abnormal images. This makes it difficult for most classification models that are effective for natural images to effectively encode the contextual information of lesion regions from these complex structures. For example, CNNs lack built-in saliency mechanisms and long-distance dependency learning capabilities, resulting in poor classification performance for small lesions and long focal regions (such as normal Z-lines and esophagitis). In addition, attention mechanism models are prone to overfitting and require pre-training on large datasets. Summary of the Invention

[0005] This application provides an image classification method and related devices, equipment, and storage medium, which can at least solve the problems of insufficient extraction of small target lesion features and insufficient classification accuracy under complex background interference in existing medical image classification methods.

[0006] In a first aspect, this application provides an image classification method, which includes the following steps:

[0007] S1: Preprocess the acquired medical images to obtain a size-standardized input image;

[0008] S2: Input the input image into the feature extraction backbone network to extract multiple feature maps at different scales. The feature extraction backbone network includes deformable convolutional modules and intra-scale feature interaction layer modules.

[0009] S3: Input multiple feature maps of different scales into the multi-scale feature fusion module, calculate the attention weights between feature maps of different scales, and fuse the feature information of feature maps of different scales based on the attention weights to generate the first fused feature map;

[0010] S4: Input the first fused feature map into the dynamic upsampling layer to decouple spatial features, and simultaneously input the first fused feature map into the enhanced convolution module to learn lesion pixel features. Then, perform a fusion operation on the outputs of the dynamic upsampling layer and the enhanced convolution module to obtain the second fused feature map.

[0011] S5: Obtain the feature map with the same scale as the second fused feature map from multiple feature maps of different scales, and then input it into the VSS fusion module with the second fused feature map to extract and reshape long-distance dependent features to obtain the third fused feature map;

[0012] S6: Repeat steps S3, S4 and S5 in sequence to output a low-dimensional feature map. Input the low-dimensional feature map into the small target enhancement module for feature enhancement to obtain the fourth fusion feature map.

[0013] S7: The first, third, and fourth fusion feature maps are concatenated along the channel dimension and then input into the classifier for classification processing, outputting the classification probability of the medical image.

[0014] In some embodiments, in step S6, the small target enhancement module is a cascaded hybrid channel cross-shuffle attention module, which includes a hybrid channel attention module and a cross-shuffle attention module.

[0015] The low-dimensional feature map is input into the hybrid channel attention module, which takes the output of the VSS fusion module as its first input. After local average pooling, it is fed into a first branch containing a one-dimensional depthwise separable convolution and a second branch consisting of global average pooling, a one-dimensional depthwise separable convolution, and inverse average pooling. The kernel size of the one-dimensional depthwise separable convolution is proportional to the number of channels, formally expressed as:

[0016]

[0017] Where C is the number of channels, k is the size of the convolution kernel, γ and b are hyperparameters, and odd indicates that k is odd; if k is even, the value is incremented by one.

[0018] The cross-shuffle attention module takes the output of the multi-scale feature fusion module as its second input and fuses it with the output of the hybrid channel attention module.

[0019] The cross-shuffle attention module enhances cross-channel feature interaction and global information fusion through the third and fourth branches, and outputs a fourth fused feature map.

[0020] The third branch adjusts the channel dimensions using a 1×1 convolutional layer, then performs a shuffling operation to mix the channel information and groups it according to the channel dimensions. The features from each group are then concatenated and extracted using a 3×3 convolutional layer, formally as follows:

[0021]

[0022] Where F1 represents the input feature map, O l Represents the output feature map, δ c (·) indicates a shuffling operation along the channel dimension;

[0023] The fourth branch employs a self-attention mechanism, extracting the query vector Q, key vector K, and value vector V through 1×1 convolution and 3×3 depthwise separable convolution, generating corresponding tensors. After tensor shape adjustment and interactive operations, the attention map is calculated, and the output of the attention map is formalized as follows:

[0024]

[0025] Here, α is a learnable scaling parameter used to control the size of matrix multiplication of K and Q before applying the Sigmoid function.

[0026] In some embodiments, the method of concatenating the data along the channel dimension and then inputting it into a classifier for classification in step S7 includes:

[0027] The first, third, and fourth fused feature maps are adjusted to the same resolution and then stitched together along the channel dimension to generate the fifth fused feature map.

[0028] Input the fifth fused feature map into the classifier, and perform the following operations in the classifier:

[0029] Spatial dimension compression is performed on the fifth fused feature map to generate image-level feature vectors;

[0030] The image-level feature vectors are mapped to the category space through a fully connected layer and then processed by the Sigmoid function to output the classification probability of the medical image.

[0031] In some embodiments, the method for training the classifier includes:

[0032] We define a loss function WIoUv3 based on bounding box regression to supervise model training. The loss function is formalized as follows:

[0033]

[0034]

[0035]

[0036] Where IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box, L IoU The value of the bounding box loss function, (x,y), (x gt ,y gt ) represent the center coordinates of the predicted bounding box and the center coordinates of the ground truth bounding box, respectively; Wg and Hg represent the width and length of the minimum closure region, respectively; β is the outlier; α and δ are manually set hyperparameters; L * IoU denoted as the bounding box loss value in the actual process, and r as the gradient gain.

[0037] In some embodiments, in step S5

[0038] The VSS fusion module performs the following operations:

[0039] Channel dimension transformation is performed on the input features of the feature map and the second fused feature map with the same scale;

[0040] The transformed input features are split into first sub-features and second sub-features along the channel dimension;

[0041] The first sub-features are processed through a multi-level nested residual structure to extract and reshape long-distance dependent features;

[0042] The processed first sub-feature is concatenated with the second sub-feature along the channel dimension.

[0043] The concatenated features are ensembled using convolution to output an enhanced third fused feature map.

[0044] In some embodiments, in step S3, the multiple feature maps of different scales include at least shallow feature maps, medium-level feature maps, and deep feature maps;

[0045] The multi-scale feature fusion module performs the following operations:

[0046] The shallow and deep feature maps are adjusted to the same size as the medium-level feature map by adjusting the resolution.

[0047] The high-dimensional information obtained from the shallow feature map is mapped to the query vector Q, and the low-dimensional information obtained from the deep feature map is mapped to the key vector K and the value vector V.

[0048] The query vector Q, key vector K, and value vector V are divided into N attention heads according to the channel dimension. Within each attention head, the attention weight A is calculated in the form of…

[0049]

[0050] in, , ;

[0051] The attention weights are weighted and the value vector V is summed, and the outputs of each attention head are concatenated and their shapes reconstructed to obtain attention feature maps at multiple scales.

[0052] The attention feature map is partitioned into feature regions, and feature information from different feature regions is selectively fused through a gating mechanism to generate the first fused feature map.

[0053] In some embodiments, in step S4, the enhanced convolutional module is a KAN-enhanced GSConv module, and the following operations are performed:

[0054] The first fused feature map is input into the enhanced convolution module, wherein the first fused feature map has C1 channels;

[0055] The number of channels in the first fused feature map is adjusted to C2 / 2 by KAN convolution, and the features are captured and stitched together by depthwise separable convolution.

[0056] The feature map with C2 channels is output through the channel shuffling operation.

[0057] In some embodiments, in step S2, the feature maps of multiple different scales include at least deep feature maps;

[0058] The deformable convolution module includes deformable convolutional layers that apply a learnable offset to each sample point of the convolutional kernel to adapt to the target geometry.

[0059] The intra-scale feature interaction layer module includes a deformable attention layer, which is used to encode global relevance in deep feature maps to enhance semantic feature extraction capabilities. The deformable attention layer is formally defined as follows:

[0060]

[0061] Flatten(·) transforms a multidimensional tensor into a two-dimensional tensor, DIFI(·) enables the exchange and fusion of feature information between query Q, K, and V, and Reshape(·) restores the shape of the two-dimensional tensor to the same shape as the deep feature map. The output feature map is F. l Q is the query vector, K is the key vector, V is the value vector, and P5 represents the deep feature map.

[0062] In some embodiments, the method for preprocessing the medical image in step S1 to obtain a size-normalized input image includes:

[0063] Medical images are adjusted to a predetermined size using an interpolation algorithm, and a combination of center cropping and edge padding is used to achieve size adaptation for aspect ratio images.

[0064] The medical image after size adaptation is subjected to channel normalization to obtain a size-standardized input image. The channel normalization process includes pixel value normalization and channel number conversion operations.

[0065] Secondly, this application provides an image classification apparatus, the apparatus comprising:

[0066] The image acquisition module is used to preprocess the acquired medical images to obtain input images with standardized dimensions;

[0067] The feature extraction backbone network is used to extract feature maps of multiple different scales from the input image. The feature extraction backbone network includes deformable convolutional modules and intra-scale feature interaction layer modules.

[0068] The multi-scale feature fusion module is used to acquire multiple feature maps at different scales, calculate the attention weights between feature maps at different scales, and fuse the feature information of feature maps at different scales based on the attention weights to generate the first fused feature map.

[0069] The first feature enhancement module is used to input the first fused feature map into the dynamic upsampling layer for spatial feature decoupling, and simultaneously input the first fused feature map into the enhancement convolution module for lesion pixel feature learning. The outputs of the dynamic upsampling layer and the enhancement convolution module are fused to obtain the second fused feature map.

[0070] The VSS fusion module is used to fuse feature maps of the same scale as the second fusion feature map from multiple feature maps of different scales, extract and reshape long-distance dependent features, and obtain the third fusion feature map.

[0071] The small target enhancement module is used to repeatedly input the third fused feature map into the multi-scale feature fusion module, the first feature enhancement module and the VSS fusion module for processing, thereby outputting a low-dimensional feature map. The low-dimensional feature map is then enhanced with small target sample features to obtain the fourth fused feature map.

[0072] The image classification module is used to concatenate the first, third, and fourth fusion feature maps along the channel dimension and input them into the classifier for classification processing, outputting the classification probability of the medical image.

[0073] Thirdly, this application provides an electronic device including a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the image classification method described in the foregoing embodiments.

[0074] Fourthly, this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the image classification method described in the foregoing embodiments.

[0075] The image classification method of this application provides a feature extraction backbone network that uses a variable convolutional module and an intra-scale feature interaction layer module to extract features at multiple scales, constructing interaction relationships between different scales and fully utilizing feature information from shallow details and deep semantics. Then, a multi-scale feature fusion module selectively fuses complementary information from different scales using a cross-scale multi-head self-attention mechanism to mine discriminative information related to lesion regions. Based on the first fused feature map, a series of feature enhancement and interaction operations are performed, including spatial feature decoupling, lesion pixel feature learning, extraction and reshaping of long-distance dependent features, and strengthening of small target sample features. This enhances the model's feature representation ability in complex backgrounds. Finally, the fused features are concatenated and input into a classifier for classification, outputting the classification probability of the medical image. This method effectively integrates shallow details and deep semantic information, enhances feature representation ability under complex interference, improves the classification accuracy of lesion regions in medical images, significantly improves classification efficiency, and provides an efficient and accurate solution for automated clinical diagnosis. Attached Figure Description

[0076] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 This is a flowchart illustrating the image classification method provided in an embodiment of this application;

[0078] Figure 2 A schematic diagram of the image classification device provided in this application embodiment;

[0079] Figure 3 yes Figure 1 A flowchart illustrating step S1.

[0080] Figure 4 This is a flowchart illustrating the operation of the multi-scale feature fusion module.

[0081] Figure 5 This is a schematic diagram of the algorithm structure of the multi-scale feature fusion module;

[0082] Figure 6 This is a flowchart illustrating the operations performed by the enhanced convolution module;

[0083] Figure 7 This is a schematic diagram of the algorithm structure of the enhanced convolution module;

[0084] Figure 8 This is a flowchart illustrating the operations performed by the VSS fusion module;

[0085] Figure 9This is a schematic diagram of the algorithm structure of the VSS fusion module;

[0086] Figure 10 This is a schematic diagram of the algorithm structure of the small target reinforcement module;

[0087] Figure 11 This is a flowchart illustrating the classifier's execution process.

[0088] Figure 12 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application.

[0089] Figure 13 This is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0090] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0091] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0092] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0093] The image classification method and related devices, equipment, and storage media provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0094] The image classification method provided in this application can be applied to scenarios involving the classification of lesion areas or lesions in medical images. Furthermore, the image classification method provided in this application can be executed by an image classification device or a control module within that device. This application uses an image classification device executing the image classification method as an example to illustrate the image classification method provided in this application.

[0095] Figure 1 A flowchart illustrating the image classification method provided in an embodiment of this application is shown. Figure 2 A schematic diagram of the image classification device provided in an embodiment of this application is shown. The image classification device includes an image acquisition module 201, a feature extraction backbone network 202, a multi-scale feature fusion module 203, a first feature enhancement module 204, a VSS fusion module 205, a small target enhancement module 206, and an image classification module 207. The image classification method may specifically include the following steps:

[0096] Step S1: Acquire medical images, preprocess the medical images to obtain size-standardized input images;

[0097] Here, medical images can be related to the digestive tract, such as gastroscopy and colonoscopy images, or various other types such as lung CT images, brain MRI images, and bone X-ray images. The specific image type can be selected according to the actual application scenario and diagnostic needs, and there is no strict limitation here.

[0098] The medical images mentioned here are usually unprocessed raw images. These raw images may have many problems that affect subsequent processing and analysis. Therefore, it is necessary to preprocess the raw images to provide a consistent input format for subsequent feature extraction and model training, which will facilitate subsequent analysis and recognition.

[0099] In one alternative implementation, such as Figure 3 As shown, step S1 above may include:

[0100] Step S101: Adjust the medical image to a predetermined size using an interpolation algorithm, and use a combination of center cropping and edge padding strategy to achieve size adaptation for aspect ratio images;

[0101] Specifically, a bilinear interpolation algorithm is performed on the medical image to adjust its size to a predetermined size. For medical images with an unbalanced aspect ratio, after center cropping to retain the central region of the medical image, pixel values ​​are filled to the predetermined size at the shorter edges to achieve size adaptation. The predetermined size can be selected as 224×224.

[0102] The specific calculation process is as follows:

[0103] First, calculate the coordinates of the medical image:

[0104] W represents width, H represents height.

[0105] Then determine the neighboring pixels (i, j), (i+1, j), (i, j+1), and (i+1, j+1). Where i = ⌊ x ⌋ and j = ⌊ y ⌋, calculate the weights u = xi and v = yj, and finally apply the bilinear interpolation formula:

[0106]

[0107] Among them, (1-u)(1-v), u(1-v), (1-u)v, and uv are weights to ensure that the interpolation result is smooth.

[0108] For images with unbalanced aspect ratios, center cropping and edge padding will be performed to adapt the image to the desired size. The formula for center cropping is as follows:

[0109] When W>H:

[0110]

[0111] When H>W:

[0112]

[0113] The cropped image is then scaled to 224×224 using bilinear interpolation.

[0114] The edge filling operation first calculates the scaling factor to make the maximum side of the medical image equal to 224, and then calculates the scaled size. However, the width or height of the scaled medical image may be less than 224, so filling is required on the shorter sides. The entire process formula is summarized as follows:

[0115]

[0116]

[0117]

[0118]

[0119] Step S102: Perform channel normalization processing on the size-adapted medical image to obtain the size-standardized input image, wherein the channel normalization processing includes pixel value normalization and channel number conversion operations.

[0120] Specifically, the pixel value normalization operation is formalized as follows:

[0121]

[0122] Where I represents the pixel value of the original medical image, μ is the mean of each channel, and σ is the standard deviation of each channel. norm This represents the normalized value, with a distribution close to the mean of 0 and a label difference of 1.

[0123] The purpose of the channel conversion operation is to convert the size-adapted medical image into a three-channel RGB mode. The specific operation is formalized as follows:

[0124]

[0125] Among them, I gray ∈R H×W×C This represents a medical image after size adaptation for multiple channels; I RGB ∈R H×W×3 This represents the output three-channel image; c∈0,1,2 corresponds to the R, G, and B channels respectively, and C represents the number of channels. H×W×C This represents a three-dimensional tensor consisting of H rows (also representing the image height), W columns (also representing the image width), and C channels.

[0126] Step S2: Input the input image into the feature extraction backbone network 202 to extract multiple feature maps at different scales; wherein the feature extraction backbone network 202 includes a deformable convolutional module 2020 for adapting to the target geometry and an intra-scale feature interaction layer module 2021 for encoding global correlations;

[0127] Specifically, after the original medical image is processed in step S1, a size-standardized input image is obtained. The input image is then fed into the feature extraction backbone network 202. This network extracts multiple feature maps of different scales through hierarchical processing. Each feature map corresponds to target region information of different sizes in the input image.

[0128] Multiple feature maps at different scales specifically cover multi-level feature representations from low to high. Shallow feature maps retain rich detailed information, such as the texture of the digestive tract mucosa and minute protrusions; mid-level feature maps take into account both details and semantics, and can present the overall morphology of the polyp and its preliminary relationship with surrounding tissues; deep feature maps focus on semantic information, and can reflect the location distribution of the lesion in the global image and its large-scale tissue correlation. These feature maps at different scales are generated sequentially from shallow to deep by the feature extraction backbone network 202, forming a multi-dimensional analysis of the input image.

[0129] The Deformable Convolution Module 2020 includes deformable convolutional layers. These layers apply a learnable offset to each sample point of the convolutional kernel to adapt to the target geometry. The calculation process is as follows:

[0130]

[0131] Where, ∆p n For p n The offset of the position. x(p0+p n +∆p n () represents the offset pixels.

[0132] The intra-scale feature interaction layer module 2021 includes a deformable attention layer. This deformable attention layer encodes global relevance in deep feature maps to enhance semantic feature extraction capabilities and capture more comprehensive semantic features. The deformable attention layer is formalized as follows:

[0133]

[0134] Flatten(·) transforms a multidimensional tensor into a two-dimensional tensor, DIFI(·) enables the exchange and fusion of feature information between queries Q, K, and V, and Reshape(·) restores the shape of the two-dimensional tensor to the same shape as the deep feature map. The output feature map is F. l Q is the query vector, K is the key vector, V is the value vector, and P5 represents the deep feature map.

[0135] S3: Input multiple feature maps of different scales into the multi-scale feature fusion module 203, calculate the attention weights between feature maps of different scales using the cross-scale multi-head self-attention mechanism, and fuse the feature information of feature maps of different scales based on the attention weights to generate the first fused feature map.

[0136] In one alternative implementation, please refer to [the following text is also provided] Figure 4 and Figure 5 , Figure 4 This is a flowchart illustrating the operation of the multi-scale feature fusion module. Figure 5 This is a schematic diagram of the algorithm structure of the multi-scale feature fusion module.

[0137] In step S3, the multiple feature maps of different scales include at least shallow feature maps, medium-level feature maps, and deep feature maps;

[0138] To more fully capture lesion features, feature maps of multiple different scales are input into the multi-scale feature fusion module 203 for selective complementary fusion. The multi-scale feature fusion module 203 utilizes a cross-scale multi-head self-attention mechanism to perform the following operations:

[0139] S301: Adjust the shallow and deep feature maps to the same size as the medium-level feature map through a resolution adjustment operation;

[0140] Specifically, P3 is defined as a shallow feature map, P4 as a medium-level feature map, and P5 as a deep feature map. Wherein:

[0141] Shallow feature map ,

[0142] Mid-level feature map ,

[0143] Deep feature map .

[0144] The shallow and deep feature maps are adjusted to the same size as the medium-level feature map through convolution and interpolation operations.

[0145] S302: Map the high-dimensional information obtained from the shallow feature map to the query vector Q (query), and map the low-dimensional information obtained from the deep feature map to the key vector K (key) and the value vector V (value).

[0146] Specifically, the high-dimensional information obtained from the P3 diagram Mapping to query vector Mapping the low-dimensional information obtained from the P5 graph to the key vector Sum value vector .

[0147] The query vector, key vector, and value vector here can also be understood as the query matrix, key matrix, and value matrix.

[0148] S303: Divide the query vector Q, key vector K, and value vector V into N attention heads according to the channel dimension, and calculate the attention weight matrix A in each attention head, in the form of:

[0149]

[0150] in, , ;

[0151] The number of channels per attention head is C m / N, where N is the number of attention heads.

[0152] S304: The attention weights are weighted and the value vector V is weighted, and the outputs of each attention head are concatenated and their shapes reconstructed to obtain attention feature maps O at multiple scales;

[0153] Specifically, the attention weight matrix A is multiplied by the value vector V, the outputs of all attention heads are concatenated and reshaped back to their original shape, and the attention feature map O is formalized as follows:

[0154]

[0155] S305: Divide the attention feature map into feature partitions and selectively fuse the feature information of different feature partitions through a gating mechanism to generate the first fused feature map.

[0156] Specifically, It is divided into four equal parts along the channel dimension. The attention feature maps fed into this multi-scale feature fusion module, such as shallow, medium, and deep features, are generated through the above operations, and are respectively... , and The attention feature map is partitioned into feature regions, and h is used for each region. i l i and m i Let represent the i-th partition feature of the high-dimensional, low-dimensional, and current-dimensional layers, respectively. Then, a gating mechanism is used to selectively fuse different partition features. The gating mechanism multiplies each partition feature by its corresponding gating coefficient, strengthening the feature contribution of important partitions (such as regions containing key lesion information) and weakening the influence of redundant or interfering partitions (such as background noise regions). The specific calculation process is as follows:

[0157]

[0158]

[0159] in Indicates in m i The value obtained by applying the activation function. This represents the selective fusion result of the corresponding channel dimension features. It is the output image of the multi-scale feature fusion module 203, namely the first fused feature map.

[0160] S4: Input the first fused feature map into the dynamic upsampling layer 2040 for spatial feature decoupling, and simultaneously input the first fused feature map into the enhanced convolution module 2041 for lesion pixel feature learning. Then, perform a fusion operation on the output of the dynamic upsampling layer 2040 and the output of the enhanced convolution module 2041 to obtain the second fused feature map.

[0161] Here, the first fused feature map is divided into two paths. One path is input to the dynamic upsampling layer 2040 to solve the feature coupling problem caused by the variable morphology of lesion regions in medical images, achieving spatial feature decoupling to eliminate interference from non-lesion tissues, improve the resolution of key regions, and output an enhanced feature map Y. up One input enhances the convolution module 2041, which strengthens the discriminative features of lesion pixels and suppresses background noise.

[0162] In one alternative implementation, please refer to [the following text is also provided] Figure 6 and Figure 7 , Figure 6 This is a flowchart illustrating the operations performed by the enhanced convolution module 2041. Figure 7 This is a schematic diagram of the algorithm structure of the enhanced convolutional module 2041. The enhanced convolutional module 2041 is a KAN-enhanced GSConv module, which performs the following operations:

[0163] S401: Input the first fused feature map into the enhanced convolution module 2041, wherein the first fused feature map has C1 channels;

[0164] S402: The number of channels in the first fused feature map is adjusted to C2 / 2 by KAN convolution, and the features are captured and stitched by depthwise separable convolution;

[0165] S403: Outputs a feature map with C2 channels through a channel shuffling operation.

[0166] Here, the output feature map Y has C2 channels. m Formalized as:

[0167]

[0168] Where KAN(•) represents a KAN convolution with C² / 2 output channels. DWConv(•) is a depthwise separable convolution with C² / 2 output channels. Concat(•) and Shuffle(•) represent concatenation and shuffling operations along the channel dimension, respectively.

[0169] The output Y of the dynamic upsampling layer 2040 up and the output of the enhanced convolution module 2041 Perform a fusion operation to obtain a second fused feature map.

[0170] S5: Obtain the feature map with the same scale as the second fused feature map from multiple feature maps of different scales. Input the feature map with the same scale and the second fused feature map into the VSS fusion module 205 to extract and reshape long-distance dependent features to obtain the third fused feature map.

[0171] Obtain feature maps of the same scale as the second fused feature map from multiple feature maps of different scales. Then, concatenate the feature map of the same scale with the second fused feature map in the channel dimension to obtain a feature stitched map. Input the feature stitched map into the VSS fusion module 205.

[0172] In one alternative implementation, please refer to [the following text is also provided] Figure 8 and Figure 9 , Figure 8 This is a flowchart illustrating the operations performed by the VSS fusion module 205. Figure 9This is a schematic diagram of the algorithm structure of the VSS fusion module 205. The VSS fusion module 205 performs the following operations:

[0173] S501: Perform channel dimension transformation on the input features of the feature map and the second fused feature map with the same scale;

[0174] Specifically, the input features of the obtained feature stitched map are linearly transformed along the channel dimension using a standard convolutional layer, such as... Figure 9 The Conv operation in part (a).

[0175] S502: Decompose the transformed input features into first sub-features and second sub-features along the channel dimension; for example... Figure 9 The Split operation in section (a).

[0176] S503: The first sub-feature is processed through a multi-level nested residual structure to extract and reshape long-distance dependent features;

[0177] In this step, the first sub-feature is processed through a multi-level nested residual structure to model long-distance dependent features, thereby achieving multi-lesion association analysis, semantic alignment between small targets and macro background, and cross-slice feature fusion in 3D volume data.

[0178] Multi-level nested residual structures, such as Figure 9 The multi-layered, serially nested VSS module (Visual State Space Module, VSS module) structure in Part (a), and the algorithmic structure of the VSS module, are as follows: Figure 9 Part (b) first stabilizes the feature distribution through layer normalization, adjusts the feature scale through linear interpolation, then extracts local features using depthwise separable convolution, introduces nonlinearity by combining the SiLU activation function, subsequently captures the global context through two-dimensional selective scanning, and finally applies layer normalization again and fuses features through successive multiplications to form a complete feature extraction and reshaping process. This approach enhances useful features, suppresses redundancy, makes the processed features more suitable for subsequent tasks, and improves the model's ability to capture and express key information.

[0179] S504: Concatenate the processed first sub-feature with the second sub-feature along the channel dimension;

[0180] S505: Perform convolutional ensemble on the spliced ​​features and output an enhanced third fused feature map.

[0181] The computational form of the operation of VSS fusion module 205 is as follows:

[0182]

[0183]

[0184]

[0185] Where σ1(·) and σ2(·) represent activation functions. Conv(·) represents the convolution operation, Split(·) and Concat(·) represent the splitting and concatenation operations, respectively, and F i This represents a set of operations preceding the i-th VSS module.

[0186] S6: Repeat steps S3, S4 and S5 sequentially according to the predetermined number of repetitions to output a low-dimensional feature map. Input the low-dimensional feature map into the small target enhancement module 206 for feature enhancement to obtain the fourth fusion feature map.

[0187] Using the third fused feature map as input, steps S3, S4, and S5 are repeated at least once. The number of repetitions N is predetermined according to actual requirements and is not limited thereto.

[0188] After repeating these steps multiple times, a low-dimensional feature map is output and input into the small target enhancement module 206 for feature enhancement.

[0189] In one alternative implementation, please refer to Figure 10 , Figure 10 This is a schematic diagram of the algorithm structure of the small target enhancement module 206. The small target enhancement module 206 is a cascaded hybrid channel cross-shuffling attention module 210, which includes a hybrid channel attention module 2060 and a cross-shuffling attention module 2061.

[0190] The low-dimensional feature map is input into the hybrid channel attention module 2060. The hybrid channel attention module 2060 takes the output of the VSS fusion module 205 as its first input. After local average pooling, it is fed into a first branch containing a one-dimensional depthwise separable convolution and a second branch consisting of global average pooling, one-dimensional depthwise separable convolution, and inverse average pooling. The kernel size of the one-dimensional depthwise separable convolution is proportional to the number of channels, formally expressed as:

[0191]

[0192] Where C is the number of channels, k is the size of the convolution kernel, γ and b are hyperparameters, and odd indicates that k is odd; if k is even, the value is incremented by one.

[0193] The cross-shuffle attention module 2061 takes the output of the multi-scale feature fusion module 203 as the second input and fuses it with the output of the hybrid channel attention module 2060; this module can learn information at different scales and semantic levels.

[0194] The cross-shuffle attention module 2061 enhances cross-channel interaction and global information fusion through the third and fourth branches, and outputs a fourth fused feature map.

[0195] The third branch adjusts the channel dimensions using a 1×1 convolutional layer, performs a shuffling operation to mix the channel information, and groups it according to the channel dimensions. The features from each group are then concatenated and extracted using a 3×3 convolutional layer, formally as follows:

[0196]

[0197] Where F1 represents the input feature map, O l Represents the output feature map, δ c (·) indicates a shuffling operation along the channel dimension;

[0198] The fourth branch employs a self-attention mechanism, extracting the query vector Q, key vector K, and value vector V through 1×1 convolution and 3×3 depthwise separable convolution, and generating corresponding tensors. These tensors are three tensors with shapes of H×W×C. Subsequently, Q and K are reshaped to Q∈R. HW×C and K∈R C×HW Then, the attention map A∈R is computed through the interaction of Q and K. C×C The output of the attention map is formalized as follows:

[0199]

[0200] Here, α is a learnable scaling parameter used to control the size of matrix multiplication of K and Q before applying the Sigmoid function. The Sigmoid function is a mathematical model widely used in machine learning and statistics, especially suitable for activation scenarios in logistic regression and neural networks. The main function of the Sigmoid function is to map any real value to the interval (0, 1), symbolizing probability or the likelihood of an event occurring.

[0201] Step S6 enhances the feature extraction capability for small target samples.

[0202] S7: Adjust the resolution of the first, third, and fourth fusion feature maps to be the same, stitch them together along the channel dimension, and input them into classifier 2070 for classification processing, outputting the classification probability of the medical image.

[0203] In one optional implementation, the method for inputting the concatenated data along the channel dimension into the classifier 2070 for classification in step S7 includes:

[0204] The first, third, and fourth fused feature maps are adjusted to the same resolution and then stitched together along the channel dimension to generate the fifth fused feature map.

[0205] The operation can be formalized as follows:

[0206]

[0207] Upsample represents the upsampling operation, which adjusts a low-resolution feature map to a high-resolution feature map. Conv 1×1 Used to adjust the number of channels.

[0208] Input the fifth fused feature map into classifier 2070, such as Figure 11 As shown, the following operations are performed in classifier 2070:

[0209] S701: Perform spatial dimension compression on the fifth fused feature map to generate an image-level feature vector;

[0210] Specifically, the spatial dimension of the fifth fused feature map is compressed into a single vector to represent the features of the entire image through global average pooling. This process can be formalized as follows:

[0211]

[0212] Among them, z c This represents the pooling result of the c-th channel.

[0213] S702: Maps image-level feature vectors to the class space through a fully connected layer and outputs the classification probability of the medical image after processing by the Sigmoid function.

[0214] Specifically, the pooled feature vector z c The entire process of mapping classification is formalized as follows: the classification is mapped to the category space through a fully connected layer, and the category scores are converted into probabilities through the Sigmoid function.

[0215]

[0216]

[0217] Among them, each s i This represents the unnormalized score of the i-th category, where each p... i ∈[0,1], representing the independent probability of the i-th category.

[0218] The training process of classifier 2070 is a core technical step in achieving high-precision diagnosis. In some embodiments, the training method of classifier 2070 includes:

[0219] We define a loss function WIoUv3 based on bounding box regression to supervise model training. This function reduces interference from low-quality samples through outlier optimization. WIoU uses outliers to evaluate the quality of anchor boxes, reducing competition among high-quality samples and minimizing harmful gradients generated by low-quality samples, thus optimizing model parameters more effectively.

[0220] The loss function is formalized as follows:

[0221]

[0222]

[0223]

[0224] Where IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box, L IoU The value of the bounding box loss function, (x,y), (x gt ,y gt ) represent the center coordinates of the predicted bounding box and the center coordinates of the ground truth bounding box, respectively; Wg and Hg represent the width and length of the minimum closure region, respectively; β is the outlier; α and δ are manually set hyperparameters; L * IoU denoted as the bounding box loss value in the actual process, and r as the gradient gain.

[0225] In summary, this application provides an effective multi-scale feature interaction and fusion learning network architecture for solving medical image classification problems. It includes a feature extraction backbone network 202 and a multi-scale feature fusion module 203 that fully integrates cross-scale feature information. The feature extraction backbone fully utilizes the complementary information of shallow details and deep semantics by constructing interaction relationships between features at different scales. The multi-scale feature fusion module further enhances the model's feature expression ability in complex backgrounds by mining the discriminative information related to the target region in shallow, medium, and deep features. Through repeated three-level feature enhancement and interaction, feature splicing and re-fusion, accurate classification is finally achieved.

[0226] Based on the above embodiments, several specific implementation scenarios will be given below to illustrate the above image classification method. At the same time, it will be compared with some classification methods in the prior art to demonstrate the high classification accuracy that can be achieved by the image classification method.

[0227] In the following application scenarios, accuracy, precision, the harmonic mean of precision and recall (F1-score), sensitivity (or recall), specificity, the proportion of samples where the model's predicted highest probability class matches the true label (Top_acc), and computational cost (GFLOPS) are used as evaluation metrics to test the performance of different algorithms.

[0228] Implementation Scenario 1

[0229] The image classification method in this embodiment achieves state-of-the-art performance compared to other algorithms on the Kvasir gastrointestinal medical image dataset classification task, as shown in Table 1.

[0230] Table 1 compares 12 methods, including AirFilterNet, ColonFuseNet, MCAN-GIC, MSCLDN, MIFN-Colon, MFFM, DenseNet201, DenseNet121, InceptionResNetV2, Xception, MobileNetV2, DeiT-Ti, DeiT-B384, ViT-L / 32, ViT-L / 16, and ViT-B / 16. The image classification method in this embodiment performs excellently on all evaluation metrics. Specifically, the accuracy of other related algorithms ranges from 85.9% to 94%, while the accuracy of the image classification method in this embodiment reaches 95.8%. Previous related algorithms achieved sensitivity from 83.1% to 95.1%, while the image classification method in this embodiment achieves 95.8% sensitivity, indicating that its ability to detect small lesions is superior to the CNN baseline. Specificity in previous studies ranged from 70.1% to 99.2%, while the graphic classification method implemented in this study achieved 99.3%. Furthermore, it outperformed all previous studies in terms of F1-score and precision, reaching 95.7% and 95.8%, respectively.

[0231] The high accuracy of the graphic classification method in the esophagitis category reflects its adaptability to ambiguous lesions. This also indicates that the graphic classification method has significant advantages in the classification of gastrointestinal lesions, especially in terms of accuracy and the ability to identify positive and negative samples.

[0232] Table 1 Quantitative comparison results of the Kvasir dataset

[0233]

[0234] Implementation Scenario 2

[0235] The image classification method of this application embodiment was compared with other methods on the HyperKvasir dataset, and the results are shown in Table 2.

[0236] The image classification method of this application embodiment performs best in most metrics. Top1 accuracy and Sensitivity both reach 90.1%. The image classification method of this application embodiment outperforms SimSaim by 17.6% on F1-score and outperforms supervised ResNet50 by 18.5%. To ensure fairness, all self-supervised models use ResNet50 as the backbone network. Furthermore, the image classification method of this application embodiment also outperforms Balanced-Mixup, which adds virtual data samples to imbalanced classes through a mixup process. This means that the image classification method of this application embodiment also has a certain degree of robustness to the imbalance problem. In addition, the image classification method of this application embodiment significantly outperforms all supervised learning baseline algorithms, including AlexNet, ResNe18, ResNet50, MobileNetV2, VGG19, and DenseNet121, highlighting the limitations of traditional CNNs in extracting complex morphological features. The image classification method of this application embodiment can accurately extract target features of small, medium, and large lesions.

[0237] Table 2 Quantitative Comparison Results of HyperKvasir Dataset

[0238]

[0239] Implementation Scenario 3

[0240] To further evaluate the performance of the image classification method in this application embodiment, a comparative experiment was conducted on the GasHisSDB gastric cancer pathology image classification dataset (160×160 size). Examples of images from the GasHisSDB dataset are shown in Table 3.

[0241] As can be seen, the image classification method of this application achieves an accuracy of 99.5%, which is superior to all methods in the table. The precision of normal samples is 99.6%, higher than that of abnormal samples (99.4%), because the number of positive samples is greater than that of abnormal samples, thus the trained model will be more biased towards the former. In addition, the sensitivity of normal samples and abnormal samples is 99.7% and 99.2%, respectively, which are 1.8% and 1.4% higher than the second highest 97.9% and 97.8%, respectively. This is crucial for avoiding missed diagnoses in clinical diagnosis.

[0242] The image classification method of this application exhibits high sensitivity and precision, with very close values, demonstrating the model's robustness. At the specificity level, it achieves 99.2% accuracy for normal samples and 99.7% accuracy for abnormal samples, also outperforming other existing excellent classification models, indicating good predictive performance for negative samples. Other models have F1-scores between 88.8% and 98.2% for normal samples and between 82.2% and 97.3% for abnormal samples, while the image classification method of this application achieves F1-scores of 99.6% and 99.4% for normal samples and abnormal samples, respectively, demonstrating the superior performance of this embodiment in classifying gastric cancer pathological images.

[0243] Table 3 Quantitative Comparison Results of GashisSDB Dataset

[0244]

[0245] Based on the three specific implementation scenarios mentioned above, the image classification method of this application embodiment demonstrates high classification accuracy in three large benchmark datasets, including the Kvasir digestive tract medical image dataset, the HyperKvasir dataset, and the GasHisSDB gastric cancer pathology image classification dataset, further confirming the efficiency and high accuracy of the image classification method of this application embodiment.

[0246] Please see Figure 12 , Figure 12 This is a schematic diagram of the hardware structure of an embodiment of the electronic device of this application. The device includes a memory and a processor coupled to each other. The processor is used to execute program instructions stored in the memory to implement the corresponding process as described in the foregoing method embodiment. In a specific implementation scenario, the electronic device 30 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 30 may also include mobile devices such as laptops and tablets, which are not limited here.

[0247] The electronic device may include a processor 301 and a memory 302 storing program instructions.

[0248] When processor 301 executes the program, it implements the steps in any of the above method embodiments.

[0249] For example, the program can be divided into one or more modules / units, one or more of which are stored in memory 302 and executed by processor 301 to complete this application. The one or more modules / units can be a series of program instruction segments capable of performing a specific function, which describe the execution process of the program in the device.

[0250] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0251] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 302 is non-volatile solid-state memory.

[0252] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0253] The processor 301 implements any of the methods described above by reading and executing program instructions stored in the memory 302.

[0254] In one example, the electronic device may also include a communication interface 303 and a bus 310. The processor 301, memory 302, and communication interface 303 are connected via the bus 310 and communicate with each other.

[0255] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0256] Bus 310 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 310 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0257] In addition, the process in conjunction with the foregoing method embodiments, such as Figure 13 As shown, this application embodiment can be implemented using a computer-readable storage medium 40. The storage medium 40 stores program instructions 401; these program instructions 401 are executed by a processor to implement any step in the above method embodiment.

[0258] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0259] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0260] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here.

[0261] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0262] The functional modules shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on machine-readable media or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer grids such as the Internet, intranets, etc.

[0263] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0264] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to create a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0265] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. An image classification method, characterized in that, The method includes the following steps: S1: Preprocess the acquired medical images to obtain a size-standardized input image; S2: Input the input image into the feature extraction backbone network to extract multiple feature maps at different scales. The feature extraction backbone network includes a deformable convolution module and an intra-scale feature interaction layer module. The multiple feature maps at different scales include at least deep feature maps. The deformable convolution module includes a deformable convolutional layer that applies a learnable offset to each sample point of the convolutional kernel to adapt to the target geometry. The intra-scale feature interaction layer module includes a deformable attention layer, which is used to encode global relevance in the deep feature map to enhance semantic feature extraction capabilities. The deformable attention layer is formally defined as follows: Where Flatten(·) transforms a multidimensional tensor into a two-dimensional tensor, DIFI(·) enables the exchange and fusion of feature information between query Q, K, and V, and Reshape(·) restores the shape of the two-dimensional tensor to the same shape as the deep feature map. The output feature map is F. l Q is the query vector, K is the key vector, V is the value vector, and P5 represents the deep feature map; S3: Input the multiple feature maps of different scales into the multi-scale feature fusion module, calculate the attention weights between feature maps of different scales, and fuse the feature information of feature maps of different scales based on the attention weights to generate the first fused feature map; S4: Input the first fused feature map into the dynamic upsampling layer for spatial feature decoupling, and simultaneously input the first fused feature map into the enhanced convolution module for lesion pixel feature learning. Then, perform a fusion operation on the outputs of the dynamic upsampling layer and the enhanced convolution module to obtain the second fused feature map. S5: Obtain the feature map with the same scale as the second fused feature map from the multiple feature maps of different scales, and then input it into the VSS fusion module along with the second fused feature map to extract and reshape long-distance dependent features to obtain the third fused feature map; S6: Repeat steps S3, S4 and S5 in sequence to output a low-dimensional feature map. Input the low-dimensional feature map into the small target enhancement module for feature enhancement to obtain the fourth fusion feature map. S7: The first fusion feature map, the third fusion feature map, and the fourth fusion feature map are concatenated along the channel dimension and then input into the classifier for classification processing, and the classification probability of the medical image is output.

2. The image classification method according to claim 1, characterized in that, In step S6, the small target enhancement module is a cascaded hybrid channel cross-shuffle attention module, which includes a hybrid channel attention module and a cross-shuffle attention module. The low-dimensional feature map is input into the hybrid channel attention module. The hybrid channel attention module takes the output of the VSS fusion module as the first input. After local average pooling, it is fed into a first branch containing a one-dimensional depthwise separable convolution and a second branch consisting of global average pooling, one-dimensional depthwise separable convolution and inverse average pooling. The kernel size of the one-dimensional depthwise separable convolution is proportional to the number of channels. The cross-shuffle attention module takes the output of the multi-scale feature fusion module as the second input and fuses it with the output of the hybrid channel attention module. The cross-shuffle attention module enhances cross-channel interaction of features and global information fusion through the third and fourth branches, and outputs the fourth fused feature map. The third branch adjusts the channel dimension through a 1×1 convolutional layer, performs a shuffling operation to mix the channel information and groups them according to the channel dimension. The features of each group are concatenated and then extracted by a 3×3 convolutional layer. The fourth branch employs a self-attention mechanism, extracting the query vector Q, key vector K, and value vector V through 1×1 convolution and 3×3 depthwise separable convolution, generating corresponding tensors, and calculating the attention map through tensor shape adjustment and interactive operations.

3. The image classification method according to claim 2, characterized in that, The method described in step S7, which involves concatenating the data along the channel dimension and then inputting it into a classifier for classification, includes: The first fused feature map, the third fused feature map, and the fourth fused feature map are adjusted to the same resolution and then stitched together in the channel dimension to generate the fifth fused feature map; The fifth fused feature map is input into the classifier, and the following operations are performed in the classifier: Spatial dimension compression is performed on the fifth fused feature map to generate an image-level feature vector; The image-level feature vectors are mapped to the category space through a fully connected layer and then processed by the Sigmoid function to output the classification probability of the medical image.

4. The image classification method according to claim 3, characterized in that, The training method for the classifier includes: Define the loss function L based on bounding box regression. WIoUv3 To supervise model training, the loss function is formalized as follows: Where IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box, L IoU The value of the bounding box loss function, (x,y), (x gt ,y gt ) represent the center coordinates of the predicted bounding box and the center coordinates of the ground truth bounding box, respectively; Wg and Hg represent the width and length of the minimum closure region, respectively; β is the outlier; α and δ are manually set hyperparameters; L * IoU denoted as the bounding box loss value in the actual process, and r as the gradient gain.

5. The image classification method according to claim 2, characterized in that, In step S5, The VSS fusion module performs the following operations: Channel dimension transformation is performed on the input features of the feature map with the same scale and the second fused feature map; The transformed input features are split into first sub-features and second sub-features along the channel dimension; The first sub-feature is processed through a multi-level nested residual structure to extract and reshape the long-distance dependency feature; The processed first sub-feature and the second sub-feature are concatenated along the channel dimension. The concatenated features are ensembled using convolution to output the enhanced third fused feature map.

6. The image classification method according to claim 1, characterized in that, In step S3, the multiple feature maps of different scales include at least shallow feature maps, medium-level feature maps, and deep feature maps; The multi-scale feature fusion module performs the following operations: The shallow and deep feature maps are adjusted to the same size as the medium-level feature map by a resolution adjustment operation. The high-dimensional information obtained from the shallow feature map is mapped to the query vector Q, and the low-dimensional information obtained from the deep feature map is mapped to the key vector K and the value vector V. The query vector Q, the key vector K, and the value vector V are divided into N attention heads according to the channel dimension, and the attention weight is calculated in each attention head. The attention weights are weighted and the value vector V is summed, and the outputs of each attention head are concatenated and their shapes reconstructed to obtain attention feature maps at multiple scales. The attention feature map is partitioned into feature regions, and feature information from different feature regions is selectively fused through a gating mechanism to generate the first fused feature map.

7. The image classification method according to claim 6, characterized in that, In step S4, the enhanced convolutional module is a KAN-enhanced GSConv module, and the following operations are performed: The first fused feature map is input into the enhanced convolution module, wherein the first fused feature map has C1 channels; The number of channels in the first fused feature map is adjusted to C2 / 2 by KAN convolution, and the features are captured and stitched together by depthwise separable convolution. The feature map with C2 channels is output through the channel shuffling operation.

8. The image classification method according to claim 1, characterized in that, The method for preprocessing the acquired medical image in step S1 to obtain a size-normalized input image includes: The medical image is adjusted to a predetermined size using an interpolation algorithm, and a combination of center cropping and edge padding is used to achieve size adaptation for aspect ratio images. The medical image after size adaptation is subjected to channel normalization processing to obtain the size-standardized input image, wherein the channel normalization processing includes pixel value normalization and channel number conversion operations.

9. An image classification device, characterized in that, include: The image acquisition module is used to preprocess the acquired medical images to obtain input images with standardized dimensions; A feature extraction backbone network is used to extract multiple feature maps of different scales from the input image. The feature extraction backbone network includes a deformable convolution module and an intra-scale feature interaction layer module, wherein the multiple feature maps of different scales include at least deep feature maps. The deformable convolution module includes a deformable convolutional layer that applies a learnable offset to each sample point of the convolutional kernel to adapt to the target geometry. The intra-scale feature interaction layer module includes a deformable attention layer, which is used to encode global relevance in the deep feature map to enhance semantic feature extraction capabilities. The deformable attention layer is formally defined as follows: Flatten(·) transforms a multidimensional tensor into a two-dimensional tensor, DIFI(·) enables the exchange and fusion of feature information between Q, K, and V, and Reshape(·) restores the shape of the two-dimensional tensor to the same shape as the deep feature map. The output feature map is F. l Q is the query vector, K is the key vector, V is the value vector, and P5 represents the deep feature map; A multi-scale feature fusion module is used to acquire the multiple feature maps of different scales, calculate the attention weights between the feature maps of different scales, and fuse the feature information of the feature maps of different scales based on the attention weights to generate a first fused feature map. The first feature enhancement module is used to input the first fused feature map into the dynamic upsampling layer for spatial feature decoupling, and simultaneously input the first fused feature map into the enhancement convolution module for lesion pixel feature learning, and perform a fusion operation on the outputs of the dynamic upsampling layer and the enhancement convolution module to obtain the second fused feature map; The VSS fusion module is used to fuse the feature maps of the multiple feature maps of different scales that have the same scale as the second fusion feature map with the second fusion feature map, extract and reshape long-distance dependent features, and obtain a third fusion feature map. The small target enhancement module is used to repeatedly input the third fused feature map into the multi-scale feature fusion module, the first feature enhancement module and the VSS fusion module for processing, thereby outputting a low-dimensional feature map, and performing small target sample feature enhancement on the low-dimensional feature map to obtain the fourth fused feature map. The image classification module is used to concatenate the first fusion feature map, the third fusion feature map, and the fourth fusion feature map along the channel dimension and input them into a classifier for classification processing, and output the classification probability of the medical image.

10. An electronic device, characterized in that, The method includes a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the image classification method according to any one of claims 1 to 8.

11. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the image classification method according to any one of claims 1 to 8.

Images