A lightweight fundus color image classification method based on factorization space and hierarchical attention.

Abstract

This invention provides a lightweight fundus color image classification method based on factorized space and hierarchical attention, belonging to the field of medical image processing technology. The method includes: acquiring fundus color images; inputting them into a lightweight classification network, which contains multiple feature extraction blocks sequentially composed of a factorized self-attention space module and a hierarchical channel enhancement module; and finally outputting the classification result. The factorized self-attention space module captures global spatial dependencies with linear complexity by decomposing two-dimensional spatial attention into vertical and horizontal directions for sequential computation; the hierarchical channel enhancement module performs hierarchical filtering and enhancement of channel information through partial convolution, channel splitting, and depthwise convolution fusion operations. This invention effectively solves the problem of existing technologies struggling to balance lightweight models and global context modeling. While maintaining low parameter count and low computational overhead, it achieves high accuracy on various retinal disease classification tasks and is suitable for deployment on resource-constrained mobile terminals.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a lightweight fundus color image classification method based on factorized space and hierarchical attention. Background Technology

[0002] Utilizing computer vision and deep learning technologies to automatically classify fundus images to assist doctors in large-scale screening and analysis of ophthalmic diseases has become an important research direction.

[0003] In recent years, deep learning models such as Convolutional Neural Networks (CNNs) and Transformers have achieved remarkable results in medical image analysis, enabling them to identify lesion patterns in fundus images with near-expert-level accuracy, significantly improving analysis efficiency and precision. Lightweight CNN architectures such as MobileNet, ShuffleNet, and GhostNet, after improvements, have demonstrated excellent performance in tasks such as fundus image disease classification and blood vessel segmentation. Furthermore, multi-instance learning methods based on pre-trained Vision Transformers (ViTs) have been used to improve the grading accuracy of diabetic retinopathy. Hybrid models combining CNNs and Transformers, such as TransUNet and MedViT, balance local details and global information, achieving performance superior to traditional CNNs in fundus image analysis.

[0004] Despite the numerous deep learning algorithms applied to retinal image analysis, achieving both lightweight models and effective modeling of global context remains a challenge. Traditional CNNs, limited by the receptive field of their convolutional kernels, struggle to capture long-range dependencies between discrete lesions in retinal images (such as microaneurysms in diabetic retinopathy). To alleviate this, some studies have introduced attention mechanisms into the CNN backbone to enhance key feature extraction. For example, CABNet highlights lesion region features by embedding channel and spatial attention modules; Mtanet constructs a multi-task attention network to collaboratively handle classification and segmentation tasks. While these methods improve attention to important regions to some extent, they still lack the ability to explicitly model long-range dependencies.

[0005] In contrast, Transformer-based models directly capture global feature associations through self-attention mechanisms. For example, the MIL-VT model, pre-trained on a large-scale fundus database and fine-tuned to achieve multi-instance learning, outperforms traditional CNNs in classification. Furthermore, some hybrid architectures attempt to combine the advantages of convolution and Transformers: the Pyramid Vision Transformer (PVT) applies self-attention to downsampled feature maps using a pyramid structure, balancing long-range relationships and computational efficiency; CoaT fuses two types of operations—convolutional positional encoding and multi-scale encoders; and Robust ViT introduces convolution into a feedforward network to enhance local robustness. In the field of medical imaging, FunSwin improves the Swin Transformer using translation window attention for diabetic retinopathy lesion grading, while MedViT optimizes its architecture for medical image classification tasks.

[0006] The aforementioned existing technologies demonstrate that combining local convolutional feature extraction with a global attention module can construct models that are both expressive and efficient. However, most existing hybrid models only introduce Transformer modules into deeper layers of the network, which may fail to fully utilize early, fine-grained features or introduce new information bottlenecks. Furthermore, the high data and computational resource requirements of Transformer models remain, limiting their direct application in high-resolution medical images and on resource-constrained devices. Therefore, designing a lightweight model that combines the efficiency of CNNs with the global awareness capabilities of Transformers under limited computational resources remains a pressing technical challenge in this field.

[0007] In summary, existing deep learning techniques for retinal image classification have the following main shortcomings:

[0008] Local field of view limitations: Traditional CNN models are limited by the receptive field of convolution, making it difficult to effectively capture the global correlations between large-scale, discretely distributed lesions (such as microhemorrhages, macular degeneration patches, etc.) in retinal images, affecting the accurate assessment of disease severity. Even with the introduction of conventional attention mechanisms, CNNs still lack the ability to explicitly model long-distance dependencies.

[0009] High model complexity: While the pure Transformer architecture can acquire global information, its self-attention computational complexity increases quadratically with image size, making it time-consuming and resource-intensive when processing high-resolution fundus images. Although some hybrid models reduce the computational burden through methods such as local window attention, they may still weaken the global perception capability. In addition, large Transformer models rely on a large amount of labeled data, making them difficult to widely apply given the high cost of medical image annotation.

[0010] The challenge lies in balancing accuracy and efficiency: existing methods often require trade-offs between these two factors. CNN models are computationally efficient but neglect global context; Transformer models possess global modeling capabilities but are difficult to deploy in real-time in resource-constrained environments. Improved versions of lightweight networks (such as MobileNet and ShuffleNet) have fewer parameters, but their classification performance improvement is limited due to the lack of an effective global feature extraction mechanism; while high-precision models introduce complex attention modules to capture global information, leading to a significant increase in parameters and computational cost, making them unsuitable for mobile terminals or portable devices. Therefore, there is currently a lack of a comprehensive solution that can simultaneously and efficiently extract local details, capture global lesion patterns, and maintain a lightweight model. Summary of the Invention

[0011] To address these issues, this invention provides a lightweight fundus color image classification method based on factorization space and hierarchical attention, which solves the problems in existing technologies such as the difficulty of convolutional neural networks in capturing global context, the high computational complexity of Transformer models, and the difficulty in achieving lightweight model deployment while maintaining high accuracy.

[0012] To address the aforementioned technical problems, embodiments of the present invention provide a lightweight fundus image classification method based on factorized space and hierarchical attention, the method comprising:

[0013] Obtain color images of the fundus to be classified;

[0014] The image is input into a trained lightweight classification network for feature extraction and classification. The lightweight classification network contains multiple sequentially connected feature extraction stages, and at least one feature extraction stage contains at least one feature extraction block.

[0015] Output the classification prediction results generated by the lightweight classification network;

[0016] The feature extraction block includes, in sequence, a factorized spatial attention module and a hierarchical channel enhancement module;

[0017] The factorized spatial attention module is used to perform spatial dimension decomposition attention calculation on the input feature map in order to capture long-range spatial dependencies in the image.

[0018] The hierarchical channel enhancement module is used to filter and enhance the channel dimensions of the feature map after it has been processed by the factorized spatial attention module, so as to improve the discriminativeness of the features.

[0019] Preferably, the process by which the factorized spatial attention module performs spatial dimension decomposition-based attention calculation on the input feature map includes:

[0020] The two-dimensional spatial attention calculation is decomposed into vertical direction attention calculation and horizontal direction attention calculation, which are performed sequentially.

[0021] The vertical attention calculation is used to aggregate global information of the input feature map in the column direction;

[0022] The horizontal attention calculation is used to aggregate global information of the feature map after the vertical attention calculation in the row direction.

[0023] Preferably, the vertical attention calculation and the horizontal attention calculation are approximated by depth-separable convolution operations;

[0024] The vertical attention calculation is performed using a size of The convolutional kernel is used to implement the horizontal attention calculation, which is performed using a kernel of size 1. Convolution kernel implementation, and This refers to the adjustable kernel size parameter.

[0025] Preferably, the factorized spatial attention module further includes a dual-path gating mechanism for filtering the attention calculation results;

[0026] The dual-path gating mechanism includes: inputting the attention calculation results into the first path and the second path respectively, wherein the first path uses the ReLU activation function to generate a sparse response and the second path uses the Sigmoid activation function to generate smooth weights; and then weighting and fusing the outputs of the two paths to highlight salient features and suppress noise.

[0027] Preferably, the process by which the hierarchical channel enhancement module filters and enhances the channel dimensions of the feature map processed by the factorized spatial attention module includes:

[0028] Partial convolution step: Perform partial channel convolution on the input feature map to perform preliminary channel filtering;

[0029] Channel expansion and grouping steps: Expand the channel dimension of some convolutional features, and divide the expanded feature map into the first group and the second group of features along the channel dimension;

[0030] Feature correction steps: Perform depthwise convolution on the second set of features to extract spatial structure information, and fuse the result with the first set of features;

[0031] Channel compression step: The fused features are reduced in dimensionality by channel, and the output is an enhanced feature map with the same number of input channels.

[0032] Preferably, the lightweight classification network adopts a multi-stage pyramid structure, including sequentially connected embedding layers and N feature extraction stages, where N is an integer greater than 1;

[0033] The embedding layer is used to perform preliminary downsampling and channel mapping on the input fundus color image;

[0034] In addition to the first feature extraction stage, each feature extraction stage is preceded by a feature merging layer, which is used to perform spatial downsampling and channel expansion on the output feature map of the previous stage.

[0035] Preferably, the process of generating the classification prediction result includes:

[0036] The output feature map from the last feature extraction stage is processed by global average pooling to obtain a global feature vector, which is then input into a fully connected layer composed of multilayer perceptrons, and the output is the classification prediction result.

[0037] This invention also provides a lightweight fundus image classification system based on factorized space and hierarchical attention. This system is used to implement the aforementioned lightweight fundus image classification method based on factorized space and hierarchical attention, specifically including:

[0038] The image acquisition module is used to acquire fundus color images to be classified;

[0039] A lightweight classification network module is used to input the image into a trained lightweight classification network for feature extraction and classification. The lightweight classification network includes multiple sequentially connected feature extraction stages, and at least one feature extraction stage includes at least one feature extraction block.

[0040] The result output module is used to output the classification prediction results generated by the lightweight classification network;

[0041] The feature extraction block includes, in sequence, a factorized spatial attention module and a hierarchical channel enhancement module;

[0042] The factorized spatial attention module is used to perform spatial dimension decomposition attention calculation on the input feature map in order to capture long-range spatial dependencies in the image.

[0043] The hierarchical channel enhancement module is used to filter and enhance the channel dimensions of the feature map after it has been processed by the factorized spatial attention module, so as to improve the discriminativeness of the features.

[0044] This invention also provides an electronic device, which includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the lightweight fundus color image classification method based on factorized space and hierarchical attention described above.

[0045] This invention also provides a computer storage medium storing a computer software product, the computer software product including several instructions to cause a computer device to execute the aforementioned lightweight fundus color image classification method based on factorized space and hierarchical attention.

[0046] As can be seen from the above technical solutions, this invention application has the following beneficial effects:

[0047] (1) High-discriminative feature extraction with both global semantics and local fine detail: The introduction of the FSAS module enables the network to effectively model the global spatial dependencies of fundus images. Ablation experiments on the ROP dataset show that the model accuracy decreased significantly by 5.26% and the F1 score decreased by 5.38% after removing the FSAS module, fully demonstrating the importance of global spatial information in identifying discrete lesions and the overall degree of lesions. On the other hand, the HCE module selectively enhances feature channels, which helps the model focus on key disease-related information. Its contribution was also demonstrated in the ablation experiments: the accuracy decreased by 2.63% after removing the HCE module, indicating that refining channel features can improve classification discriminative power. By combining the FSAS and HCE modules sequentially, each ASC block first extracts global spatial associations and then optimizes channel details. The features extracted by the network have both macroscopic layout and microscopic detail discriminative power. This makes the method of this invention exhibit stronger feature recognition ability and higher classification accuracy in complex fundus color image classification tasks.

[0048] (2) Lightweight and efficient model, suitable for deployment in resource-constrained environments: This invention significantly reduces the model size while ensuring performance through innovative designs such as factorized attention and partial convolution. The ASCN model of this invention has only about 2.76M parameters and a computational cost of only 1.18 GFLOPs. In contrast, many existing models often have tens of millions of parameters, such as OverLoCK-T with as many as 33.46M parameters and a computational cost of 5.90 GFLOPs. The ASCN model achieves comparable or even better classification performance with less than one-tenth of their parameters. For example, on the retinopathy of prematurity (ROP) grading dataset, the ASCN model achieves a classification accuracy of 86.84% and an F1-score of 86.81%, both of which are currently the best levels, while the number of parameters of the ASCN model is only 1 / 10 of the best CNN model. The design that combines high accuracy and low complexity makes the model of this invention suitable for deployment on mobile devices or portable medical terminals for large-scale vision screening and remote medical auxiliary analysis. The lightweight design of the model significantly reduces the pressure on storage and real-time inference, which is conducive to its widespread adoption in practical applications.

[0049] (3) Strong adaptability and robustness to multiple diseases: The method of this invention has achieved excellent results on multiple public (such as ROP, Eye-Fundus, and DDR) and private (RP) retinal disease fundus color image datasets, demonstrating the universality and robustness of the method. Except for the ROP grading, on the Eye-Fundus multi-class fundus disease dataset, the ASCN model achieves an accuracy of approximately 90.07% and a weighted Kappa value of 94.05%, outperforming most comparative methods. In the DDR dataset for diabetic retinopathy grading, the ASCN model achieves an accuracy of 85.09% and a Kappa value of 86.84%, close to the current best model, but with fewer parameters. In the private retinitis pigmentosa (RP) dataset, the ASCN model achieves an average accuracy of 83.75% ± 4.28% through five-fold cross-validation, an improvement of 4-5 percentage points compared to traditional CNN models. These results indicate that the method of this invention is not only applicable to the classification of a single disease, but also performs well in the classification of multiple retinal diseases, demonstrating strong practical value. Meanwhile, the attention mechanism of the network structure enables it to exhibit robust adaptability to fundus images of different image qualities and under different acquisition devices. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Referring to the drawings will make the features and advantages of the present invention clearer. The drawings are illustrative and should not be construed as limiting the present invention in any way. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0051] Figure 1 This is a flowchart of a lightweight fundus color image classification method based on factorization space and hierarchical attention provided by the present invention;

[0052] Figure 2 This is a schematic diagram of the overall structure of the adaptive sparse convolutional network model in this invention;

[0053] Figure 3 This is the factorized self-attention space module in this invention;

[0054] Figure 4 This is the layered channel enhancement module in this invention;

[0055] Figure 5 This is the Grad-CAM visualization heatmap in this invention;

[0056] Figure 6 This is a block diagram of a lightweight fundus color image classification system based on factorization space and hierarchical attention provided by the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Example 1: To address the problems in existing technologies, such as the difficulty of convolutional neural networks in capturing global context, the high computational complexity of Transformer models, and the difficulty in achieving lightweight model deployment while maintaining high accuracy, such as... Figure 1 As shown, this invention proposes a lightweight fundus color image classification method based on factorized space and hierarchical attention. The method includes:

[0059] S1: Obtain the fundus color image to be classified;

[0060] S2: Input the image into a trained lightweight classification network for feature extraction and classification. The lightweight classification network contains multiple sequentially connected feature extraction stages, and at least one feature extraction stage contains at least one feature extraction block.

[0061] S3: Output the classification prediction results generated by the lightweight classification network;

[0062] The feature extraction block includes, in sequence, a factorized spatial attention module and a hierarchical channel enhancement module;

[0063] The factorized spatial attention module is used to perform spatial dimension decomposition attention computation on the input feature map to capture long-range spatial dependencies in the image.

[0064] The hierarchical channel enhancement module is used to filter and enhance the channel dimensions of the feature map after it has been processed by the factorized spatial attention module, so as to improve the discriminativeness of the features.

[0065] As can be seen from the above technical solution, this invention proposes a lightweight fundus color image classification method based on factorized spatial and hierarchical attention. This method first acquires fundus color images to be classified, providing high-quality input data for subsequent analysis. Then, the images are input into a trained lightweight classification network for feature extraction and classification. This network progressively extracts multi-scale features through multiple sequentially connected feature extraction stages. Each feature extraction block sequentially includes a factorized spatial attention module and a hierarchical channel enhancement module. The factorized spatial attention module effectively captures long-range spatial dependencies in the image by performing spatial dimension decompositional attention calculations on the feature map, overcoming the limitation of the limited receptive field in traditional convolutional neural networks. The hierarchical channel enhancement module further filters and enhances the feature channels, suppressing redundant information and highlighting discriminative features, thereby further improving the model's ability to identify lesion areas. Finally, the network outputs classification prediction results, achieving accurate and efficient identification of disease types in fundus images. While ensuring high accuracy, it significantly reduces model complexity, making it suitable for resource-constrained clinical deployment environments.

[0066] Combination Figure 2 The implementation steps of the method of the present invention are described in detail below, including the parameters, operational details, and logical relationships of each step:

[0067] In step S1, fundus color images to be classified are obtained.

[0068] In step S2, the fundus color image is input into a trained lightweight classification network for feature extraction and classification. The lightweight classification network contains multiple sequentially connected feature extraction stages, and at least one feature extraction stage contains at least one feature extraction block.

[0069] Specifically, this invention designs an Adaptive Sparse Convolutional Network (ASCN) architecture as a lightweight classification network. Figure 2 This is a schematic diagram of the overall structure of the ASCN model of the present invention. The architecture integrates the Factorized Self-Attentive Spatial module (FSAS module) and the Hierarchical Channel Enhancement module (HCE module), which are stacked alternately to extract features for the classification of retinal fundus color images.

[0070] like Figure 2 As shown, the entire network adopts a four-stage hierarchical feature extraction structure, progressively extracting multi-scale feature maps from shallow to deep. The input fundus color image to be classified is first downsampled through an embedding layer. Then, the first feature extraction stage expands the number of channels. The subsequent second to fourth feature extraction stages respectively use feature merging layers (3×3 convolution, stride 2) to halve the spatial size and expand the number of channels. The specific process is as follows:

[0071] Input and initial embedding: Input is a fundus color image to be classified. The size is ,in and Let be the height and width of the image. First, the image is divided into non-overlapping patches and mapped to the feature space using an embedding layer. This is equivalent to a convolution operation with a stride of 4. The resulting initial feature map is denoted as . , This represents the initial number of channels.

[0072] First feature extraction stage: ... enter Each cascaded adaptive sparse convolutional block (ASC block, i.e., a basic unit containing both FSAS and HCE modules) extracts features layer by layer. The output feature map after the first feature extraction stage is as follows: ,in This represents the number of channels.

[0073] Second to fourth feature extraction stages: Before the second feature extraction stage begins, first... A feature merging layer is applied for downsampling, and a convolution with a stride of 2 is used to halve the feature map size. At the same time, expand the channel to Then send it in Each ASCII block receives output. Similarly, the third and fourth feature extraction stages perform merged downsampling before they begin, and each feature extraction stage undergoes... , Processing each ASC block to generate feature maps. and ,in , This represents the number of channels.

[0074] Classified output: Finally, The global feature vector is obtained by global average pooling, input into a fully connected layer composed of a multilayer perceptron (MLP), and outputs the classification prediction result.

[0075] Furthermore, the design and working principle of two key modules (FSAS module and HCE module) within the ASC block are explained in detail below:

[0076] (1) Factorized Self-Attention Space Module (FSAS Module): The structure of the FSAS module is as follows Figure 3 As shown, this approach aims to obtain global spatial feature relationships with low computational cost by effectively modeling long-range dependencies between different locations in an image through decompositional self-attention. Traditional global self-attention has a spatial complexity of O(n log n) for the input feature map. (in and (representing the height and width of the input feature map, respectively), making it difficult to directly apply to high-resolution images. The FSAS module designed in this invention decomposes two-dimensional spatial attention into two one-dimensional sequential attentions, calculated in two steps along the vertical and horizontal directions. This ensures that any two pixels can establish a connection through two interactions, while reducing the complexity to linear levels. The specific implementation is as follows:

[0077] Vertical attention (column direction): Let the input feature map be... ( (representing the number of channels), first aggregate the vertical information in each column, and then calculate the intermediate features. Output location intermediate features at the point Equal to input In the The weighted sum of all row features in a column, which represents global information integration along the column direction, can be expressed as:

[0078] ,

[0079] in, For learnable column-direction attention weight matrix, It is the summation index used to iterate through rows.

[0080] Horizontal attention (row direction): Next, the intermediate features... Aggregating along the horizontal direction yields the output features of the FSAS module. . equal In the The weighted sum of all column features in a row, which achieves global information integration in the row direction, can be expressed as:

[0081] ,

[0082] in, This is a learnable row direction attention weight matrix. It is a summation index used for iterating through columns.

[0083] The FSAS module output features are obtained after two steps: vertical attention and horizontal attention. It contains feature information from the global scope of the original image.

[0084] Convolution Approximation Implementation: In practice, to improve efficiency, the FSAS module uses depthwise separable convolution (SConv) to approximate the attention operation described above. Specifically, it utilizes two asymmetric depthwise convolution kernels in two directions, with sizes of... and Sequential application to the feature map approximates both vertical and horizontal attention, where, Indicates the vertical convolution kernel size. This represents the horizontal convolution kernel size. This not only decouples computation from feature map size but also allows for flexible adjustment of the convolution kernel size based on the feature map size. and This controls the attention span. The computational cost of the FSAS module implemented using convolution is approximately... Far below standard self-attention .

[0085] Dual-path gating mechanism: Since fundus color images contain large areas of clinically insignificant background, applying attention to all regions indiscriminately could introduce irrelevant noise, leading to decreased model performance. Therefore, the FSAS module calculates the global feature map... Then, a gating unit is introduced to filter the output features, highlighting salient features and suppressing useless information. The gating unit adopts a dual-path filtering mechanism: the first path uses the ReLU activation function to truncate negative values ​​to generate a sparse response, and the second path uses the Sigmoid activation function to generate smooth weights between 0 and 1 to preserve overall information. The outputs of the two paths are then weighted and fused according to their respective importance. This can be expressed by the following formula:

[0086] ReLU path: ,Will Negative values ​​are set to 0, while positive responses are retained.

[0087] Sigmoid path: ,right By taking the Sigmoid function element by element, we obtain an attention map between [0,1].

[0088] Fusion Output: .

[0089] in , For learnable weights, Softmax normalization is used to satisfy... . Specifically, , The parameter is initialized to 1.

[0090] Through the aforementioned dual-path gating, the FSAS module effectively filters out noise introduced in the global attention calculation while retaining key lesion features. The dual-path mechanism ensures that the FSAS module highlights salient patterns without becoming excessively sparsity-based, thus improving the effectiveness of global features for classification.

[0091] (2) Hierarchical Channel Enhancement Module (HCE Module): The HCE module follows the FSAS module, and its structure is as follows: Figure 4 As shown, this is used to further filter and enhance the feature map after spatial attention in the channel dimension. Convolutional features in fundus color images often contain high-dimensional channels, including both discriminative features closely related to lesions and a considerable amount of redundant information. Indiscriminately utilizing all channels may affect the final classification performance. Therefore, the HCE module employs a step-by-step filtering and enhancement strategy, selectively highlighting useful channel features and suppressing irrelevant channels through partial convolution and depthwise convolution operations, achieving a refined reorganization of channel information. The structure and processing flow of HCE are as follows (see appendix). Figure 2 (Schematic diagram of HCE module structure)

[0092] Partial convolutional filtering: Given an input feature map (in , and The height, width, and number of channels of the input feature map are represented by , respectively. First, a partial convolution (PConv) operation is performed on it. Partial convolution is a sparse channel convolution that performs convolution calculations only on a subset of the input channels, leaving the remaining channels unchanged. This sparse operation is equivalent to a preliminary screening along the channel dimension, extracting only potentially important channel combinations, thus reducing interference with subsequent processing. Next, the output features of PConv are expanded in channel dimension by a standard 1×1 convolution and then non-linearly activated using ReLU to obtain an intermediate feature map. :

[0093] ,

[0094] Among them, the features generated by the PConv operation are first processed Convolution restores or expands the number of channels from a selected subset to a larger subset. Then, ReLU activation is applied to introduce nonlinearity. This process is similar to "compression followed by expansion" in the inverse residual structure, which not only filters out unnecessary channel information but also provides a richer dimensional space for subsequent operations.

[0095] Channel grouping and depthwise convolution enhancement: enhancing intermediate feature maps Divided into two groups along the passage (in , ). For the second set of features A depthwise convolution operation is performed, with spatial filtering applied independently to each channel to extract the spatial patterns of the features in that group. The output of the depthwise convolution is then compared with the first group of features. The fusion feature is obtained by adding elements one by one according to their corresponding positions. :

[0096] ,

[0097] in, This represents element-wise addition. This indicates a depthwise convolution operation.

[0098] This step can be viewed as an "information correction" process: through depthwise convolution from The extracted features (reflecting the important spatial structure of some channels) are used to adjust To supplement or emphasize Missing discriminant information. This design makes channel enhancement hierarchical: intermediate features are first obtained through PConv coarse screening and expansion. Furthermore, by splitting the channels, one part of the channels guides the optimization of another part of the channels, which reflects the interaction of channel features at different levels. Therefore, it is called "layered" channel enhancement.

[0099] Channel compression output: Finally, the fused features are... Through a Convolution performs channel dimensionality reduction, projecting back to the original number of channels. The output of the HCE module is obtained. :

[0100] .

[0101] At this point, the HCE module has completed the filtering and enhancement of the input features in the channel dimension, and outputs features with the same input dimension but more refined information and stronger discriminative power.

[0102] The HCE module, through the cascaded operations of "partial convolution + channel splitting + depthwise convolution," selectively preserves effective channels and enhances the hierarchy of key features, thereby significantly improving the discriminative power of feature representation. It complements the global spatial information provided by the FSAS module: while the output features of the FSAS module contain global information but may contain redundancy, the HCE module further filters and optimizes this information, ensuring that the features output by the ASC module are both globally focused and highly refined.

[0103] In step S3, the classification prediction results generated by the ASCN model are output.

[0104] In summary, the ASCN model of this invention uses FSAS and HCE modules as basic units (ASC blocks) stacked layer by layer, extracting multi-scale features through a four-level feature pyramid structure. The FSAS module solves the problem of limited local field of view in CNNs by injecting global spatial dependency information at the beginning of each feature extraction stage; the HCE module optimizes channel information, highlighting key lesion features. The two modules are combined sequentially, fully leveraging the synergistic effect of "spatial first, channel later" attention. The entire network achieves efficient capture and classification of retinal lesion patterns while maintaining low parameter and computational costs. Therefore, the technical solution provided by this invention can achieve classification performance comparable to or even better than large-scale models with lower storage and computational costs, possessing good practical value and promising prospects for widespread application.

[0105] To further illustrate the advantages of the method of the present invention, the following explanation is based on specific simulation verification.

[0106] The experiments include dataset introduction, evaluation metrics, comparative experimental results and analysis, and ablation experimental results and analysis. These are described below:

[0107] 1. Dataset

[0108] (1) ROP public dataset (Retinopathy of Prematurity (ROP) grading: This dataset contains 1099 color images of the retinas from 483 preterm infants. The images are divided into 5 categories: normal retinas (236 images), ROP stage 1 (94 images), ROP stage 2 (165 images), ROP stage 3 (261 images), and ROP cases treated with laser therapy (343 images). The number of images in each category in the dataset is unbalanced; for example, there are more images in the normal and laser-treated categories, while there are only 94 images in stage 1. The dataset is divided into training, validation, and test sets in a 7:2:1 ratio.

[0109] (2) Eye-Fundus multi-disease public dataset: This public dataset contains 10,670 eye images covering 10 retinal health conditions, including: diabetic retinopathy (DR), glaucoma, macular scarring, optic disc edema, central serous chorioretinopathy (CSCR), retinal detachment, retinitis pigmentosa (RP), high myopia, pterygium, and normal retina. Pterygium is an anterior segment eye disease, with only 34 images, and this class was not used in the experiments of this invention. The dataset is divided into a training set of 70%, a validation set of 20%, and a test set of 10%. This dataset is large in scale and has many categories, making it an important test of the model's multi-classification ability and generalization.

[0110] (3) DDR public dataset (diabetic retinopathy grading): contains 12,522 fundus color images, with image annotations divided into 5 grades: no DR (6,266 images), mild DR (630 images), moderate DR (4,477 images), severe DR (236 images), and proliferative DR (913 images). The dataset is divided into training, validation, and test sets in a 7:2:1 ratio. The class distribution of this dataset is extremely imbalanced (severe cases are rare), posing a challenge to the robustness of the model.

[0111] (4) RP Private Dataset (Retinitis Pigmentosa Grading): Contains 596 retinal color images, labeled into four grades: normal (300 images), early RP (70 images), intermediate RP (138 images), and late RP (88 images). Due to the small size and imbalanced class of this dataset, five-fold cross-validation was used to evaluate the model performance. Experiments on this private dataset can verify the model's adaptability to different disease types.

[0112] 2. Evaluation Indicators

[0113] To comprehensively measure classification performance, the experiment employed multiple evaluation metrics, primarily including: classification accuracy (Accuracy, Acc), F1 score, specificity (Spe), and quadratic weighted kappa (QWK). In addition, an average metric was calculated, which is the average of QWK, F1, and Spe. This approach integrates the biases of individual metrics to provide an overall score for model performance. Especially in cases of class imbalance, focusing solely on accuracy might mask the model's bias towards the majority class; introducing QWK and the macro-average metric allows for a fairer evaluation of the model.

[0114] 3. Comparison and Analysis of Experimental Results

[0115] To comprehensively evaluate the performance of the ASCN model, a variety of advanced CNN and Transformer models were selected for comparative experiments against the ASCN model of this invention. These included lightweight models and models specifically designed for medical imaging, such as classic CNN models like ResNet18, MobileNetV2, and EfficientNet; CNN models improved for medical tasks like CABNet and nnMobileNet; and the latest CNN model, OverLoCK-T. Additionally, Transformer or hybrid models such as PvT-Tiny, CoaT, RvT, MedViT, BiFormer, and ViT were also included. Experiments were conducted on four datasets, and the results are as follows:

[0116] (1) Comparative experimental results on the ROP dataset: Table 1 below lists the main metrics of the ASCN model and other methods in the five-class classification on the ROP dataset. The ASCN model achieved the highest accuracy of 86.84% and the highest F1 score of 86.81%, with QWK reaching 96.57% and specificity of 96.81%, all of which are currently the best. The overall average score of the ASCN model is 93.40%, significantly higher than other comparative methods. In particular, compared with the OverLoCK-T model with 33.46M parameters, the QWK of the ASCN model is 0.17 percentage points higher, but the number of parameters is only about 1 / 12 of that of the OverLoCK-T model, and the computational cost is only 1.18 GFLOPs. The experimental results show that the ASCN model of this invention maintains or even improves the classification performance while significantly reducing the model complexity. Compared to other CNN models such as ResNet18, MobileNetV2, and EfficientNet, the ASCN model exhibits significant advantages in accuracy and Kappa scores, while also being smaller in size, demonstrating a high performance and efficiency that traditional CNN models struggle to achieve simultaneously. Compared to Transformer models, the ASCN model also offers comprehensive advantages: for example, compared to BiFormer, the ASCN model achieves an average score of 93.40%, with less than half the number of parameters; compared to PvT, CoaT, and ViT, the ASCN model shows significant superiority across all evaluation metrics, demonstrating the superiority of the ASCN model's structural design in the ROP (Regression on Optimization) grading task.

[0117] Table 1 Comparative experimental results on the ROP dataset

[0118]

[0119] (2) Comparative experimental results on the Eye-Fundus dataset: Table 2 below shows the performance of each model in the nine-class classification on the Eye-Fundus dataset. The ASCN model achieved an accuracy of 90.07% and a macro-average F1 score of 90.08%, with a QWK of 94.05% and a specificity of 98.70%, achieving the highest average score of 94.28% among all models. The ASCN model outperformed the CNN model in all metrics. For example, CABNet, an attention network designed specifically for the DR task, had a QWK of 93.53%, slightly lower than the ASCN model's 94.05%, while CABNet had as many as 26.15M parameters. Among the Transformer series models, PvT-Tiny had similar accuracy and specificity to the ASCN model, but its QWK was slightly lower (93.74% vs 94.05%), and its overall average score was lower than the ASCN model. Experimental results show that the ASCN model of this invention remains competitive for complex multi-class classification tasks. It not only has high accuracy but also alleviates the class imbalance problem and has advantages in metrics such as Kappa. This is also due to the global perspective provided by the FSAS module, which enables the model to compare the importance of lesion features across classes, while the channel filtering enhancement of the HCE module ensures that the key features of each class are not submerged in the information of other classes.

[0120] Table 2 Comparative experimental results on the EyeFundus dataset

[0121]

[0122] (3) Comparative experimental results on the DDR dataset: Table 3 below shows the five-class classification performance of each model on the diabetic retinopathy DDR dataset. The ASCN model achieved the best accuracy (85.09%), F1 score of 83.91%, QWK score of 86.84%, and average score of 88.66%. Due to the extreme imbalance of classes in the DDR dataset, some models experienced significant performance fluctuations in a few classes. MedViT achieved a slightly higher average score of 89.01%, and BiFormer achieved the highest QWK score of 87.55%, but both had a larger number of parameters than the ASCN model. The ASCN model of this invention achieves a better balance between efficiency and performance.

[0123] Table 3 Comparative experimental results on the DDR dataset

[0124]

[0125] (4) Comparative Experiment Results on the RP Dataset: Table 4 below shows the four-class classification performance of each model on the RP dataset using five-fold cross-validation. The ASCN model achieved an average accuracy of 83.75% ± 4.28% and a macro-average F1 score of 83.40% ± 4.77%, outperforming other comparative methods. In terms of QWK, PvT-Tiny performed slightly better, but the ASCN model outperformed PvT and had fewer model parameters. Compared with lightweight CNNs such as ResNet18 and MobileNetV2, the ASCN model achieved 4.59% and 4.42% higher accuracy, respectively. The RP dataset presents challenges for model feature extraction and accurate classification due to its unique and dispersed lesion features (pigmentation, retinal degeneration). The model of this invention significantly outperforms traditional methods in this task, further demonstrating that the combination of the FSAS and HCE modules endows the network with the ability to comprehensively distinguish different types of lesions.

[0126] Table 4 Comparative experimental results on the RP dataset

[0127]

[0128] 4. Ablation Experiment Results and Analysis

[0129] To verify the effectiveness of each module and design strategy, the present invention conducted the following ablation experiments on the ROP dataset, as shown in Table 5 below, and performed Grad-CAM visualization, as follows. Figure 5 As shown, (a) presents the original fundus image. (b) through (h) show the attention distribution of different model variants: (b) without the FSAS module, (c) without the HCE module, (d) FSAS and HCE modules in reverse order, (e) DWConv in reverse order in Sconv, (f) standard self-attention, (g) without bipath filtering, and (h) the complete ASCN model. (h) shows more focused and clinically relevant activation.

[0130] Table 5 Ablation experimental results on the ROP dataset

[0131]

[0132] (1) Removal of FSAS module (w / o FSAS): The FSAS module is completely removed from the ASC block, leaving only the HCE module. The model performance deteriorated significantly, with accuracy decreasing by 5.26%, F1 score by 5.38%, weighted Kappa score by 2.72%, and specificity by 1.26%, validating the crucial role of the FSAS module in extracting global spatial information. The model without the FSAS module cannot effectively model long-distance correlation features in fundus color images, leading to increased class confusion.

[0133] (2) Removing the HCE module (w / o HCE): That is, after each ASC block outputs only the FSAS module, the HCE module is skipped directly. The model performance also decreased, with accuracy decreasing by 2.63%, F1 by 2.08%, Kappa by 1.23%, and specificity by 0.56%. The experimental results show that the HCE module plays an important role in refining discriminative features. Without the HCE module, redundant information and noise in the feature channels cannot be effectively filtered, resulting in a decrease in classification performance.

[0134] (3) Swapping the order of the FSAS and HCE modules (FSAS-HCE order reversal): The module order within the ASC block was changed to HCE first, then FSAS. This resulted in a decrease in model performance, with accuracy decreasing by 3.51%, F1 score by 3.32%, Kappa score by 0.69%, and specificity by 0.76%. Experimental results indicate that the order of spatial attention first, followed by channel enhancement, is more reasonable. This is because the FSAS module extracts global features first, ensuring that the HCE module selects effective channels within the global context. If the order is reversed, the HCE module may not select channel features accurately when lacking global information, and the global attention of the FSAS module may also be affected by channel selection, resulting in the loss of some global information and thus poor overall performance.

[0135] (4) Reversal of the depthwise convolution order within the FSAS module: The spatial convolution in the FSAS module is essentially composed of two depthwise convolutions, one vertical and one horizontal. Experiments tested the swapping of the order of these two convolutions (i.e., horizontal convolution before vertical convolution). Experimental results show that the performance of this variant, where horizontal convolution precedes vertical convolution, is very close to the performance of the default FSAS module where vertical convolution precedes horizontal convolution, indicating that the order of vertical-horizontal convolution within FSAS has little impact on model performance. To maintain consistency, this invention adopts a "vertical convolution before horizontal" design.

[0136] (5) Replace the FSAS module with standard self-attention: Replace the FSAS module in each ASC block with a standard global self-attention module (which has a higher computational cost). Experimental results show that the model classification performance is not significantly improved, but the number of parameters and computational cost increase significantly, which is not conducive to lightweight design. This verifies that the FSAS module of this invention has the advantage of balancing efficiency and performance: the decomposed attention using convolution approximation is comparable to explicit self-attention in performance, but significantly reduces the computational cost.

[0137] (6) Removing the dual-path gating of the FSAS module: This explores how removing the ReLU and Sigmoid dual-path filtering in the FSAS module and retaining only a single path affects model performance. Experimental results show that the classification accuracy of the model lacking the dual-path gating mechanism decreases, and in cases such as Figure 5In the Grad-CAM visualization shown, the attention heatmap focuses too much on a single region, missing some key lesion structures. This indicates that dual-path gating can guide the model to pay balanced attention to multiple important regions. If this mechanism is removed, the features generated by the FSAS module may become overconfident (overly high-response) in some locations, ignoring information from other regions, leading to unstable classification.

[0138] The ablation experiments above demonstrate that all the modules and design strategies proposed in this invention are necessary and effective. The arrangement of the FSAS and HCE modules within the ASC block also verifies the rationality of spatial attention followed by channel attention. Furthermore, the dual-path gating mechanism within the FSAS module enhances the robustness of feature extraction and avoids information bias that may arise from the attention mechanism.

[0139] Example 2: Figure 6 As shown, this invention provides a lightweight fundus image classification system based on factorized space and hierarchical attention. This system is used to implement the lightweight fundus image classification method based on factorized space and hierarchical attention described in Embodiment 1 above, specifically including:

[0140] Image acquisition module 100 is used to acquire fundus color images to be classified;

[0141] The lightweight classification network module 200 is used to input images into a trained lightweight classification network for feature extraction and classification. The lightweight classification network contains multiple sequentially connected feature extraction stages, and at least one feature extraction stage contains at least one feature extraction block.

[0142] The result output module 300 is used to output the classification prediction results generated by the lightweight classification network;

[0143] The feature extraction block includes, in sequence, a factorized spatial attention module and a hierarchical channel enhancement module;

[0144] The factorized spatial attention module is used to perform spatial dimension decomposition attention computation on the input feature map to capture long-range spatial dependencies in the image.

[0145] The hierarchical channel enhancement module is used to filter and enhance the channel dimensions of the feature map after it has been processed by the factorized spatial attention module, so as to improve the discriminativeness of the features.

[0146] This embodiment presents a lightweight fundus image classification system based on factorized space and hierarchical attention, used to implement the aforementioned lightweight fundus image classification method based on factorized space and hierarchical attention. Therefore, the specific implementation of the lightweight fundus image classification system based on factorized space and hierarchical attention can be found in the previous embodiment section of the lightweight fundus image classification method based on factorized space and hierarchical attention. For example, the image acquisition module 100, the lightweight classification network module 200, and the result output module 300 are respectively used to implement steps S1, S2, and S3 in the aforementioned lightweight fundus image classification method based on factorized space and hierarchical attention. Therefore, its specific implementation can be referred to the description of the corresponding embodiments. To avoid redundancy, it will not be repeated here.

[0147] Example 3: This embodiment of the invention provides an electronic device, which includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the above-mentioned lightweight fundus color image classification method based on factorized space and hierarchical attention.

[0148] Example 4: This embodiment of the invention provides a computer storage medium storing a computer software product. The computer software product includes several instructions to cause a computer device to execute the above-described lightweight fundus color image classification method based on factorized space and hierarchical attention.

[0149] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0151] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0152] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A lightweight fundus color photograph classification method based on factorized space and hierarchical attention, characterized in that, The method comprises: obtaining an eye fundus color image to be classified; inputting the image into a trained lightweight classification network for feature extraction and classification, the lightweight classification network comprising a plurality of sequentially connected feature extraction stages, at least one of the feature extraction stages comprising at least one feature extraction block; outputting a classification prediction result generated by the lightweight classification network; wherein the feature extraction block sequentially comprises a factorized spatial attention module and a hierarchical channel enhancement module; the factorized spatial attention module is configured to perform spatial dimension decomposition attention calculation on the input feature map to capture long-range spatial dependencies in the image, the decomposition attention calculation comprising decomposing two-dimensional spatial attention calculation into sequentially executed vertical direction attention calculation and horizontal direction attention calculation, the vertical direction attention calculation being configured to aggregate global information of the input feature map in the column direction, and the horizontal direction attention calculation being configured to aggregate global information of the feature map after the vertical direction attention calculation in the row direction; the hierarchical channel enhancement module is configured to filter and enhance the feature map processed by the factorized spatial attention module in the channel dimension to improve the discriminability of the feature.

2. The lightweight fundus color photograph classification method based on factorized space and hierarchical attention according to claim 1, characterized in that, The vertical direction attention calculation and the horizontal direction attention calculation are approximated by depth separable convolution operations; wherein the vertical direction attention calculation is implemented by a convolution kernel with a size of and the horizontal direction attention calculation is implemented by a convolution kernel with a size of , and are adjustable convolution kernel size parameters.

3. The lightweight fundus photograph classification method based on factorized space and hierarchical attention according to claim 1, characterized in that, the factorized spatial attention module further comprises a double-path gating mechanism for filtering the attention calculation result; the double-path gating mechanism comprises: inputting the attention calculation result into a first path and a second path respectively, the first path using a ReLU activation function to generate sparse responses, and the second path using a Sigmoid activation function to generate smooth weights; then the outputs of the two paths are weighted and fused to highlight significant features and suppress noise.

4. The lightweight fundus photograph classification method based on factorized space and hierarchical attention according to claim 1, characterized in that, The process of the hierarchical channel enhancement module for filtering and enhancing the feature map processed by the factorized spatial attention module in the channel dimension comprises: a partial convolution step: performing partial channel convolution on the input feature map to perform preliminary channel filtering; a channel expansion and grouping step: expanding the feature after the partial convolution in the channel dimension, and dividing the expanded feature map into a first group of features and a second group of features along the channel dimension; a feature correction step: performing deep convolution on the second group of features to extract spatial structure information, and fusing the result with the first group of features; a channel compression step: performing channel dimension reduction on the fused feature to output an enhanced feature map with the same number of channels as the input.

5. The lightweight fundus color photograph classification method based on factorized space and hierarchical attention according to claim 1, characterized in that, The lightweight classification network adopts a multi-stage pyramid structure, comprising an embedding layer and N feature extraction stages connected in sequence, N being an integer greater than 1; the embedding layer is configured to perform preliminary down-sampling and channel mapping on the input eye fundus color image; except for the first feature extraction stage, a feature merging layer is provided before each of the feature extraction stages, configured to perform spatial down-sampling and channel expansion on the output feature map of the previous stage.

6. The lightweight fundus photograph classification method based on factorized space and hierarchical attention according to claim 1, characterized in that, the generation process of the classification prediction result comprises: The output feature map of the last feature extraction stage is input into a global average pooling to obtain a global feature vector, which is input into a fully connected layer composed of multiple layers of perceptron to output a classification prediction result.

7. A lightweight fundus color photograph classification system based on factorized space and hierarchical attention, characterized in that, The system is used to implement the lightweight fundus color photograph classification method based on factorized spatial and hierarchical attention according to any one of claims 1 to 6, and specifically comprises: An image acquisition module is configured to acquire a fundus color photograph image to be classified. A lightweight classification network module is configured to input the image into a trained lightweight classification network to perform feature extraction and classification. The lightweight classification network comprises a plurality of sequentially connected feature extraction stages, and at least one of the feature extraction stages comprises at least one feature extraction block. A result output module is configured to output a classification prediction result generated by the lightweight classification network. The feature extraction block sequentially comprises a factorized spatial attention module and a hierarchical channel enhancement module. The factorized spatial attention module is configured to perform spatial dimension decomposition attention calculation on the input feature map to capture long-range spatial dependency in the image.

8. An electronic device, comprising: The hierarchical channel enhancement module is configured to perform channel dimension screening and enhancement on the feature map processed by the factorized spatial attention module to improve the discriminability of the feature.

9. A computer storage medium, characterized in that The electronic device comprises a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the lightweight fundus color photograph classification method based on factorized spatial and hierarchical attention according to any one of claims 1 to 6. The computer storage medium stores a computer software product, and the computer software product comprises a plurality of instructions for enabling a computer device to execute the lightweight fundus color photograph classification method based on factorized spatial and hierarchical attention according to any one of claims 1 to 6.

Images