Multi-granularity cataract classification method and system based on lesion perception

Abstract

The invention discloses a multi-granularity cataract classification method and system based on lesion perception, belongs to the field of computer vision and medical image analysis, and is characterized in that a slit lamp image is used as input data, and a lesion perception feature extraction method is adopted, so that a model can accurately focus a lesion area, and the accuracy of cataract classification is improved; a focus region with high weight features is screened out by adopting mutual attention calculation and a maximum connected region search algorithm, and it is ensured that the classification model can focus on core features of the focus region; through a fine-grained feature optimization module, in combination with up-sampling convolution, expansion convolution and jump connection strategies, fine-grained information of a lesion area is enhanced, and the capability of distinguishing different types of cataract is improved. According to the method, high-value focus information can be extracted from complex slit lamp image data, real-time and high-precision multi-granularity cataract classification is realized, clinical diagnosis is assisted, and early cataract screening and treatment efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and medical image analysis, and in particular to a multi-granularity cataract classification method and system based on lesion perception. The method utilizes deep learning technology to perform feature extraction, denoising, and classification on cataract lesions in slit lamp images, thereby improving the accuracy of automated diagnosis of cataracts and achieving cataract classification at different granularities. Background Art

[0002] Cataracts are one of the leading causes of visual impairment and blindness worldwide. Early screening and accurate classification are crucial for timely intervention and treatment. Slit lamp microscopy is a key tool in the clinical diagnosis of cataracts in ophthalmology. Physicians use slit lamp images to assess the degree of lens opacity and classify it. However, manual diagnosis not only relies on the physician's experience but is also significantly influenced by subjective factors, which can easily lead to inconsistent diagnoses.

[0003] Deep learning technology has been widely used in medical image analysis, especially methods based on convolutional neural networks and Transformers, which have been used for ophthalmic disease detection. However, multi-granularity cataract classification still faces the impact of light interference and noise, making the task complex. Summary of the Invention

[0004] In order to solve the problems of low classification accuracy in the existing technology due to light interference, difficulty in accurately locating the lesion area, and complexity of multi-granularity classification tasks, the present invention provides a multi-granularity cataract classification method and system based on lesion perception. By combining a feature extractor, a lesion-perception feature extraction module, a fine-grained feature optimization module and a classifier, it can accurately extract lesion features, remove irrelevant noise, and improve the robustness and accuracy of cataract classification.

[0005] The technical solution provided by the present invention is:

[0006] A multi-granularity cataract classification method based on lesion perception includes the following steps:

[0007] Step 1: Build a multi-granularity cataract classification network model based on lesion perception to enhance the network's ability to extract and discriminate multi-granularity features of cataracts in slit lamp images;

[0008] Step 2: Train the multi-granularity cataract classification network model. The process is as follows:

[0009] Step 2.1: Prepare a slit lamp cataract image and resize it to N × N pixels. The image is then fed into the feature extractor. The image then passes through the lesion-aware feature extraction module, the fine-grained feature optimization module, and the classifier.

[0010] Step 2.2: Use the pre-trained model as the initial weights and perform multiple rounds of iterative training. During the training process, a linear warm-up strategy will be used for the first 20 rounds to gradually increase the learning rate. After that, a cosine annealing learning rate schedule will be used to gradually decay the learning rate to the minimum value to help the model converge in the later stages.

[0011] Step 2.3: Calculate the loss between the model output and the true label at each iteration;

[0012] Step 2.4: Use the AdamW optimizer to minimize the loss function and save the model with the highest accuracy in the training set during training.

[0013] Step 3: Dataset preprocessing: Screen high-quality slit lamp images and classify them into ten categories based on existing labels. The process is as follows:

[0014] Step 3.1: Slit lamp images were quality screened. Core criteria included: the lens area must be clearly visible; samples with optical obscuration of key lens areas due to light source reflection were excluded to ensure the extractability of cataract features.

[0015] Step 3.2: uniformly crop the filtered slit lamp image so that the lens area occupies most of the slit lamp image area;

[0016] Step 3.3: Merge data with insufficient sample size and similar pathological characteristics;

[0017] Step 3.4: After quality control and category optimization, the dataset contains ten fine-grained classification labels: nuclear cataract grade 2, nuclear cataract grade 3, nuclear cataract grade 4, nuclear cataract grade 5, cortical cataract grade 1, cortical cataract grade 2, cortical cataract grade 3, cortical cataract grade 4, posterior subcapsular cataract, and healthy. Stratified random sampling is used to divide the dataset into training, validation, and test sets according to the set ratio.

[0018] Step 4: Cataract test set testing: Input the processed slit lamp images into the trained network architecture to evaluate the model performance.

[0019] Furthermore, the process of step one is as follows:

[0020] Step 1.1: The multi-granularity cataract classification network model based on lesion perception includes a feature extractor, a lesion-perceived feature extraction module, a fine-grained feature optimization module, and a classifier. The feature extractor includes four feature extraction modules for extracting pathological feature information from slit lamp cataract images layer by layer. The lesion-perceived feature extraction module is composed of a mutual attention weight calculation and a maximum connected region search algorithm layer to extract features around the lesion. The fine-grained feature optimization module is composed of upsampling, two-dimensional convolution, and dilated convolution to obtain more comprehensive lesion detail information. The classifier is composed of a fully connected layer, which concatenates the lesion features obtained by the fine-grained feature optimization module with the pathological features extracted by the feature extractor, and feeds them into the fully connected layer as input, ultimately mapping them to the corresponding multi-granularity cataract category.

[0021] Step 1.2: Before inputting into the feature extractor, the original slit lamp image is first normalized and uniformly resized to the specified pixel size; then, a convolutional layer with the same kernel size and stride is used to transform the image features. This operation not only achieves image patching (i.e., patching), evenly dividing the image into fixed-sized regions, providing a basis for subsequent window attention (WA) and shifted window attention (SWA) calculations; it also maps the original 3-channel image to a feature space with a dimension greater than or equal to 128, thereby enhancing the network's ability to represent fine-grained cataract features.

[0022] Step 1.3: Construct a feature extractor for the lesion-aware multi-granularity cataract classification network model. The feature extractor consists of four feature extraction modules, named the first, second, third, and fourth feature extraction modules in order of network depth. Each feature extraction module consists of multiple cascaded sub-encoders (Swin Transformer Blocks), where the first, second, and fourth modules each contain two sub-encoders, and the third module contains eighteen sub-encoders.

[0023] Although all sub-encoders maintain the same structure, the number of input channels of their internal fully connected layers is adjusted according to the feature dimension. Specifically, sub-encoders in the same feature extraction module have the same number of input channels; however, the number of input channels of sub-encoders in different modules gradually increases as the network deepens to enhance the model's ability to express more complex and fine-grained features.

[0024] Step 1.4: Feature extraction modules are connected via a patch merging module. Patch merging effectively extracts fine-grained features by reducing the spatial resolution of the image and increasing the feature dimension. Specifically, patch merging concatenates adjacent image patches along the channel dimension, reducing the image's length and width to half and quadrupling the number of channels. A fully connected layer is then used to compress the number of feature channels to twice the original number, achieving both reconstruction and compression of feature dimensions.

[0025] Step 1.5: Construct a lesion-aware feature extraction module for the lesion-aware multi-granularity cataract classification network model. The lesion-aware feature extraction module consists of a mutual attention weight calculation and a maximum connected region search algorithm layer. The pathological feature information extracted layer by layer by the feature extractor is input into the mutual attention weight. Through the calculation of the mutual attention weight, the mutual attention weight is screened to determine the mutual weight feature regions with high scores. The maximum connected region search algorithm is used to select the largest lesion area and calculate the lesion center point. Based on the lesion center position, the area surrounding the lesion is selected to filter out low-correlation or irrelevant areas, including corneal reflections and light interference.

[0026] Step 1.6: Construct a fine-grained feature optimization module for the multi-granularity cataract classification network model based on lesion perception. The fine-grained feature optimization module consists of upsampling, two-dimensional convolution, and dilated convolution. The features around the lesion extracted by the lesion perception feature extraction module are upsampled to enlarge the size of the features of the lesion area. Subsequently, a two-dimensional convolution operation is used to further extract the key features of the lesion area, thereby improving the fine-grained recognition ability of the classification model. The receptive field of the model is expanded by the dilated convolution module operation. After processing, the two-dimensional convolution module operation is used to extract the key features of the lesion area, which are then added to the upsampled features preserved by the jump connection mechanism. The key feature information of the lesion area is retained during the deep feature extraction process. The optimized lesion features are spliced ​​with the features of the slit lamp image extracted step by step by the feature extractor, and then input into the classifier for further classification prediction.

[0027] Step 1.7: The classifier of the multi-granularity cataract classification network model consists of a fully connected layer, which maps the spliced ​​lesion features and the features of the slit lamp image extracted step by step by the feature extractor to the number of fine-grained cataract categories.

[0028] Furthermore, in step 1.2, the mathematical relationship of the patching operation is expressed as follows:

[0029] Convolution kernel size (Kernel Size) = Patch size (Patch Size) × Patch size (Patch Size).

[0030] Furthermore, in step 1.3, the input-output mapping relationship of the four feature extraction modules is defined as follows: the width and height of the input slit lamp image are N×N, and the four feature extraction modules satisfy the following recursive relationship. The output feature of the feature extraction module 1 is The output features of feature extraction module 2 are The output features of the feature extraction module are The four output features of the feature extraction module are

[0031] Furthermore, in step 1.3, each feature extractor performs window self-attention and offset window self-attention mechanisms. The calculation formula for self-attention is as follows:

[0032]

[0033] Based on the above window self-attention architecture, a multi-head expansion mechanism is introduced to enhance the feature expression capability. The formula is as follows:

[0034] MSA(Q,K,V)=Concat(head1,…,head h )·W O

[0035] where head i =Attention(QWi i Q ,KW i K ,VW i V )

[0036] Among them, MSA represents the multi-head self-attention mechanism, W O 、W Q 、W K 、W V All are learnable parameters, defined as floating-point variables;

[0037] Furthermore, in step 1.5, the maximum connected region search algorithm performs the following operations: determine the mutual weight feature region with a high score in the previous step, obtain the input binary mask matrix, which is used to represent the distribution of the lesion area, where the pixel value of the lesion area is 1 and the pixel value of the non-lesion area is 0. Set an initial search point, select the first unmarked lesion area as the seed point for search, use the flood fill algorithm, starting from the initial seed point, recursively or iteratively mark all connected lesion areas to form a connected component, record the size of the connected component, search for the next unmarked lesion area in the entire mask matrix, use it as a new seed point, repeat the flood fill operation, mark the next connected component, and continue this process until all lesion areas are marked, ensuring that all lesion areas are identified and counted, count all marked connected areas, compare their sizes, and filter out the lesion area with the largest area.

[0038] Furthermore, in step 1.6, the upsampling performs the following operations: the spatial resolution of the feature map is increased by 2 times, local features are extracted from the upsampled feature map using standard convolution, the convolved features are normalized through batch normalization, and then the ReLU activation function is used to introduce nonlinear transformation to improve feature representation capabilities.

[0039] Furthermore, in step 1.6, the two-dimensional convolution module performs the following operations: first, the input features are processed by a standard convolution operation and batch normalization, and then the features are nonlinearly transformed using the ReLU activation function.

[0040] Furthermore, in step 1.6, the dilated convolution performs the following operations: first, the input features are extracted using a dilated convolution with a dilation rate of 2 to expand the receptive field, and then batch normalization and ReLU activation function are used for nonlinear changes.

[0041] Furthermore, the process of step 4 is as follows:

[0042] Step 4.1: Evaluate the performance of the optimized weighted model on the test set using quantitative metrics including precision, recall, F1 value, accuracy, Matthews correlation coefficient (MCC), and Cohen's Kappa coefficient.

[0043] Step 4.2: For the ten categories of multi-granularity cataracts, the calculated indicators are compared with mainstream algorithms to prove the effectiveness of this method in the multi-granularity cataract classification scenario.

[0044] A multi-granularity cataract classification system based on lesion perception, including:

[0045] Multi-granularity cataract classification network framework building module, used to build a multi-granularity cataract classification network framework;

[0046] Multi-granularity cataract classification network training module, used to train multi-granularity cataract classification network;

[0047] Dataset preprocessing module, used to preprocess the dataset;

[0048] The multi-granularity cataract classification test module is used to classify and test multi-granularity cataracts.

[0049] The advantages of the present invention are: solving the problems of low classification accuracy in the existing technology caused by light interference, difficulty in accurately locating the lesion area and the complexity of multi-granularity classification tasks; being able to accurately extract lesion features, remove irrelevant noise, and improve the robustness and accuracy of cataract classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a schematic diagram of the multi-granularity cataract classification network framework based on lesion perception in the present invention.

[0051] Figure 2 Schematic diagram of data preprocessing of the present invention.

[0052] Figure 3 This is the model reasoning flow chart of the present invention.

[0053] Figure 4 It is the overall flow chart of the model of the present invention.

[0054] Figure 5 Schematic diagram of the structure of the lesion perception feature extraction module of the present invention.

[0055] Figure 6 Schematic diagram of the structure of the fine-grained feature optimization module of the present invention. DETAILED DESCRIPTION

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0058] Reference Figures 1 to 6 A multi-granularity cataract classification method based on lesion perception is proposed, which includes four processes: construction of a multi-granularity cataract classification network framework, multi-granularity cataract classification network training, dataset preprocessing and cataract classification testing.

[0059] Slit lamps are widely used in the clinical diagnosis of cataracts and can clearly present details of the ocular structure, especially the opacity of the lens. Currently, there are many ways to obtain slit lamp images. One common method is to use a digital slit lamp microscope. By connecting the slit lamp to a digital camera, not only the contrast and resolution of the image are improved, but also the information expression of the lens area is enhanced, thereby helping ophthalmologists to more accurately assess the patient's condition. The present invention conducts multi-granularity classification research on cataracts based on slit lamp images.

[0060] The method comprises the following steps:

[0061] Step 1: Build a multi-granularity cataract classification network model based on lesion perception to enhance the network's ability to extract and discriminate multi-granularity features of cataracts in slit lamp images. The process is as follows:

[0062] Step 1.1: The multi-granularity cataract classification network model based on lesion perception includes a feature extractor, a lesion-perceived feature extraction module, a fine-grained feature optimization module, and a classifier. The feature extractor includes four feature extraction modules for extracting pathological feature information from slit lamp cataract images layer by layer. The lesion-perceived feature extraction module is composed of mutual attention weight calculation and maximum connected region search algorithm to extract features around the lesion. The fine-grained feature optimization module is composed of upsampling, two-dimensional convolution, and dilated convolution to obtain more comprehensive lesion detail information. The classifier is composed of a fully connected layer, which concatenates the lesion features obtained by the fine-grained feature optimization module with the pathological features extracted by the feature extractor, and feeds them into the fully connected layer as input, ultimately mapping them to ten fine-grained cataract categories.

[0063] Step 1.2: Before inputting into the feature extractor, the original slit lamp image is first normalized and uniformly adjusted to the specified pixel size; then, a convolutional layer with the same kernel size (Kernel Size) and stride (Stride) is used to perform feature conversion on the image; this operation not only realizes the image patching (i.e., patching), but also divides the image into fixed-size regions. The specific implementation method is to use the convolutional layer in the convolutional neural network for processing. The layer sets the number of input channels (In Channels) to 3, the number of output channels (Out Channels) to 128, and configures a 4×4 convolution kernel (Kernel Size) and a stride (Stride) of 4; in the preprocessing stage, after the original image resolution is adjusted to 224×224, the image is segmented into To adapt to the subsequent window-based self-attention mechanism operation, each window is defined to consist of a fixed number of adjacent image patches, thereby dividing the entire image into multiple local windows;

[0064] Step 1.3: The lesion-aware multi-granularity cataract classification system constructed in this example includes a feature extractor. The feature extractor contains four feature extraction modules, named the first, second, third, and fourth feature extraction modules in order of network depth. Each feature extraction module consists of multiple cascaded sub-encoders (Swin Transformer Blocks). The first, second, and fourth modules each contain two sub-encoders, and the third module contains eighteen sub-encoders.

[0065] Specifically, in each sub-encoder (Swin Transformer Block), the modular calculation process is strictly constructed in the following order: first, parameter-free layer normalization (LayerNorm_1) is performed, whose dimension is completely aligned with the input feature dimension, and only the features are standardized. Then, the window self-attention (WA) is calculated to capture the associated features of the patch in the local window, and Dropout_1 is used to randomly inactivate the attention weight to prevent overfitting; the second layer of parameter-free LayerNorm_2 is used to normalize the features and stabilize the gradient propagation. The multi-layer perceptron (MLP) is used for nonlinear transformation to expand the feature expression capability, and Dropout_2 is used to regularize the MLP output. The third layer of parameter-free LayerNorm_3 further optimizes the feature distribution, alleviates the instability of deep network training, and introduces shifted window attention (Shifted Window Attention). Attention (SWA) breaks through the fixed window limit and establishes long-range dependencies across windows. Dropout_3 is used to suppress noise interference and improve model generalization. LayerNorm_4 completes the final stage of feature normalization to ensure numerical stability. The second group of MLPs performs high-level semantic encoding on the features to enhance the ability to extract abstract features. Finally, Dropout_4 outputs the normalized feature vector.

[0066] Step 1.4: Patch merging is used to achieve hierarchical connectivity between feature extraction modules. After completing intra-window and inter-window self-attention calculations, local features are enhanced by aggregating adjacent 2×2 image patches. After splicing along the channel dimension, the spatial resolution of the feature map is reduced to 1 / 2 of its original size, and the number of channels is quadrupled. The number of channels is then compressed to 2C using an unbiased fully connected layer. This mechanism ensures that with each module, the spatial resolution is halved and the number of channels is doubled. After four modules (including three patch fusions), the feature dimension eventually evolves to 1024 channels.

[0067] Step 1.5: Input the extracted features into the lesion-aware feature extraction module to obtain the attention weight matrix from the multi-head self-attention. Multi-head self-attention is averaged, aggregating information from multiple channels to obtain a global weight. The product of the attention contribution and the weight is calculated to obtain the mutual attention weight. The previously high-scoring mutual weight regions are identified and the input binary mask matrix is ​​obtained. This matrix represents the distribution of lesion regions, where the pixel value of the lesion region is 1 and the pixel value of the non-lesion region is 0. An initial search point is set and the first unlabeled lesion region is selected as the seed point for the search. A flood-fill algorithm is used, starting from the initial seed point, to recursively or iteratively mark all connected lesion regions to form a connected component, and the size of the connected component is recorded. The next unlabeled lesion region is searched for in the entire mask matrix and used as a new seed point. The flood-fill operation is repeated, marking the next connected component. This process continues until all lesions are marked, ensuring that all lesions are identified and counted. All marked connected regions are counted and their sizes are compared to select the lesion region with the largest area. The centroid of this lesion region is calculated. Obtaining a position index of the lesion area, and extracting the lesion area through the position index;

[0068] Step 1.6: The extracted lesion features are input into the fine-grained feature optimization module. The input features are expanded in size using the bilinear interpolation method to increase the spatial resolution of the feature map by 2 times. The upsampled feature map is extracted using standard convolution. The convolved features are normalized using batch normalization. The ReLU activation function is then used to introduce nonlinear transformations to improve feature representation capabilities. The upsampled features are input and processed through a standard convolution operation and batch normalization. Then, the ReLU activation function is used to perform nonlinear transformations on the features. The features processed by two-dimensional convolution are input and feature extraction is performed using dilated convolution with a dilation rate of 2 to expand the receptive field. Then, batch normalization and ReLU activation functions are used for nonlinear changes. The features processed by dilated convolution again are input and processed through a standard convolution operation and batch normalization. Then, the ReLU activation function is used to perform nonlinear transformations on the features.

[0069] Step 1.7: The classifier consists of a single fully connected layer with an input feature dimension of 1024 and an output dimension that matches the number of target categories for the cataract fine-grained classification task (10 categories).

[0070] Step 2: The training process of the cataract fine-grained classification network is as follows:

[0071] Step 2.1: Prepare a slit lamp cataract image and resize it to N × N pixels. The image is then fed into the feature extractor. The image then passes through the lesion-aware feature extraction module, the fine-grained feature optimization module, and the classifier.

[0072] Step 2.2: Use the pre-trained model as the initial weights and perform multiple rounds of iterative training. During the training process, a linear warm-up strategy will be used for the first 20 rounds to gradually increase the learning rate. After that, a cosine annealing learning rate schedule will be used to gradually decay the learning rate to the minimum value to help the model converge in the later stages.

[0073] Step 2.3: Calculate the loss between the model output and the true label at each iteration;

[0074] Step 2.4: Use the AdamW optimizer to minimize the loss function and save the model with the highest accuracy in the training set during training.

[0075] Step 3: Dataset preprocessing: Screen high-quality slit lamp images and classify them into ten categories based on existing labels. The process is as follows:

[0076] Step 3.1: Slit lamp images were quality screened. Core criteria included: the lens area must be clearly visible; samples with optical obscuration of key lens areas due to light source reflection were excluded to ensure the extractability of cataract features.

[0077] Step 3.2: uniformly crop the filtered slit lamp image so that the lens area occupies most of the slit lamp image area;

[0078] Step 3.3: Merge data with insufficient sample size and similar pathological characteristics;

[0079] Step 3.4: After quality control and category optimization, the dataset contains ten fine-grained classification labels: nuclear cataract grade 2, nuclear cataract grade 3, nuclear cataract grade 4, nuclear cataract grade 5, cortical cataract grade 1, cortical cataract grade 2, cortical cataract grade 3, cortical cataract grade 4, posterior subcapsular cataract, and healthy. Stratified random sampling is used to divide the dataset into training, validation, and test sets in a ratio of 6:2:2.

[0080] Step 4: Input the processed slit lamp images into the trained network architecture and evaluate the model performance on the test set. The process is as follows:

[0081] Step 4.1: Evaluate the performance of the optimized weighted model on the test set using quantitative metrics including precision, recall, F1 value, accuracy, Matthews correlation coefficient (MCC), and Cohen's Kappa coefficient.

[0082] Step 4.2: For the ten categories of multi-granularity cataract, the calculated indicators are compared with the mainstream algorithms to verify the effectiveness of this method in the multi-granularity cataract classification scenario. A multi-granularity cataract classification system based on lesion perception includes:

[0083] Multi-granularity cataract classification network framework building module, used to build a multi-granularity cataract classification network framework;

[0084] Multi-granularity cataract classification network training module, used to train multi-granularity cataract classification network;

[0085] Dataset preprocessing module, used to preprocess the dataset;

[0086] Cataract classification test module, used for classification testing of multi-granularity cataracts.

[0087] Each of the above modules corresponds to steps 1 to 4 of Example 1. The present invention also provides a multi-granularity cataract classification device based on lesion perception, comprising a non-volatile memory, a processor component, and a graphics processing unit (GPU), wherein the memory stores executable code, and when the processor component executes the executable code, it is used to implement a multi-granularity cataract classification method based on lesion perception of Example 1.

[0088] The present invention also provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the multi-granularity cataract classification method based on lesion perception described in the first embodiment is implemented.

[0089] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A multi-granularity cataract classification method based on lesion perception, characterized in that: The method comprises the following steps: Step 1: Build a multi-granularity cataract classification network model based on lesion perception to enhance the network's ability to extract and discriminate multi-granularity features of cataracts in slit lamp images; Step 2: Train the multi-granularity cataract classification network model. The process is as follows: Step 2.1: Prepare a slit lamp cataract image and resize it to N × N pixels. The image is then fed into the feature extractor. The image then passes through the lesion-aware feature extraction module, the fine-grained feature optimization module, and the classifier. Step 2.2: Use the pre-trained model as the initial weights and perform multiple rounds of iterative training. During the training process, a linear warm-up strategy will be used for the first 20 rounds to gradually increase the learning rate. After that, a cosine annealing learning rate schedule will be used to gradually decay the learning rate to the minimum value to help the model converge in the later stages. Step 2.3: Calculate the loss between the model output and the true label at each iteration; Step 2.4: Use the AdamW optimizer to minimize the loss function and save the model with the highest accuracy in the training set during training. Step 3: Dataset preprocessing: Screen high-quality slit lamp images and classify them into ten categories based on existing labels. The process is as follows: Step 3.1: Slit lamp images were quality screened. Core criteria included: the lens area must be clearly visible; samples with optical obscuration of key lens areas due to light source reflection were excluded to ensure the extractability of cataract features. Step 3.2: uniformly crop the filtered slit lamp image so that the lens area occupies most of the slit lamp image area; Step 3.3: Merge data with insufficient sample size and similar pathological characteristics; Step 3.4: After quality control and category optimization, the dataset contains ten fine-grained classification labels: nuclear cataract grade 2, nuclear cataract grade 3, nuclear cataract grade 4, nuclear cataract grade 5, cortical cataract grade 1, cortical cataract grade 2, cortical cataract grade 3, cortical cataract grade 4, posterior subcapsular cataract, and healthy. Stratified random sampling is used to divide the dataset into training, validation, and test sets according to the set ratio. Step 4: Cataract test set testing: Input the processed slit lamp images into the trained network architecture to evaluate the model performance.

2. The multi-granularity cataract classification method based on lesion perception according to claim 1, characterized in that: The process of step one is as follows: Step 1.1: The multi-granularity cataract classification network model based on lesion perception includes a feature extractor, a lesion-perceived feature extraction module, a fine-grained feature optimization module, and a classifier. The feature extractor includes four feature extraction modules for extracting pathological feature information from slit lamp cataract images layer by layer. The lesion-perceived feature extraction module is composed of a mutual attention weight calculation and a maximum connected region search algorithm layer to extract features around the lesion. The fine-grained feature optimization module is composed of upsampling, two-dimensional convolution, and dilated convolution to obtain more comprehensive lesion detail information. The classifier is composed of a fully connected layer, which concatenates the lesion features obtained by the fine-grained feature optimization module with the pathological features extracted by the feature extractor, and feeds them into the fully connected layer as input, ultimately mapping them to the corresponding multi-granularity cataract category. Step 1.2: Before inputting into the feature extractor, the original slit lamp image is first normalized and uniformly adjusted to the specified pixel size; Subsequently, a convolutional layer with the same kernel size and stride was used to perform feature transformation on the image. This operation not only partitioned the image into fixed-sized regions, providing a basis for subsequent window attention and offset window attention calculations, but also mapped the original three-channel image into a feature space with a dimension greater than or equal to 128, thereby enhancing the network's ability to represent fine-grained cataract features. Step 1.3: Construct a feature extractor for the lesion-aware multi-granularity cataract classification network model. The feature extractor consists of four feature extraction modules, named the first, second, third, and fourth feature extraction modules in order of network depth. Each feature extraction module consists of multiple cascaded sub-encoders, where the first, second, and fourth modules each contain two sub-encoders, and the third module contains eighteen sub-encoders. Although all sub-encoders maintain the same structure, the number of input channels of their internal fully connected layers is adjusted according to the feature dimension. Sub-encoders in the same feature extraction module have the same number of input channels. However, the number of input channels of sub-encoders in different modules gradually increases as the network deepens to enhance the model's ability to express more complex and fine-grained features. Step 1.4: The feature extraction modules are connected through the patch fusion module, which effectively extracts fine-grained features by reducing the spatial resolution of the image and increasing the feature dimension; The patch fusion module concatenates adjacent image patches according to the channel dimension, reducing the image length and width to half of their original size while increasing the number of channels to four times the original size. Subsequently, a fully connected layer is used to compress the number of feature channels to twice the original size, thereby achieving reconstruction and compression of feature dimensions. Step 1.5: Construct a lesion-aware feature extraction module for the lesion-aware multi-granularity cataract classification network model. The lesion-aware feature extraction module consists of a mutual attention weight calculation and a maximum connected region search algorithm layer. The pathological feature information extracted layer by layer by the feature extractor is input into the mutual attention weight. Through the calculation of the mutual attention weight, the mutual attention weight is screened to determine the mutual weight feature regions with high scores. The maximum connected region search algorithm is used to select the largest lesion area and calculate the lesion center point. Based on the lesion center position, the area surrounding the lesion is selected to filter out low-correlation or irrelevant areas, including corneal reflections and light interference. Step 1.6: Construct a fine-grained feature optimization module for the lesion-aware multi-granularity cataract classification network model. The fine-grained feature optimization module consists of upsampling, two-dimensional convolution, and dilated convolution. The features around the lesion extracted by the lesion-aware feature extraction module are upsampled to enlarge the size of the features in the lesion area. Subsequently, a two-dimensional convolution operation is used to further extract the key features of the lesion area, improving the fine-grained recognition ability of the classification model. The dilated convolution module is used to expand the receptive field of the model. After processing, the two-dimensional convolution module is used to extract the key features of the lesion area. These features are then added to the upsampled features preserved by the skip connection mechanism. The key feature information of the lesion area is retained during the deep feature extraction process. The optimized lesion features are spliced ​​with the features of the slit lamp image extracted by the feature extractor step by step, and then input into the classifier for further classification prediction. Step 1.7: The classifier of the multi-granularity cataract classification network model consists of a fully connected layer, which maps the spliced ​​lesion features and the features of the slit lamp image extracted step by step by the feature extractor to the number of fine-grained cataract categories.

3. The multi-granularity cataract classification method based on lesion perception according to claim 2, characterized in that: In step 1.2, the mathematical relationship of the patching operation is expressed as follows: convolution kernel size = patch size × patch size.

4. The multi-granularity cataract classification method based on lesion perception according to claim 2, characterized in that: In step 1.3, the input-output mapping relationship of the four feature extraction modules is defined as follows: the width and height of the input slit lamp image are N×N, and the four feature extraction modules satisfy the following recursive relationship. The output feature of the feature extraction module is The output features of feature extraction module 2 are The output features of the feature extraction module are The four output features of the feature extraction module are 5. The multi-granularity cataract classification method based on lesion perception according to claim 2, characterized in that: In step 1.3, each feature extractor performs window self-attention and offset window self-attention mechanisms. The calculation formula for self-attention is as follows: Based on the above window self-attention architecture, a multi-head expansion mechanism is introduced to enhance the feature expression capability. The formula is as follows: MSA(Q,K,V)=Concat(head1,…,head h )·W O whereabouts i =Attention(QWi i Q ,KW i K ,VW i V ) Among them, MSA represents the multi-head self-attention mechanism, W O 、W Q 、W K 、W V These are all learnable parameters, defined as floating-point variables.

6. The multi-granularity cataract classification method based on lesion perception according to claim 2, characterized in that: In step 1.5, the maximum connected region search algorithm performs the following operations: determining the mutual weight feature area with a high score in the previous step, obtaining the input binary mask matrix, which is used to represent the distribution of the lesion area, where the pixel value of the lesion area is 1 and the pixel value of the non-lesion area is 0, setting an initial search point, selecting the first unmarked lesion area as the seed point for search, using a flood fill algorithm, starting from the initial seed point, recursively or iteratively marking all connected lesion areas to form a connected component, recording the size of the connected component, searching for the next unmarked lesion area in the entire mask matrix, using it as a new seed point, repeating the flood fill operation, marking the next connected component, and continuing the process until all lesion areas are marked, ensuring that all lesion areas are identified and counted, counting all marked connected areas, and comparing their sizes to filter out the lesion area with the largest area.

7. The multi-granularity cataract classification method based on lesion perception according to claim 2, characterized in that: In step 1.6, the upsampling performs the following operations: the spatial resolution of the feature map is increased by 2 times, the local features of the upsampled feature map are extracted using standard convolution, the convolved features are normalized through batch normalization, and then the ReLU activation function is used to introduce nonlinear transformation to improve the feature representation capability.

8. The multi-granularity cataract classification method based on lesion perception according to claim 2, characterized in that: In step 1.6, the two-dimensional convolution module performs the following operations: first, the input features are processed by a standard convolution operation and batch normalization, and then the features are nonlinearly transformed using the ReLU activation function; the dilated convolution performs the following operations: first, the input features are extracted using a dilated convolution with a dilation rate of 2 to expand the receptive field; then, batch normalization and the ReLU activation function are used for nonlinear changes.

9. The multi-granularity cataract classification method based on lesion perception according to any one of claims 1 to 8, characterized in that: The process of step 4 is as follows: Step 4.1: Evaluate the performance of the optimized optimal weight model on the test set. Quantitative indicators include precision, recall, F1 value, accuracy, Matthews correlation coefficient, and Cohen's Kappa coefficient. Step 4.2: For the ten categories of multi-granularity cataracts, the calculated indicators are compared with mainstream algorithms to prove the effectiveness of this method in the multi-granularity cataract classification scenario.

10. A system for implementing the multi-granularity cataract classification method based on lesion perception as claimed in claim 1, characterized in that: The system comprises: Multi-granularity cataract classification network framework building module, used to build a multi-granularity cataract classification network framework; Multi-granularity cataract classification network training module, used to train multi-granularity cataract classification network; Dataset preprocessing module, used to preprocess the dataset; The multi-granularity cataract classification test module is used to classify and test multi-granularity cataracts.

Images