Lesion-aware multi-granularity cataract classification method and system

Abstract

The application discloses a kind of based on lesion perception multi-granularity cataract classification method and system, belong to computer vision and medical image analysis field, take slit lamp image as input data, adopt lesion perception feature extraction method, make model can accurately focus on lesion area, improve the accuracy of cataract classification;Adopt mutual attention calculation and maximum connected region search algorithm, filter out the lesion area of high weight feature, ensure that classification model can focus on the core features of lesion area;Through fine-grained feature optimization module, combined with up-sampling convolution, dilated convolution and jump connection strategy, enhance the fine-grained information of lesion area, improve the distinguishing ability of different types of cataract.The application can extract high-value lesion information from complex slit lamp image data, realize real-time, high-precision multi-granularity cataract classification, assist clinical diagnosis, improve the screening and treatment efficiency of early cataract.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and medical image analysis, specifically to a multi-granularity cataract classification method and system based on lesion perception. This method utilizes deep learning technology to extract features, denoise, and classify cataract lesions in slit-lamp images, thereby improving the accuracy of automated cataract diagnosis and achieving cataract classification at different granularities. Background Technology

[0002] Cataracts are a leading cause of vision impairment and blindness worldwide, and early screening and accurate classification are crucial for timely intervention and treatment. Slit-lamp microscopy is an important tool in ophthalmological clinical diagnosis of cataracts, allowing doctors to assess and classify the degree of lens opacity by observing slit-lamp images. However, manual diagnosis not only relies on the doctor's experience but is also significantly influenced by subjective factors, easily leading to inconsistencies in diagnosis.

[0003] Currently, deep learning technology has been widely applied in the field of medical image analysis, especially methods based on convolutional neural networks and Transformers, which have been used for ophthalmic disease detection. However, multi-granularity cataract classification tasks still face challenges due to light interference and noise, making the task complex. Summary of the Invention

[0004] To address the low classification accuracy issues in existing technologies caused by light interference, difficulty in accurately locating lesion areas, and the complexity of multi-granularity classification tasks, this invention provides a multi-granularity cataract classification method and system based on lesion perception. By combining a feature extractor, a lesion perception feature extraction module, a fine-grained feature optimization module, and a classifier, it can accurately extract lesion features, remove irrelevant noise, and improve the robustness and accuracy of cataract classification.

[0005] The technical solution provided by this invention is as follows:

[0006] A multi-granularity cataract classification method based on lesion perception includes the following steps:

[0007] Step 1: Build a multi-granularity cataract classification network model based on lesion perception to enhance the network's ability to extract and distinguish multi-granularity features of cataracts in slit-lamp images;

[0008] Step 2: Train the multi-granularity cataract classification network model, as follows:

[0009] Step 2.1: Prepare slit-lamp cataract images and adjust the image size to N×N pixels, then input them into the feature extractor; they pass through the lesion perception feature extraction module, the fine-grained feature optimization module, and the classifier in sequence;

[0010] Step 2.2: Using the pre-trained model as the initial weights, perform multiple rounds of iterative training. During the training process, a linear warm-up strategy will be adopted for the first 20 rounds to gradually increase the learning rate; thereafter, cosine annealing learning rate scheduling will be adopted to gradually decay the learning rate to the minimum value to help the model converge in the later stages.

[0011] Step 2.3: Calculate the loss between the model output and the true label in each iteration;

[0012] Step 2.4: Use the AdamW optimizer to minimize the loss function and save the model with the highest accuracy in the training set during training;

[0013] Step 3: Dataset preprocessing; High-quality slit lamp images are selected, and the selected slit lamp images are divided into ten categories based on existing labels; the process is as follows:

[0014] Step 3.1: Perform quality screening on slit-lamp images. The core criteria include: the lens region must be clearly visible; exclude samples where key areas of the lens are optically obscured due to light source reflection, and ensure the extractability of cataract features.

[0015] Step 3.2: Perform uniform cropping on the screened slit lamp images so that the lens region occupies most of the slit lamp image area;

[0016] Step 3.3: Merge data with insufficient sample size and similar pathological features;

[0017] Step 3.4: After quality control and category optimization, the dataset contains ten fine-grained classification labels: nuclear cataract grade II, nuclear cataract grade III, nuclear cataract grade IV, nuclear cataract grade V, cortical cataract grade I, cortical cataract grade II, cortical cataract grade III, cortical cataract grade IV, posterior subcapsular cataract, and healthy. Stratified random sampling is used to divide the dataset into training, validation, and test sets according to a set ratio.

[0018] Step 4: Cataract test set testing; Input the processed slit-lamp images into the trained network architecture to evaluate the model performance.

[0019] Furthermore, the process of step one is as follows:

[0020] Step 1.1: The multi-granularity cataract classification network model based on lesion perception includes a feature extractor, a lesion perception feature extraction module, a fine-grained feature optimization module, and a classifier. The feature extractor contains four feature extraction modules, used to extract pathological feature information from slit-lamp cataract images layer by layer. The lesion perception feature extraction module consists of a mutual attention weight calculation and a maximum connected region search algorithm layer to extract features around the lesion. The fine-grained feature optimization module consists of upsampling, two-dimensional convolution, and dilated convolution to obtain more comprehensive lesion detail information. The classifier consists of a fully connected layer, which concatenates the lesion features obtained by the fine-grained feature optimization module with the pathological features extracted by the feature extractor, and feeds them as input into the fully connected layer, finally mapping them to the corresponding multi-granularity cataract category.

[0021] Step 1.2: Before inputting the feature extractor, the original slit lamp image is first normalized and uniformly adjusted to a specified pixel size. Then, a convolutional layer with the same kernel size and stride is used to perform feature transformation on the image. This operation not only achieves image patching (i.e., uniformly dividing the image into fixed-size regions, providing a foundation for subsequent window attention (WA) and shifted window attention (SWA) calculations, but also maps the original 3-channel image to a feature space with a dimension greater than or equal to 128, thereby enhancing the network's ability to represent fine-grained features of cataracts.

[0022] Step 1.3: Construct a feature extractor for a multi-granularity cataract classification network model based on lesion perception. The feature extractor contains four feature extraction modules, named as the first, second, third and fourth feature extraction modules in order of network depth. Each feature extraction module consists of multiple cascaded sub-encoders (Swin Transformer Blocks), where the first, second and fourth modules each contain two sub-encoders, and the third module contains eighteen sub-encoders.

[0023] Although all sub-encoders maintain a consistent structure, the number of input channels in their internal fully connected layers is adjusted according to the feature dimension. Specifically, sub-encoders within the same feature extraction module have the same number of input channels; while sub-encoders in different modules gradually increase the number of input channels as the network deepens, in order to enhance the model's ability to express more complex and fine-grained features.

[0024] Step 1.4: The feature extraction modules are connected through the Patch Merging module. Patch Merging effectively extracts fine-grained features by reducing the spatial resolution of the image and increasing the feature dimension. Specifically, Patch Merging stitches adjacent image patches along the channel dimension, reducing the image's length and width to half of their original size while expanding the number of channels to four times. Subsequently, a fully connected layer is used to compress the number of feature channels to twice their original size, thereby achieving feature dimension reconstruction and compression.

[0025] Step 1.5: Construct a lesion perception feature extraction module based on a multi-granularity cataract classification network model for lesion perception. The lesion perception feature extraction module consists of a mutual attention weight calculation layer and a maximum connected region search algorithm layer. The pathological feature information extracted layer by layer by the feature extractor is input into the mutual attention weight. Through the calculation of the mutual attention weight, the mutual attention weight is filtered to determine the mutual weight feature regions with high scores. Through the maximum connected region search algorithm, the lesion region with the largest area is selected, and the lesion center point is calculated. Based on the lesion center position, the area around the lesion is selected, and low-relevance or irrelevant areas, including corneal reflection and light interference, are filtered out.

[0026] Step 1.6: Construct a fine-grained feature optimization module for a multi-granularity cataract classification network model based on lesion perception. This module consists of upsampling, 2D convolution, and dilated convolution. Features around the lesion extracted by the lesion perception feature extraction module are upsampled to enlarge the size of the lesion region. Then, 2D convolution is used to further extract key features of the lesion region, improving the fine-grained recognition capability of the classification model. Dilated convolution expands the model's receptive field. After processing, 2D convolution is used to extract key features of the lesion region, which are then added to the upsampled features saved through a skip connection mechanism. Key feature information of the lesion region is preserved during deep feature extraction. The optimized lesion features are then concatenated with features extracted step-by-step from the slit-lamp image by the feature extractor and input into the classifier for further classification and prediction.

[0027] Step 1.7: The classifier of the multi-granularity cataract classification network model consists of a fully connected layer, which maps the spliced ​​lesion features and the features extracted by the feature extractor from the slit-lamp image to the number of fine-grained cataract categories.

[0028] Furthermore, in step 1.2, the mathematical relationship of the patching operation is expressed as follows:

[0029] Kernel Size = Patch Size × Patch Size

[0030] Furthermore, in step 1.3, the input-output mapping relationship of the four feature extraction modules is defined as follows: the width and height of the input slit lamp image are N×N, and the four feature extraction modules satisfy the following recursive relationship, with feature extraction module one outputting features as follows: The second feature extraction module outputs the following features: The three output features of the feature extraction module are: The four output features of the feature extraction module are:

[0031] Furthermore, in step 1.3, each feature extractor internally implements window self-attention and offset window self-attention mechanisms. The calculation formula for self-attention is as follows:

[0032]

[0033] Based on the above window self-attention architecture, a multi-head expansion mechanism is introduced to enhance feature representation capabilities, as shown in the following formula:

[0034] MSA(Q,K,V)=Concat(head1,…,head h )·W O

[0035] where head i =Attention(QWi i Q ,KW i K VW i V )

[0036] Where MSA represents multi-head self-attention mechanism, W O W Q W K W V All of these are learnable parameters, defined as floating-point variables;

[0037] Furthermore, in step 1.5, the maximum connected component search algorithm performs the following operations: First, it identifies the high-scoring cross-weighted feature regions and obtains an input binary mask matrix. This matrix represents the distribution of lesion regions, where pixel values ​​in lesion regions are 1 and pixel values ​​in non-lesion regions are 0. An initial search point is set, and the first unlabeled lesion region is selected as the seed point for the search. Using a flooding fill algorithm, starting from the initial seed point, all connected lesion regions are recursively or iteratively labeled to form a connected component. The size of this connected component is recorded. The next unlabeled lesion region is searched in the entire mask matrix and used as the new seed point. The flooding fill operation is repeated, and the next connected component is labeled, continuing this process until all lesion regions are labeled, ensuring that all lesion regions are identified and counted. All labeled connected components are counted, their sizes are compared, and the lesion region with the largest area is selected.

[0038] Furthermore, in step 1.6, the upsampling is performed as follows: the spatial resolution of the feature map is increased by 2 times, local features are extracted from the upsampled feature map using standard convolution, the convolutional features are normalized by batch normalization, and then the ReLU activation function is used to introduce nonlinear transformation to improve the feature representation capability.

[0039] Furthermore, in step 1.6, the two-dimensional convolution module performs the following operations: First, the input features are processed by a standard convolution operation and batch normalization. Then, the features are nonlinearly transformed using the ReLU activation function.

[0040] Furthermore, in step 1.6, the dilated convolution performs the following operations: First, the input features are extracted using a dilated convolution with a dilation rate of 2 to expand the receptive field; then, batch normalization and the ReLU activation function are applied for nonlinear transformation.

[0041] Furthermore, the process of step four is as follows:

[0042] Step 4.1: Evaluate the performance of the optimized optimal weight model on the test set. The quantitative metrics include precision, recall, F1 score, accuracy, Matthews correlation coefficient (MCC), and Kappa coefficient.

[0043] Step 4.2: For the ten categories of multi-granularity cataracts, the calculated indicators are compared with mainstream algorithms to demonstrate the effectiveness of this method in the multi-granularity cataract classification scenario.

[0044] A multi-granularity cataract classification system based on lesion perception includes:

[0045] A module for building a multi-granularity cataract classification network framework is used to build such a framework.

[0046] A multi-granularity cataract classification network training module is used to train a multi-granularity cataract classification network.

[0047] The dataset preprocessing module is used to preprocess datasets;

[0048] The multi-granularity cataract classification test module is used for classifying and testing multi-granularity cataracts.

[0049] The advantages of this invention are: it solves the problems of low classification accuracy caused by light interference, difficulty in accurately locating lesion areas, and the complexity of multi-granularity classification tasks in the prior art; it can accurately extract lesion features, remove irrelevant noise, and improve the robustness and accuracy of cataract classification. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the multi-granularity cataract classification network framework based on lesion perception of the present invention.

[0051] Figure 2 This is a schematic diagram of data preprocessing according to the present invention.

[0052] Figure 3 This is a flowchart of the model reasoning process of the present invention.

[0053] Figure 4 This is the overall flowchart of the model of the present invention.

[0054] Figure 5 This is a schematic diagram of the lesion perception feature extraction module of the present invention.

[0055] Figure 6 This is a schematic diagram of the fine-grained feature optimization module of the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0058] Reference Figures 1-6 A multi-granularity cataract classification method based on lesion perception is proposed. The method includes four processes: construction of a multi-granularity cataract classification network framework, training of the multi-granularity cataract classification network, dataset preprocessing, and cataract classification testing.

[0059] Slit lamps are widely used in the clinical diagnosis of cataracts, as they can clearly reveal details of the eye's structure, especially the opacity of the lens. Currently, there are various ways to acquire slit lamp images. One common method is to use a digital slit lamp microscope. By connecting the slit lamp to a digital camera, not only is the image contrast and resolution improved, but the information representation of the lens area is also enhanced, thereby helping ophthalmologists to more accurately assess the patient's condition. This invention is based on slit lamp images to conduct multi-granularity classification research on cataracts.

[0060] The method includes the following steps:

[0061] Step 1: Construct a multi-granularity cataract classification network model based on lesion perception to enhance the network's ability to extract and discriminate multi-granularity features of cataracts in slit-lamp images. The process is as follows:

[0062] Step 1.1: The multi-granularity cataract classification network model based on lesion perception includes a feature extractor, a lesion perception feature extraction module, a fine-grained feature optimization module, and a classifier. The feature extractor contains four feature extraction modules, used to extract pathological feature information from slit-lamp cataract images layer by layer. The lesion perception feature extraction module consists of mutual attention weight calculation and maximum connected region search algorithm to extract features around the lesion. The fine-grained feature optimization module consists of upsampling, two-dimensional convolution, and dilated convolution to obtain more comprehensive lesion detail information. The classifier consists of fully connected layers, which concatenate the lesion features obtained by the fine-grained feature optimization module with the pathological features extracted by the feature extractor, and feed them as input to the fully connected layers, finally mapping them to ten fine-grained cataract categories.

[0063] Step 1.2: Before inputting the feature extractor, the original slit lamp image is first normalized and uniformly adjusted to a specified pixel size. Then, a convolutional layer with the same kernel size and stride is used to perform feature transformation on the image. This operation not only achieves image patching (i.e., dividing the image into regions of fixed size), but also uses a convolutional layer in a convolutional neural network. This layer has 3 input channels, 128 output channels, and a 4×4 kernel size with a stride of 4. In the preprocessing stage, after adjusting the resolution of the original image to 224×224, this convolutional operation segments the image into... To accommodate subsequent window-based self-attention mechanism operations, each window is defined as consisting of a fixed number of adjacent image patches, thereby dividing the entire image into multiple local windows.

[0064] Step 1.3: The multi-granularity cataract classification based on lesion perception constructed in this example includes a feature extractor. The feature extractor contains four feature extraction modules, which are named the first, second, third and fourth feature extraction modules in order of network depth. Each feature extraction module consists of multiple cascaded sub-encoders (Swin Transformer Blocks). The first, second and fourth modules each contain two sub-encoders, and the third module contains eighteen sub-encoders.

[0065] Specifically, within each sub-encoder (Swin Transformer Block), the modular computation process is rigorously constructed in the following order: First, parameterless layer normalization (LayerNorm_1) is performed, with its dimensions perfectly aligned to the input feature dimensions, standardizing only the features. Then, window self-attention (WA) is computed to capture the associated features of patches within a local window. Dropout_1 is used to randomly deactivate the attention weights to prevent overfitting. Next, the second parameterless layer (LayerNorm_2) performs secondary feature normalization to stabilize gradient propagation. A nonlinear transformation is then performed via a multilayer perceptron (MLP) to expand the feature representation capability. Dropout_2 is applied to regularize the MLP output. Finally, the third parameterless layer (LayerNorm_3) further optimizes the feature distribution, alleviating the instability of deep network training, and introduces shifted window attention. Attention (SWA) breaks through the fixed window limitation, establishing long-range dependencies across windows. Dropout_3 is used to suppress noise interference and improve model generalization. LayerNorm_4 completes the final stage of feature normalization to ensure numerical stability. The second MLP performs high-order semantic encoding on the features, enhancing the ability to extract abstract features. Finally, Dropout_4 outputs the normalized feature vector.

[0066] Step 1.4: Hierarchical connections are achieved between feature extraction modules through patch merging. After completing the self-attention calculation within and between windows, local feature enhancement is achieved by aggregating adjacent 2×2 image patches: after stitching along the channel dimension, the spatial resolution of the feature map is reduced to half of its original size, and the number of channels is expanded to 4 times. Subsequently, the number of channels is compressed to 2C through a biasless fully connected layer. This mechanism ensures that with each module, the spatial resolution is halved and the number of channels doubles. After 4 modules (including 3 patch merging), the feature dimension finally evolves to 1024 channels.

[0067] Step 1.5: Input the extracted features into the lesion perception feature extraction module to obtain the attention weight matrix in multi-head self-attention. Perform mean processing on the multi-head self-attention to aggregate information from multiple channels to obtain global weights. Calculate the product of attention contribution and weight to obtain mutual attention weights. Identify the mutual weight regions with high scores and obtain the input binary mask matrix. This matrix represents the distribution of lesion regions, where lesion region pixels have a value of 1 and non-lesion region pixels have a value of 0. Set an initial search point and select the first unlabeled lesion region as the seed point for searching. Use a flooding filling algorithm to recursively or iteratively label all connected lesion regions starting from the initial seed point, forming a connected component, and record the size of this connected component. Search for the next unlabeled lesion region in the entire mask matrix and use it as the new seed point. Repeat the flooding filling operation to label the next connected component, continuing this process until all lesion regions are labeled, ensuring that all lesion regions are identified and counted. Count all labeled connected regions and compare their sizes, selecting the lesion region with the largest area and calculating its centroid. Obtain the location index of the lesion area, and extract the lesion area using this location index;

[0068] Step 1.6: Input the extracted lesion features into the fine-grained feature optimization module. The input features are expanded using bilinear interpolation to increase the spatial resolution of the feature map by 2 times. Standard convolution is used to extract local features from the upsampled feature map. Batch normalization is used to normalize the convolutional features. Then, the ReLU activation function is used to introduce nonlinear transformation to improve feature representation. The input features after upsampling are processed by a standard convolution operation and batch normalization. Then, the ReLU activation function is used to perform nonlinear transformation on the features. The input features after two-dimensional convolution are extracted using dilated convolution with a dilation rate of 2 to expand the receptive field. Then, batch normalization and the ReLU activation function are used for nonlinear transformation. The input features after dilated convolution are processed by a standard convolution operation and batch normalization. Then, the ReLU activation function is used to perform nonlinear transformation on the features.

[0069] Step 1.7: The classifier consists of a single fully connected layer with an input feature dimension of 1024 and an output dimension consistent with the number of target categories (10 categories) in the cataract fine-grained classification task.

[0070] Step 2: The training process for the cataract fine-grained classification network is as follows:

[0071] Step 2.1: Prepare slit-lamp cataract images and adjust the image size to N×N pixels, then input them into the feature extractor; they pass through the lesion perception feature extraction module, the fine-grained feature optimization module, and the classifier in sequence;

[0072] Step 2.2: Using the pre-trained model as the initial weights, perform multiple rounds of iterative training. During the training process, a linear warm-up strategy will be adopted for the first 20 rounds to gradually increase the learning rate; thereafter, cosine annealing learning rate scheduling will be adopted to gradually decay the learning rate to the minimum value to help the model converge in the later stages.

[0073] Step 2.3: Calculate the loss between the model output and the true label in each iteration;

[0074] Step 2.4: Use the AdamW optimizer to minimize the loss function and save the model with the highest accuracy in the training set during training;

[0075] Step 3: Dataset preprocessing; Filter high-quality slit lamp images and divide the filtered slit lamp images into ten categories based on existing labels; the process is as follows;

[0076] Step 3.1: Perform quality screening on slit-lamp images. The core criteria include: the lens region must be clearly visible; exclude samples where key areas of the lens are optically obscured due to light source reflection, and ensure the extractability of cataract features.

[0077] Step 3.2: Perform uniform cropping on the screened slit lamp images so that the lens region occupies most of the slit lamp image area;

[0078] Step 3.3: Merge data with insufficient sample size and similar pathological features;

[0079] Step 3.4: After quality control and category optimization, the dataset contains ten fine-grained classification labels: nuclear cataract grade II, nuclear cataract grade III, nuclear cataract grade IV, nuclear cataract grade V, cortical cataract grade I, cortical cataract grade II, cortical cataract grade III, cortical cataract grade IV, posterior subcapsular cataract, and healthy. Stratified random sampling is used to divide the dataset into training, validation, and test sets in a 6:2:2 ratio.

[0080] Step 4: Input the processed slit lamp image into the trained network architecture and evaluate the model performance on the test set. The process is as follows:

[0081] Step 4.1: Evaluate the performance of the optimized optimal weight model on the test set. The quantitative metrics include precision, recall, F1 score, accuracy, Matthews correlation coefficient (MCC), and Kappa coefficient.

[0082] Step 4.2: For the ten categories of multi-granularity cataracts, the calculated indicators are compared with mainstream algorithms to verify the effectiveness of this method in multi-granularity cataract classification scenarios. A multi-granularity cataract classification system based on lesion perception includes:

[0083] A module for building a multi-granularity cataract classification network framework is used to build such a framework.

[0084] A multi-granularity cataract classification network training module is used to train a multi-granularity cataract classification network.

[0085] The dataset preprocessing module is used to preprocess datasets;

[0086] The cataract classification test module is used to classify and test cataracts at multiple granularities.

[0087] The above modules correspond to steps one through four of Embodiment 1. This invention also provides a multi-granularity cataract classification device based on lesion perception, including a non-volatile memory, a processor component, and a graphics processing unit (GPU). The memory stores executable code, and when the processor component executes the executable code, it implements a multi-granularity cataract classification method based on lesion perception according to Embodiment 1.

[0088] The present invention also provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the multi-granularity cataract classification method based on lesion perception as described in Embodiment 1.

[0089] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A multi-granularity cataract classification method based on lesion perception, characterized in that, The method includes the following steps: Step 1: Build a multi-granularity cataract classification network model based on lesion perception to enhance the network's ability to extract and distinguish multi-granularity features of cataracts in slit-lamp images; Step 2: Train the multi-granularity cataract classification network model, as follows: Step 2.1: Prepare slit-lamp cataract images and adjust the image size. The pixels are then input into the feature extractor; they pass sequentially through the lesion perception feature extraction module, the fine-grained feature optimization module, and the classifier; Step 2.2: Using the pre-trained model as the initial weights, perform multiple rounds of iterative training. During the training process, a linear warm-up strategy will be adopted for the first 20 rounds to gradually increase the learning rate; thereafter, cosine annealing learning rate scheduling will be adopted to gradually decay the learning rate to a minimum value to help the model converge in the later stages. Step 2.3: Calculate the loss between the model output and the true label in each iteration; Step 2.4: Use the AdamW optimizer to minimize the loss function and save the model with the highest accuracy in the training set during training; Step 3: Dataset preprocessing; High-quality slit lamp images are selected, and the selected slit lamp images are divided into ten categories based on existing labels; the process is as follows: Step 3.1: Perform quality screening on slit-lamp images. The core criteria include: the lens region must be clearly visible; exclude samples where key areas of the lens are optically obscured due to light source reflection, and ensure the extractability of cataract features. Step 3.2: Perform uniform cropping on the screened slit lamp images so that the lens region occupies most of the slit lamp image area; Step 3.3: Merge data with insufficient sample size and similar pathological features; Step 3.4: After quality control and category optimization, the dataset contains ten fine-grained classification labels: nuclear cataract grade II, nuclear cataract grade III, nuclear cataract grade IV, nuclear cataract grade V, cortical cataract grade I, cortical cataract grade II, cortical cataract grade III, cortical cataract grade IV, posterior subcapsular cataract, and healthy. Stratified random sampling is used to divide the dataset into training, validation, and test sets according to a set ratio. Step 4: Cataract test set testing; Input the processed slit-lamp images into the trained network architecture to evaluate the model performance.

2. The multi-granularity cataract classification method based on lesion perception as described in claim 1, characterized in that, The process of step one is as follows: Step 1.1: The multi-granularity cataract classification network model based on lesion perception includes a feature extractor, a lesion perception feature extraction module, a fine-grained feature optimization module, and a classifier. The feature extractor contains four feature extraction modules, used to extract pathological feature information from slit-lamp cataract images layer by layer. The lesion perception feature extraction module consists of a mutual attention weight calculation and a maximum connected region search algorithm layer to extract features around the lesion. The fine-grained feature optimization module consists of upsampling, two-dimensional convolution, and dilated convolution to obtain more comprehensive lesion detail information. The classifier consists of a fully connected layer, which concatenates the lesion features obtained by the fine-grained feature optimization module with the pathological features extracted by the feature extractor, and feeds them as input into the fully connected layer, finally mapping them to the corresponding multi-granularity cataract category. Step 1.2: Before inputting the feature extractor, the original slit lamp image is first normalized and uniformly adjusted to the specified pixel size; Subsequently, convolutional layers with the same kernel size and stride are used to perform feature transformation on the image. This operation not only realizes image segmentation, dividing the image into regions of fixed size evenly, providing a basis for subsequent window attention and offset window attention calculations; at the same time, it also maps the original 3-channel image to a feature space with a dimension greater than or equal to 128, thereby enhancing the network's ability to represent fine-grained features of cataracts. Step 1.3: Construct a feature extractor for a multi-granularity cataract classification network model based on lesion perception. The feature extractor contains four feature extraction modules, named as the first, second, third and fourth feature extraction modules in order of network depth. Each feature extraction module consists of multiple cascaded sub-encoders, where the first, second and fourth modules each contain two sub-encoders, and the third module contains eighteen sub-encoders. Although all sub-encoders maintain a consistent structure, the number of input channels in their internal fully connected layers is adjusted according to the feature dimension. Sub-encoders within the same feature extraction module have the same number of input channels; while the number of input channels in sub-encoders across different modules gradually increases as the network deepens, in order to enhance the model's ability to express more complex and fine-grained features. Step 1.4: The feature extraction modules are connected through the patch fusion module. The patch fusion module effectively extracts fine-grained features by reducing the spatial resolution of the image and increasing the feature dimension. The patch fusion module stitches adjacent image patches along the channel dimension, reducing the image's length and width to half of their original size while expanding the number of channels to four times the original size. Subsequently, a fully connected layer is used to compress the number of feature channels to twice the original size, thereby achieving feature dimension reconstruction and compression. Step 1.5: Construct a lesion perception feature extraction module based on a multi-granularity cataract classification network model for lesion perception. The lesion perception feature extraction module consists of a mutual attention weight calculation layer and a maximum connected region search algorithm layer. The pathological feature information extracted layer by layer by the feature extractor is input into the mutual attention weight. Through the calculation of the mutual attention weight, the mutual attention weight is filtered to determine the mutual weight feature regions with high scores. Through the maximum connected region search algorithm, the lesion region with the largest area is selected, and the lesion center point is calculated. Based on the lesion center position, the area around the lesion is selected, and low-relevance or irrelevant areas, including corneal reflection and light interference, are filtered out. Step 1.6: Construct a fine-grained feature optimization module for a multi-granularity cataract classification network model based on lesion perception. The fine-grained feature optimization module consists of upsampling, two-dimensional convolution, and dilated convolution. The features around the lesion extracted by the lesion perception feature extraction module are upsampled to enlarge the size of the features in the lesion area. Then, two-dimensional convolution is used to further extract the key features of the lesion area, improving the fine-grained recognition capability of the classification model. The dilated convolution module expands the receptive field of the model. After processing, the two-dimensional convolution module is used to extract the key features of the lesion area. Then, it is added to the upsampled features saved through the skip connection mechanism. The key feature information of the lesion area is preserved in the deep feature extraction process. The optimized lesion features are stitched together with the features extracted step by step from the slit lamp image by the feature extractor and then input into the classifier for further classification and prediction. Step 1.7: The classifier of the multi-granularity cataract classification network model consists of a fully connected layer, which maps the spliced ​​lesion features and the features extracted by the feature extractor from the slit-lamp image to the number of fine-grained cataract categories.

3. The multi-granularity cataract classification method based on lesion perception as described in claim 2, characterized in that, In step 1.4, the mathematical relationship of the patching operation is expressed as follows: .

4. The multi-granularity cataract classification method based on lesion perception as described in claim 2, characterized in that, In step 1.3, the input-output mapping relationship of the four feature extraction modules is defined as follows: the width and height of the input slit lamp image are... The four feature extraction modules satisfy the following recursive relationship, and the output feature of feature extraction module one is... The second feature extraction module outputs the following features: The feature extraction module outputs three features as follows: The feature extraction module outputs the following features: .

5. The multi-granularity cataract classification method based on lesion perception as described in claim 2, characterized in that, In step 1.3, each feature extractor internally performs window self-attention and offset window self-attention mechanisms. The calculation formula for self-attention is as follows: ； Based on the above window self-attention architecture, a multi-head expansion mechanism is introduced to enhance feature representation capabilities, as shown in the following formula: ； ； in This indicates a multi-head self-attention mechanism. , All of these are learnable parameters, defined as floating-point variables.

6. The multi-granularity cataract classification method based on lesion perception as described in claim 2, characterized in that, In step 1.5, the maximum connected component search algorithm performs the following operations: First, it determines the cross-weighted feature regions with high scores. Then, it obtains an input binary mask matrix, which represents the distribution of lesion regions, where lesion region pixels have a value of 1 and non-lesion region pixels have a value of 0. An initial search point is set, and the first unmarked lesion region is selected as the seed point for searching. Using a flooding fill algorithm, starting from the initial seed point, all connected lesion regions are recursively or iteratively marked to form a connected component. The size of this connected component is recorded. The next unmarked lesion region is searched in the entire mask matrix and used as a new seed point. The flooding fill operation is repeated, marking the next connected component, and this process continues until all lesion regions are marked, ensuring that all lesion regions are identified and counted. All marked connected components are counted, their sizes are compared, and the lesion region with the largest area is selected.

7. The multi-granularity cataract classification method based on lesion perception as described in claim 2, characterized in that, In step 1.6, the upsampling is performed as follows: the spatial resolution of the feature map is increased by 2 times, local features are extracted from the upsampled feature map using standard convolution, the convolutional features are normalized by batch normalization, and then the ReLU activation function is used to introduce nonlinear transformation to improve the feature representation capability.

8. The multi-granularity cataract classification method based on lesion perception as described in claim 2, characterized in that, In step 1.6, the two-dimensional convolution module performs the following operations: First, the input features are processed by a standard convolution operation and batch normalization. Then, the features are nonlinearly transformed using the ReLU activation function. The dilated convolution performs the following operations: First, the input features are extracted using a dilated convolution with a dilation rate of 2 to expand the receptive field. Then, batch normalization and the ReLU activation function are used for nonlinear transformation.

9. The multi-granularity cataract classification method based on lesion perception as described in any one of claims 1 to 8, characterized in that, The process of step four is as follows: Step 4.1: Evaluate the performance of the optimized optimal weight model on the test set. The quantitative metrics include precision, recall, F1 score, accuracy, Matthews correlation coefficient, and Cohen-Kappa coefficient. Step 4.2: For the ten categories of multi-granularity cataracts, the calculated indicators are compared with mainstream algorithms to demonstrate the effectiveness of this method in the multi-granularity cataract classification scenario.

10. A system for implementing the multi-granularity cataract classification method based on lesion perception as described in claim 1, characterized in that, The system includes: A module for building a multi-granularity cataract classification network framework is used to build such a framework. A multi-granularity cataract classification network training module is used to train a multi-granularity cataract classification network. The dataset preprocessing module is used to preprocess datasets; The multi-granularity cataract classification test module is used for classifying and testing multi-granularity cataracts.

Images