Myopic maculopathy fundus color photo image classification method and system

Abstract

The invention discloses a myopic maculopathy fundus color photo image classification method and system, and belongs to the technical field of image classification. Obtaining a to-be-analyzed fundus color photo; preprocessing the fundus color photo to extract an effective view area and reduce image acquisition difference interference; and inputting the preprocessed image into a deep learning classification network, and outputting an META-PM classification result corresponding to the myopic maculopathy. According to the method, on the premise of not depending on accurate focus labeling, the global structure and the local focus evidence are fused at the same time, the ordered level relation is explicitly modeled, and the cross-device robustness and the long-tail distribution adaptability are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and more specifically, to a method and system for classifying fundus color photos of myopic maculopathy. Background Art

[0002] Pathological myopia (PM) and its related myopic maculopathy (MM) are important causes of irreversible central visual loss. Clinically, the META-PM international grading system is often used to classify fundus color photos into five grades: C0 (without MM); C1 (tigroid fundus); C2 (diffuse chorioretinal atrophy); C3 (patchy chorioretinal atrophy); C4 (macular atrophy), which are used for disease screening, follow-up, and grading management.

[0003] Existing MM grading mainly relies on manual reading of fundus photos by fundus specialists, which has problems such as poor subjective consistency, insufficient primary medical resources, and low screening efficiency for a large number of high myopia populations; at the same time, the black edges / background interference, dark corners, and exposure differences caused by differences in fundus color photo acquisition equipment and lighting conditions will reduce the robustness of traditional computer vision and simple deep learning models.

[0004] Existing automatic grading schemes based on deep learning mostly use single-branch global feature extraction, which is difficult to stably focus on local key lesions such as macular atrophy, the boundary of patchy atrophy, and tigroid texture; in addition, META-PM belongs to an ordered grading task (C0 < C1 < C2 < C3 < C4), and confusion between adjacent grades is more common. If only trained according to ordinary multi-class classification, it is easy to have cross-level misjudgments inconsistent with clinical grading, and the performance is prone to degradation under long-tailed distributions (few high-grade samples).

[0005] Therefore, there is a need for an MM automatic grading method and system that can fuse global structure and local lesion evidence without relying on accurate lesion annotation, and explicitly model the ordered grade relationship, improve cross-device robustness, and adaptability to long-tailed distributions. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for classifying fundus color photos of myopic maculopathy to solve the technical problems existing in the above-mentioned prior art.

[0007] To achieve the above object, the present invention adopts the following technical solutions: A method for classifying fundus color photos of myopic maculopathy includes: Obtaining the fundus color photo to be analyzed; Preprocessing the fundus color photo to extract the effective visual field area and reduce the interference of image acquisition differences; The preprocessed image is input into a deep learning classification network, which includes: The global branch is used to encode the entire image and extract global features; Local branches are used to encode multiple local regions extracted from the image, and to perform weighted aggregation of multiple local features through an attention mechanism to obtain local aggregated features; The global features and the local aggregated features are fused to obtain the fused features; The fused features are input into the ordered grading output head, and the ordered grading output head is used to explicitly model the ordered progressive relationship between different lesion levels to output the META-PM grading results of the corresponding myopic macular lesions.

[0008] Furthermore, the preprocessing of the fundus photograph includes: Locate the effective circular area of ​​the fundus image; Based on the center and radius of the effective field of view circular region, the boundary is expanded outward and cropped to obtain a square region of interest image containing the complete effective field of view; The square region of interest image is scaled to obtain a standardized image.

[0009] Furthermore, locating the effective circular area of ​​vision in the fundus photograph includes: locating it through at least one of thresholding, color channel constraints, and morphological operations.

[0010] Furthermore, the preprocessing of the fundus photograph also includes: The standardized image is then subjected to illumination correction and color intensity normalization.

[0011] Furthermore, in the local branch, extracting multiple local regions from the image includes: Multiple candidate center points are set in the image; A random perturbation is superimposed on each candidate center point; Multiple local image blocks are obtained by cropping the image to a fixed size, with the perturbed position as the center.

[0012] Furthermore, in the local branch, the weighted aggregation of multiple local features through an attention mechanism includes: Calculate the attention weights corresponding to each local feature; Based on the attention weights, the multiple local features are weighted and summed to obtain the local aggregated features; Interpretive information indicating key lesion areas is obtained based on the attention weights.

[0013] Furthermore, the fusion of the global features and the local aggregated features includes: fusion through a mapping layer containing at least one of a fully connected layer, a normalization layer, or a dropout layer.

[0014] Furthermore, the deep learning classification network is trained using an ordered regression loss function during the training phase. This ordered regression loss function is configured to penalize prediction errors that are inconsistent with the clinical grading distance. During the training phase, adaptive weights are set for different threshold subtasks in the ordered regression loss function based on the sample distribution of each category in the training data to alleviate class imbalance.

[0015] A system for classifying fundus color images of myopic macular degeneration includes: Image acquisition module: Acquires color images of the fundus to be analyzed; Preprocessing module: preprocesses the fundus images to extract the effective visual field and reduce interference from differences in image acquisition; Classification module: Inputs the preprocessed image into a deep learning classification network, the deep learning classification network including: The global branch is used to encode the entire image and extract global features; Local branches are used to encode multiple local regions extracted from the image, and to perform weighted aggregation of multiple local features through an attention mechanism to obtain local aggregated features; The global features and the local aggregated features are fused to obtain the fused features; The fused features are input into the ordered grading output head, and the ordered grading output head is used to explicitly model the ordered progressive relationship between different lesion levels to output the META-PM grading results of the corresponding myopic macular lesions.

[0016] As can be seen from the above technical solution, compared with the prior art, the present invention provides a method and system for classifying fundus color images of myopic macular degeneration, which has the following beneficial effects: (1) Robustness improvement: By removing black edges / borders and invalid backgrounds through FOV extraction, and correcting vignetting and exposure differences through illumination normalization, the influence of domain offset across devices and acquisition conditions can be significantly reduced, thereby improving model stability from the source.

[0017] (2) Global-local evidence complementarity: The dual-branch structure utilizes the global structural information of the whole image and the local lesion evidence of multiple patches simultaneously; Attention-MIL can adaptively highlight key areas in the absence of precise lesion-level annotation, thereby improving the ability to identify complex lesions (such as patchy atrophy boundaries, macular atrophy, etc.) and reducing confusion between adjacent levels.

[0018] (3) It conforms to the logic of clinical grading: the ordinal output head explicit modeling of the progressive relationship of C0-C4 makes the model more consistent in the sense of "grading distance", reduces the probability of serious cross-grade misjudgment, and improves the consistency with clinical expert grading.

[0019] (4) Long-tail distribution adaptation: By setting adaptive weights for threshold subtasks, the training bias caused by the scarcity of high-level samples is alleviated, and the overall performance of tail categories such as C3 / C4 is improved.

[0020] (5) Deployability: The above process can be automatically graded using only conventional fundus color photography and computer equipment, and is suitable for scenarios such as primary screening, follow-up grading and clinical decision support. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the method flow provided by the present invention; Figure 2 This is a schematic diagram of the system structure provided by the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Example 1: See Figure 1 The present invention discloses a method for classifying fundus color images of myopic macular degeneration, comprising: Obtain color images of the fundus to be analyzed; The fundus images are preprocessed to extract the effective visual field and reduce interference from differences in image acquisition; The preprocessed image is input into a deep learning classification network, which includes: The global branch is used to encode the entire image and extract global features; Local branches are used to encode multiple local regions extracted from the image, and to perform weighted aggregation of multiple local features through an attention mechanism to obtain local aggregated features; The global features and the local aggregated features are fused to obtain the fused features; The fused features are input into the ordered grading output head, and the ordered grading output head is used to explicitly model the ordered progressive relationship between different lesion levels to output the META-PM grading results of the corresponding myopic macular lesions.

[0025] In one specific embodiment, the preprocessing of the fundus photograph includes: Locate the effective circular area of ​​the fundus image; Based on the center and radius of the effective field of view circular region, the boundary is expanded outward and cropped to obtain a square region of interest image containing the complete effective field of view; The square region of interest image is scaled to obtain a standardized image.

[0026] In one specific embodiment, locating the effective circular region of the fundus image includes: locating it through at least one of thresholding, color channel constraints, and morphological operations.

[0027] In one specific embodiment, the preprocessing of the fundus photograph further includes: The standardized image is then subjected to illumination correction and color intensity normalization.

[0028] In one specific embodiment, the local branch, extracting multiple local regions from the image, includes: Multiple candidate center points are set in the image; A random perturbation is superimposed on each candidate center point; Multiple local image blocks are obtained by cropping the image to a fixed size, with the perturbed position as the center.

[0029] In one specific embodiment, the local branch, which involves weighted aggregation of multiple local features through an attention mechanism, includes: Calculate the attention weights corresponding to each local feature; Based on the attention weights, the multiple local features are weighted and summed to obtain the local aggregated features; Interpretive information indicating key lesion areas is obtained based on the attention weights.

[0030] In one specific embodiment, fusing the global features and the local aggregated features includes fusing them through a mapping layer that includes at least one of a fully connected layer, a normalization layer, or a dropout layer.

[0031] In one specific embodiment, the deep learning classification network is trained using an ordered regression loss function during the training phase. The ordered regression loss function is configured to penalize prediction errors that are inconsistent with the clinical grading distance. During the training phase, adaptive weights are set for different threshold subtasks in the ordered regression loss function based on the sample distribution of each category in the training data to alleviate class imbalance.

[0032] On the other hand, see Figure 2 Embodiment 1 of the present invention also discloses a system for classifying fundus color images of myopic macular degeneration, comprising: Image acquisition module: Acquires color images of the fundus to be analyzed; Preprocessing module: preprocesses the fundus images to extract the effective visual field and reduce interference from differences in image acquisition; Classification module: Inputs the preprocessed image into a deep learning classification network, the deep learning classification network including: The global branch is used to encode the entire image and extract global features; Local branches are used to encode multiple local regions extracted from the image, and to perform weighted aggregation of multiple local features through an attention mechanism to obtain local aggregated features; The global features and the local aggregated features are fused to obtain the fused features; The fused features are input into the ordered grading output head, and the ordered grading output head is used to explicitly model the ordered progressive relationship between different lesion levels to output the META-PM grading results of the corresponding myopic macular lesions.

[0033] Example 2: Embodiment 2 of this invention discloses a method and system for automatic grading of myopic macular degeneration (META-PM, C0-C4) based on fundus color photography. This scheme includes at least: (1) Preprocessing workflow for effective field of view (FOV) extraction and illumination normalization, used to suppress black borders, camera borders and lighting differences; (2) A dual-branch deep learning network that integrates global and local evidence: the global branch encodes the entire fundus image, and the local branch encodes multiple patches and adaptively aggregates key lesion regions through attention-based multiple instance learning (Attention-MIL); (3) The Ordinal output head and corresponding training strategy are used to explicitly model the progressive relationship between C0 and C4, and the adaptive weights of the class distribution are combined to alleviate the bias caused by the scarcity of high-level samples; thereby achieving high-precision, deployable automatic classification and assisted screening of MM.

[0034] In one specific embodiment, a five-level intelligent grading method for myopic macular degeneration (META-PM) based on fundus photography (CFP) is disclosed. This method is executed by a computing device, oriented towards a clinical grading system, directly outputting the input CFP as one of five levels (C0, C1, C2, C3, C4), and can also output interpretive information of key local evidence. The steps are as follows: S1: Data Acquisition and Standardized Input Obtain color images of the fundus of the subject. I (Images can come from different devices and shooting conditions), unify the image format and color space, and use them as input for subsequent processing.

[0035] S2: Effective Field of View (FOV) Positioning and ROI Clipping: To reduce the effects of black borders, background, and shooting off-center, the input image is... I Perform effective field of view (FOV) positioning and ROI clipping, specifically including: 1. Convert the image to grayscale and use adaptive thresholding to obtain an initial binary mask for the field of view. The thresholding method can be Otsu thresholding. When the foreground / background polarity of the mask is inconsistent with the actual field of view, perform phase inversion processing on the mask. 2. Perform morphological closing operations on the binary mask to fill holes and connect the field of view. The morphological structuring element can be selected as an elliptical structuring element, and the number of iterations of the closing operation can be 1 to 3. 3. Perform connected component analysis on the mask after the closing operation, and select the connected component with the largest area as the effective field of view; when the area of ​​the largest connected component is lower than the preset ratio threshold of the image area, the localization is determined to be unsuccessful and the entire image is used as the candidate field of view; 4. Extract the outer contour of the effective connected region of the field of view, and estimate the center of the field of view using the minimum circumcircle. c x , c y ) and radius r ; 5. In the radius r Based on the preset expansion ratio, redundant expansion is performed, the cropping boundary is calculated, and a square ROI containing the complete effective field of view is obtained; the ROI is then scaled to a uniform input size to obtain a standardized image. I roi .

[0036] S3: Illumination and Color Normalization Preprocessing: Standardized images I roi Perform illumination correction and intensity normalization to mitigate exposure differences and cross-device domain shift: 1. Normalize or correct the luminance channel (e.g., enhance local contrast / correct illumination). 2. Standardize the strength of the three channels; 3. During the training phase, data augmentation (rotation, scaling, brightness / contrast perturbation, mild blurring / noise, low-probability compression artifact simulation, etc.) is introduced to obtain augmented samples. I aug .

[0037] S4: Global View Feature Extraction Using the entire standardized ROI image ( I roi Or enhance the sample I aug ) as global input x g Construct a global branch network to extract global representations. f g The global branch network preferably adopts a ConvNeXt backbone network, and its hierarchical structure may include: 1) Input mapping layer: for input x g Perform initial convolution mapping and downsampling to obtain the initial feature map; 2) Fourth-level feature extraction stage: Step 1: Composed of several ConvNeXt Blocks connected in series, used to extract low-level texture and edge information; Step 2: Before entering this stage, the spatial resolution is reduced and the channel dimension is increased through a downsampling layer, and the mid-layer structural features are extracted by several ConvNeXt Blocks; Step 3: Continue downsampling and stacking ConvNeXt Blocks to enhance the representation of overall retinal morphological changes and wide-area texture patterns; Step 4: Further downsample and stack ConvNeXt Blocks to obtain high-level semantic representations; 3) Global Pooling and Flattening: Perform global average pooling and flattening on the feature map output from step 4 to obtain a fixed-dimensional global feature vector.

[0038] In the formula, ConvNeXt (·) denotes a global feature extraction network with ConvNeXt as its backbone; GAP(·) indicates global average pooling; Flatten (·) indicates a flattening operation; f g This is the global representation vector.

[0039] In a specific implementation, the global branch can use ConvNeXt-Base, and its feature extraction part can be cascaded with global average pooling and flattening layers to form a global encoder, thereby outputting fg.

[0040] Global representation is used to depict overall structural changes and wide-area texture patterns, avoiding omissions or instability caused by relying solely on local clues.

[0041] S5: Multi-local patch generation: To capture fine-grained cues around the macula, multiple local views are generated without relying on precise anatomical localization annotations: 1. In I roi Several candidate center point sets are set (e.g., center, left and right off-center, top and bottom off-center, etc.); 2. Add slight random perturbations to the candidate center points; 3. Cut to a fixed size N A local patch: .

[0042] This strategy covers potentially critical lesion areas with low prior dependence.

[0043] S6: Local field-of-view feature extraction: Each local patch is input into the local branch feature extraction network:

[0044] In the formula, P i This represents the i-th local region image patch, which is obtained by cropping a standardized ROI image according to a preset candidate center point and perturbation strategy; i represents the index number of the local region image patch, i=1,…,N; N represents the number of local region image patches; F l (·) represents a local branch feature extraction network (local encoder), which is used to map the input local region image patch into a feature representation. It can be composed of a convolutional neural network backbone and a global average pooling and flattening layer cascaded together. f i This indicates that the local branch feature extraction network is used to extract the image patch of the i-th local region. P i The extracted local feature vectors are used to characterize fine-grained lesion clues within the local region.

[0045] Obtain the local feature set { f i It is used to characterize detailed clues such as focal atrophy, pigmentation changes, and patchy changes.

[0046] S7: Attention-based multi-instance fusion: Attention weighting is applied to local feature sets to automatically select the most critical local evidence for grading: 1. Calculate the importance weight of each patch. α i ; 2. Weighted convergence is performed to obtain localized aggregation characteristics:

[0047] In the formula, f l This represents the local aggregated feature vector output by the attention multi-instance learning aggregation module, used to characterize key local lesion evidence in the entire image; N represents the number of local region image patches; i represents the index number of the i-th local region image patch, i=1,…,N; f i This represents the local feature vector extracted from the i-th local region image patch by the local branch feature extraction network; α i Represents the local feature vector f i The corresponding attention weight coefficient is used to characterize the contribution of the i-th local region to the final judgment.

[0048] Attention weights { α i This can be used to output key local evidence and improve the interpretability of the results.

[0049] S8: Global-Local Collaborative Fusion global representation f g Local aggregation characterization f l The fusion process, through a mapping layer, yields the fused features. f :

[0050] in Φ(·) It can include fully connected layers, normalization, and Dropout to achieve complementarity between global structural information and local lesion evidence.

[0051] S9: C0–C4 five-level hierarchical output: Fusion features fInput an ordered hierarchical output header to output five levels. C 0~ C 4 (K=5) is transformed into K-1=4 threshold subtasks {y>0, y>1, y>2, y>3}, outputting 4 logit values:

[0052] The threshold pass probability is obtained by performing a sigmoid mapping on the logit:

[0053] Inference using a threshold τ = 0.5 converts the probability to an indication:

[0054] The final predicted grade is the sum of the number of grades that pass the threshold:

[0055] And map them respectively to { C 0, C 1, C 2, C 3, C 4} is the final classification result.

[0056] In the above formula, f This represents the fused feature vector obtained by fusing global branch features and local aggregate features; W and b These represent the weight matrix and bias vector of the ordered hierarchical output head, respectively. z This represents the threshold subtask logit vector generated by the output head; z k The logit represents the k-th threshold subtask; σ(·) represents the sigmoid function. p k This represents the probability that the k-th threshold subtask "passes"; τ The threshold value represents the probability discrimination threshold; I(·) represents the indicator function, which takes the value 1 when the condition in parentheses is true, and 0 otherwise. b k This indicates the pass / fail indication for the k-th threshold subtask; This represents the final predicted classification result, with values ​​ranging from {0, 1, 2, 3, 4}, corresponding to... C 0, C 1, C 2, C 3, C 4.

[0057] This five-level output strictly corresponds to the META-PM grading system and can be directly used for clinical stratification management and follow-up assessment.

[0058] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0059] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for classifying fundus color images of myopic macular degeneration, characterized in that, include: Obtain color images of the fundus to be analyzed; The fundus images are preprocessed to extract the effective visual field and reduce interference from differences in image acquisition; The preprocessed image is input into a deep learning classification network, which includes: The global branch is used to encode the entire image and extract global features; Local branches are used to encode multiple local regions extracted from the image, and to perform weighted aggregation of multiple local features through an attention mechanism to obtain local aggregated features; The global features and the local aggregated features are fused to obtain the fused features; The fused features are input into the ordered grading output head, and the ordered grading output head is used to explicitly model the ordered progressive relationship between different lesion levels to output the META-PM grading results of the corresponding myopic macular lesions.

2. The method for classifying fundus color images of myopic macular degeneration according to claim 1, characterized in that, The preprocessing of the fundus color image includes: Locate the effective circular area of ​​the fundus image; Based on the center and radius of the effective field of view circular region, the boundary is expanded outward and cropped to obtain a square region of interest image containing the complete effective field of view; The square region of interest image is scaled to obtain a standardized image.

3. The method for classifying fundus color images of myopic macular degeneration according to claim 2, characterized in that, The method of locating the effective circular area of ​​vision in the fundus color image includes: locating it through at least one of thresholding, color channel constraints, and morphological operations.

4. The method for classifying fundus color images of myopic macular degeneration according to claim 2, characterized in that, The preprocessing of the fundus color image also includes: The standardized image is then subjected to illumination correction and color intensity normalization.

5. The method for classifying fundus color images of myopic macular degeneration according to claim 1, characterized in that, In the local branch, extracting multiple local regions from the image includes: Multiple candidate center points are set in the image; Random perturbations are superimposed on each candidate center point; Multiple local image blocks are obtained by cropping the image to a fixed size, with the perturbed position as the center.

6. The method for classifying fundus color images of myopic macular degeneration according to claim 1, characterized in that, In the local branch, the weighted aggregation of multiple local features through an attention mechanism includes: Calculate the attention weights corresponding to each local feature; Based on the attention weights, the multiple local features are weighted and summed to obtain the local aggregated features; Interpretive information indicating key lesion areas is obtained based on the attention weights.

7. The method for classifying fundus color images of myopic macular degeneration according to claim 1, characterized in that, The fusion of the global features and the local aggregated features includes fusing them through a mapping layer containing at least one of a fully connected layer, a normalization layer, or a dropout layer.

8. The method for classifying fundus color images of myopic macular degeneration according to claim 1, characterized in that, The deep learning classification network is trained using an ordered regression loss function during the training phase. This ordered regression loss function is configured to penalize prediction errors that are inconsistent with the clinical grading distance. During the training phase, adaptive weights are set for different threshold subtasks in the ordered regression loss function based on the sample distribution of each category in the training data to alleviate class imbalance.

9. A system for classifying fundus color images of myopic macular degeneration according to any one of claims 1 to 8, characterized in that, include: Image acquisition module: Acquires color images of the fundus to be analyzed; Preprocessing module: preprocesses the fundus images to extract the effective visual field and reduce interference from differences in image acquisition; Classification module: Inputs the preprocessed image into a deep learning classification network, the deep learning classification network including: The global branch is used to encode the entire image and extract global features; Local branches are used to encode multiple local regions extracted from the image, and to perform weighted aggregation of multiple local features through an attention mechanism to obtain local aggregated features; The global features and the local aggregated features are fused to obtain the fused features; The fused features are input into the ordered grading output head, and the ordered grading output head is used to explicitly model the ordered progressive relationship between different lesion levels to output the META-PM grading results of the corresponding myopic macular lesions.

Images