DR focus segmentation method and system based on dynamic loss optimization function
By dynamically adjusting the weights and thresholds of the loss function, a large RTSAM model is constructed, which solves the problem of imbalanced datasets in DR lesion segmentation, achieves high-precision segmentation of rare and small lesions, and optimizes the training effect of the model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-14
AI Technical Summary
Existing loss functions struggle to effectively identify rare and minute lesions when dealing with imbalanced datasets in DR lesion segmentation, resulting in suboptimal segmentation performance.
A dynamic loss optimization function is adopted, and by adjusting the weights of the loss function and the dynamic threshold, the model's attention to rare and small lesion areas is enhanced, and a large RTSAM model is constructed for training.
It significantly improved the model's segmentation accuracy for minor lesions, optimized the training effect for difficult samples, and improved the accuracy and visualization results of DR lesion segmentation.
Smart Images
Figure CN121860979A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of image segmentation technology, specifically relating to a method and system for DR lesion segmentation based on dynamic loss optimization function. Background Technology
[0002] Diabetic retinopathy (DR) is a common fundus disease. Due to the complex structure of the fundus and the diverse and significantly different types of lesions, it places high demands on medical resources and the diagnostic experience of physicians. With the development of computer technology, automated segmentation, which is more realistic, efficient, and simpler than manual segmentation, has played an important role in ophthalmic diagnosis. In DR multi-lesion segmentation tasks, it is necessary to effectively handle the conflicts and imbalances between different types of lesions. These lesions have significant differences in distribution and features in the images, causing the model to tend to favor common lesions during training. However, because some small lesions occur less frequently in the images, and background areas or other common lesions (such as hard exudates) occupy a large proportion, this imbalance affects the ability to identify rare lesions. Therefore, the selection of the loss function becomes crucial to improving segmentation accuracy during model training.
[0003] In segmentation networks with different structures, loss functions are continuously optimized and improved to further enhance the training performance of the models. For example, Liu Yuanyuan's "E2E-DRNet: Diabetic Retinopathy Recognition Based on EfficientNetV2 Model" introduces the Focal Loss function in E2E-DRNet to address the imbalanced sample problem. While improving the screening efficiency of DR lesions, it only completes basic lesion classification and has not yet achieved accurate lesion region segmentation. Furthermore, Guo Xiaoxin's "Multimodal Fundus Disease Diagnosis Model Based on Multi-level Multi-scale Attention Fusion Network" designs a multi-branch loss function that weights and sums the loss functions of color fundus image analysis, retinal optical coherence tomography, and fusion feature analysis, improving the diagnostic performance of multimodal fundus diseases through a collaborative optimization scheme. Although these loss functions effectively improve the segmentation ability of the models, they mainly target U-Net series networks and Transformer-related technologies; the design of loss functions for large SAM models has not received sufficient attention and research. Therefore, designing a loss function that enhances the SAM model's focus on rare and minor lesions during training is a problem worthy of current research. Summary of the Invention
[0004] This application provides a DR lesion segmentation method and system based on dynamic loss optimization function to solve the technical problems of traditional loss function when dealing with imbalanced datasets.
[0005] To solve the above-mentioned technical problems, this application adopts the following technical solution: a DR lesion segmentation method based on a dynamic loss optimization function, comprising:
[0006] S1. Collect image data from the IDRID dataset and preprocess it to obtain preprocessed data;
[0007] S2. Based on the preprocessed data, construct a large visual SAM model and design a loss function weight optimization strategy to obtain an RTSAM segmentation model;
[0008] S3. Segment the DR fundus image based on the RTSAM segmentation model to obtain the DR lesion segmentation results;
[0009] S4. Based on the DR lesion segmentation results, provide automatic identification and segmentation visualization results of DR lesions.
[0010] Furthermore, the method in step S1 includes:
[0011] S11. Collect IDRiD fundus image data, scale the fundus images with a resolution of 4288×2848 in the IDRiD dataset to the target resolution of 268×178, and obtain the first dataset;
[0012] S12. Based on the mask image corresponding to each fundus image in the first dataset, the mask size is uniformly adjusted to 268×178 using the same scaling parameters as the original image to obtain the second dataset;
[0013] S13. Perform grayscale value mapping on the four lesion types in the mask of the second dataset to obtain grayscale lesion mask images;
[0014] S14. The four grayscale lesion mask images are superimposed and fused at the pixel level to generate a single fused segmentation mask image, and the preprocessed data is obtained; where different grayscale values correspond to different lesion regions.
[0015] Furthermore, the method in step S2 includes:
[0016] S21. Based on the preprocessed data, use an image encoder to convert the input DR fundus image into feature encoding to generate a feature map;
[0017] S22. Based on the preprocessed data, the cue encoder is used to convert the point cue and box cue information into cue embedding vectors;
[0018] S23. Based on the mask decoder, the feature map and the prompt embedding vector are fused and stitched together, and the resolution of the image is gradually restored to restore the lesion boundary and output the binary mask of the lesion region;
[0019] S24. Construct a dynamic loss optimization function;
[0020] S25. Input the preprocessed fundus images and their corresponding masks into the fine-tuned visual SAM large model, and train it using the designed loss function;
[0021] S26. Iteratively train the fine-tuned visual SAM large model, save the training results of each round, and determine the RTSAM large model.
[0022] Furthermore, the method in step S24 includes:
[0023] Based on formula (1), the loss function is obtained; where formula (1) is:
[0024] (1);
[0025] Where M represents the total number of pixels, and C represents the total number of lesion categories. It is the label value of the i-th pixel. This indicates an indicator function; its value is 1 when a certain condition is met, and 0 otherwise. It is the predicted probability value that the i-th pixel belongs to the k-th class. This represents a dynamic threshold, the value of which changes dynamically with the model's performance.
[0026] Furthermore, based on formula (2), the probability value is obtained. ; where formula (2) is:
[0027] (2);
[0028] in, It is the original prediction score of the i-th pixel for the k-th class. This indicates that the mean is 0 and the variance is 0. Gaussian noise; Let be the noise weight parameter for the k-th class;
[0029] Based on formula (3), the noise weight parameters are obtained. ; where formula (3) is:
[0030] (3);
[0031] in, This represents the total number of pixels in the k-th class across all images in the training set. This represents the total number of pixels in all images in the training set.
[0032] Furthermore, the method in step S3 includes:
[0033] S31. Collect common DR fundus images, preprocess the DR fundus images and input them into the trained RTSAM large model;
[0034] S32. Based on the RTSAM large model, the automatic identification and segmentation results of the lesion region in the input image are obtained.
[0035] Another technical solution adopted in this application is: a DR lesion segmentation system based on a dynamic loss optimization function, comprising:
[0036] The image input module is used to configure the high-speed image acquisition device to receive color fundus images.
[0037] The preprocessing module is used to perform preprocessing operations on the input fundus image, including but not limited to image size normalization, color correction, noise removal and enhancement processing;
[0038] The RTSAM model inference module is used to load the trained RTSAM large model, input the preprocessed fundus image, and automatically identify and segment the diabetic retinopathy region.
[0039] The segmentation result generation module is used to generate a binary segmentation mask based on the model output, which identifies the specific location and extent of the lesion area in the fundus image; the segmentation mask is superimposed on the original fundus image to clearly display the lesion area;
[0040] The interaction and output module provides a user-friendly interface, enabling image uploading, result display, and interactive operations. It also supports saving and exporting segmentation result images.
[0041] The beneficial effects of this application are as follows: This application addresses the problem of uneven lesion distribution and uneven lesion frequency in color fundus images of DR, which leads to poor segmentation results. Therefore, a dynamic loss optimization strategy is used to adaptively adjust the weight allocation in the loss function according to the model segmentation index, thereby enhancing the model's attention to rare and small lesion regions. This application proposes a loss function optimization strategy for large SAM models, which effectively overcomes the limitations of traditional loss functions in handling imbalanced datasets, significantly improves the model's segmentation accuracy for small lesions, and enables the model to focus more on difficult-to-segment regions by dynamically adjusting the threshold and weighted training, thus optimizing the training effect of difficult samples. Attached Figure Description
[0042] Figure 1 This is a flowchart illustrating an embodiment of the DR lesion segmentation method based on dynamic loss optimization function of this application;
[0043] Figure 2 yes Figure 1 A flowchart illustrating step S1 of an embodiment;
[0044] Figure 3 yes Figure 1 A flowchart illustrating step S2 of an embodiment;
[0045] Figure 4 yes Figure 1 A flowchart of an embodiment of step S2;
[0046] Figure 5 This is an overview diagram of the loss function optimization strategy of an embodiment of step S2 of this application;
[0047] Figure 6 This is a structural block diagram of an embodiment of the DR lesion segmentation system based on dynamic loss optimization function of this application;
[0048] Figure 7 This is a visualization of the results of a comparison of different algorithms for an embodiment of the DR lesion segmentation method based on dynamic loss optimization function of this application. Detailed Implementation
[0049] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments.
[0050] Numerous specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways than those described herein, and therefore the invention is not limited to the specific embodiments disclosed in the following specification.
[0051] See Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the DR lesion segmentation method based on a dynamic loss optimization function according to this application. The method includes:
[0052] S1. Collect image data from the IDRID dataset and preprocess it to obtain preprocessed data.
[0053] For details, please refer to Figure 2 The method of step S1 includes:
[0054] S11. Collect IDRiD fundus image data. Scale the fundus images in the IDRiD dataset with a resolution of 4288×2848 to a target resolution of 268×178 to ensure the integrity of the image content and that the proportions are not distorted, thereby obtaining the first dataset.
[0055] S12. For the mask image corresponding to each fundus image in the first dataset, use the same scaling parameters as the original image to uniformly adjust the mask size to 268×178 to ensure that the spatial correspondence between the mask and the fundus image remains consistent, and obtain the second dataset;
[0056] S13. Perform grayscale value mapping on the four lesion types in the mask in the second dataset to obtain grayscale lesion mask images; the specific mapping rules are as follows: microaneurysm corresponds to grayscale value 70, hemorrhage corresponds to grayscale value 120, soft exudate corresponds to grayscale value 170, and hard exudate corresponds to grayscale value 220.
[0057] S14. The four grayscale lesion mask images after grayscale value conversion are superimposed and fused at the pixel level to generate a single fused segmentation mask image, in which different grayscale values correspond to different lesion regions; and the fused segmentation mask image is uniformly named and renamed according to the pre-set rules to facilitate subsequent data management and retrieval.
[0058] S2. Based on the preprocessed data, construct a large visual SAM model and design a loss function weight optimization strategy to obtain the RTSAM segmentation model.
[0059] For details, please refer to Figure 3-4 The method of step S2 includes:
[0060] S21. Based on the preprocessed data, use an image encoder to convert the input DR fundus image into feature encoding to generate a feature map.
[0061] S22. Based on the preprocessed data, the cue encoder is used to convert the point cue and box cue information into cue embedding vectors.
[0062] S23. Based on the mask decoder, the feature map and the prompt embedding vector are fused and stitched together, and the resolution of the image is gradually restored to restore the lesion boundary and output the binary mask of the lesion region.
[0063] S24. Construct a dynamic loss optimization function to further improve the robustness of the model during training. The specific formula for this loss function is as follows:
[0064] (1);
[0065] Where M represents the total number of pixels, and C represents the total number of lesion categories. It is the label value of the i-th pixel. This indicates an indicator function; its value is 1 when a certain condition is met, and 0 otherwise. It is the predicted probability value that the i-th pixel belongs to the k-th class. This represents a dynamic threshold, the value of which changes dynamically with model performance. (Probability value) The specific formula is as follows:
[0066] (2);
[0067] in, It is the original prediction score of the i-th pixel for the k-th class. This indicates that the mean is 0 and the variance is 0. Gaussian noise; Let be the noise weight parameter for the k-th class;
[0068] Based on formula (3), the noise weight parameters are obtained. ; where formula (3) is:
[0069] (3);
[0070] in, This represents the total number of pixels in the k-th class across all images in the training set. This represents the total number of pixels in all images in the training set. Adjusted parameters. It is inversely proportional to the number of categories. This means that by increasing the prediction probability of lesions and perturbing the background information, the model can make the features of the lesion region account for a larger proportion in the entire prediction process, thereby improving the recognition rate of the lesion region in the final prediction.
[0071] It is inversely proportional to the number of categories. This means that by increasing the prediction probability of lesions and perturbing the background information, the model can make the features of the lesion region account for a larger proportion in the entire prediction process, thereby improving the recognition rate of the lesion region in the final prediction.
[0072] S25. Input the preprocessed fundus images and their corresponding masks into the fine-tuned visual SAM large model, and train it using the designed loss function;
[0073] S26. Select the AdaGrad optimizer, set the batch size and initial learning rate to 32 and 0.001 respectively, and use the calculator of the NVIDIA RTX 4090 GPU with 48GB of video memory to iteratively train the fine-tuned large model; save the training results of each round and determine the final RTSAM large model.
[0074] S3. Segment the DR fundus image based on the RTSAM segmentation model to obtain the DR lesion segmentation results.
[0075] Specifically, the method in step S3 includes:
[0076] S31. Collect common DR fundus images, preprocess the DR fundus images and input them into the trained RTSAM large model.
[0077] S32. Based on the RTSAM large model, the automatic identification and segmentation results of the lesion region in the input image are obtained.
[0078] S4. Based on the DR lesion segmentation results, provide automatic identification and segmentation visualization results of DR lesions.
[0079] See Figure 6 , Figure 6 This is a structural block diagram of an embodiment of the DR lesion segmentation system based on a dynamic loss optimization function according to this application. The system includes:
[0080] Image input module 1 is used to configure a high-speed image acquisition device to receive color fundus images as input;
[0081] Preprocessing module 2 is used to perform preprocessing operations on the input fundus image, including but not limited to image size normalization, color correction, noise removal and enhancement processing;
[0082] RTSAM model inference module 3 is used to load the trained RTSAM large model, input the preprocessed fundus image, and automatically identify and segment the diabetic retinopathy region.
[0083] The segmentation result generation module 4 is used to generate a binary segmentation mask based on the model output, which identifies the specific location and extent of the lesion area in the fundus image; the segmentation mask is superimposed on the original fundus image to clearly display the lesion area;
[0084] The interaction and output module provides a user-friendly interface, enabling image uploading, result display, and interactive operations. It also supports saving and exporting segmentation result images.
[0085] To verify the effectiveness of the present invention, this embodiment was compared with the current mainstream fundus image segmentation models on the IDRid dataset, and the specific results are shown in Table 1.
[0086] Table 1 shows the comparison of segmentation results for the IDRID dataset.
[0087] method Dice (%) IoU (%) HD TransUnet 54.69 45.36 9.23 nnUnet 67.54 55.46 6.22 nnFormer 65.02 53.86 5.93 SAM (original) 26.83 22.49 8.85 MedSAM 49.42 41.73 9.07 RTSAM (ours) 83.19 77.90 5.68
[0088] Table 1 shows the performance metrics of this embodiment's method compared to other models in DSC, IoU, and HD on the IDRiD dataset. The DSC value of this embodiment reaches 83.19%, at least 15.65% higher than methods such as nnUNet; the IoU value reaches 77.90%, at least 22.44% higher than methods such as nnUNet; and the HD value is 5.68, at least 0.54 lower than segmentation methods such as nnFormer. This indicates that the method of this embodiment effectively optimizes the model training process, thereby achieving more accurate results in the DR lesion segmentation task.
[0089] See Figure 2The model itself was not improved. Instead, during training, a dynamic loss optimization strategy was proposed, which modifies the traditional loss function. This strategy automatically adjusts the weights of the loss function based on the difficulty of segmenting different lesions, thereby enhancing its focus on rare and small lesion regions. By dynamically adjusting the threshold and weighting the training, the model can more effectively handle difficult-to-segment regions, thus optimizing the training effect on challenging samples.
[0090] See Figure 5 , Figure 5 This diagram outlines the loss function optimization strategy of the DR lesion segmentation method based on dynamic loss optimization proposed in this embodiment. The dynamic loss optimization strategy enhances the training of small lesion area samples while balancing the lesion category distribution and reducing the negative impact of the background on the model. This allows the model to focus more on processing difficult-to-segment areas and optimizes the training effect of the model.
[0091] See Figure 7 , Figure 7 This is a visualization of the algorithm comparison results of the DR lesion segmentation method based on dynamic loss optimization function proposed in this embodiment on the IDRiD dataset. The fundus image contains various lesions such as microaneurysms, hemorrhages, and soft infiltrations. The lesions have significant differences in shape and size. It is not difficult to find that the method in this embodiment can accurately distinguish small lesions next to large lesions and reduce the probability of misjudgment and missed judgment.
[0092] This application addresses the issue of uneven lesion distribution and frequency in color fundus images of DR (Diabetic Retinopathy), leading to suboptimal segmentation results. Therefore, a dynamic loss optimization strategy is employed, adaptively adjusting the weight allocation in the loss function based on the model's segmentation metrics to enhance the model's focus on rare and small lesion regions. For large SAM (Resilient Segmentation Model) models, this application proposes a loss function optimization strategy that effectively overcomes the limitations of traditional loss functions in handling imbalanced datasets, significantly improving the model's segmentation accuracy for minute lesions. By dynamically adjusting the threshold and weighted training, the model can more effectively focus on segmenting difficult regions, optimizing the training effect on challenging samples. This application provides a DR lesion segmentation device based on a dynamic loss optimization function, utilizing the RTSAM model to segment DR lesions and providing and storing visualized lesion segmentation results to users, thereby improving the efficiency of doctors' diagnosis and treatment.
[0093] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A DR lesion segmentation method based on a dynamic loss optimization function, characterized in that, include: S1. Collect image data from the IDRID dataset and preprocess it to obtain preprocessed data; S2. Based on the preprocessed data, construct a large visual SAM model and design a loss function weight optimization strategy to obtain an RTSAM segmentation model; S3. Segment the DR fundus image based on the RTSAM segmentation model to obtain the DR lesion segmentation result; S4. Based on the DR lesion segmentation results, provide automatic identification and segmentation visualization results of DR lesions.
2. The method according to claim 1, characterized in that, The method of step S1 includes: S11. Collect IDRiD fundus image data, scale the fundus images with a resolution of 4288×2848 in the IDRiD dataset to the target resolution of 268×178, and obtain the first dataset; S12. Based on the mask image corresponding to each fundus image in the first dataset, the mask size is uniformly adjusted to 268×178 using the same scaling parameters as the original image to obtain the second dataset; S13. Perform grayscale value mapping on the four lesion types in the mask of the second dataset to obtain grayscale lesion mask images; S14. The four grayscale lesion mask images are pixel-level superimposed and fused to generate a single fused segmentation mask image, and the preprocessed data is obtained; wherein different grayscale values correspond to different lesion regions.
3. The method according to claim 1, characterized in that, The method of step S2 includes: S21. Based on the preprocessed data, the input DR fundus image is converted into feature encoding using an image encoder to generate a feature map; S22. Based on the preprocessed data, the point hints and box hints are converted into hint embedding vectors using a cue encoder; S23. Based on the mask decoder, the feature map and the prompt embedding vector are fused and spliced together, and the resolution of the image is gradually restored to restore the lesion boundary and output the binary mask of the lesion region; S24. Construct a dynamic loss optimization function; S25. Input the preprocessed fundus images and their corresponding masks into the fine-tuned visual SAM large model, and train it using the designed loss function; S26. Iteratively train the fine-tuned visual SAM large model, save the training results of each round, and determine the RTSAM large model.
4. The method according to claim 3, characterized in that, The method of step S24 includes: Based on formula (1), the loss function is obtained; wherein, formula (1) is: （1）； Where M represents the total number of pixels, and C represents the total number of lesion categories. It is the label value of the i-th pixel. This indicates an indicator function; its value is 1 when a certain condition is met, and 0 otherwise. It is the predicted probability value that the i-th pixel belongs to the k-th class. This represents a dynamic threshold, the value of which changes dynamically with the model's performance.
5. The method according to claim 4, characterized in that, Based on formula (2), the probability value is obtained. ; where formula (2) is: （2）； in, It is the original prediction score of the i-th pixel for the k-th class. This indicates that the mean is 0 and the variance is 0. Gaussian noise; Let be the noise weight parameter for the k-th class; Based on formula (3), the noise weight parameters are obtained. ; where formula (3) is: （3）； in, This represents the total number of pixels in the k-th class across all images in the training set. This represents the total number of pixels in all images in the training set.
6. The method according to claim 3, characterized in that, The method of step S3 includes: S31. Collect common DR fundus images, preprocess the DR fundus images and input them into the trained RTSAM large model; S32. Based on the RTSAM large model, the automatic identification and segmentation results of the lesion region in the input image are obtained.
7. A DR lesion segmentation system based on a dynamic loss optimization function, characterized in that, include: The image input module is used to configure the high-speed image acquisition device to receive color fundus images. The preprocessing module is used to perform preprocessing operations on the input fundus images, including but not limited to image size normalization, color correction, noise removal, and enhancement processing. The RTSAM model inference module is used to load the trained RTSAM large model, input the preprocessed fundus image, and automatically identify and segment the diabetic retinopathy region. The segmentation result generation module is used to generate a binary segmentation mask based on the model output, which identifies the specific location and extent of the lesion area in the fundus image; the segmentation mask is superimposed on the original fundus image to clearly display the lesion area; The interaction and output module provides a user-friendly interface, enabling image uploading, result display, and interactive operations. It also supports saving and exporting segmentation result images.