A ROP laser spot quantification method and device

Abstract

This application belongs to the field of image processing technology and discloses a method and device for quantifying ROP laser spots, including: outlining the boundaries of laser spots in multiple fundus laser images; binarizing each fundus laser image to obtain a mask; using the mask as a true label to obtain a trained laser spot segmentation model; obtaining the area of the laser spot enclosed by the laser spot boundary in each fundus laser image; using the area of the laser spot in each fundus laser image as a true label for training to obtain a trained area prediction model; obtaining a to-be-diagnosed image, inputting it into the laser spot segmentation model to obtain a laser spot segmentation region; inputting the laser spot segmentation region into the area prediction model to obtain a predicted laser spot area; calculating the circular area of the fundus circular region, and dividing the predicted laser spot area by the circular area to obtain a laser area index. This application can provide an objective quantification method for predicting the fundus prognosis and refractive state of children after laser photocoagulation, and improve the diagnosis and treatment efficiency of ROP children.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a method and device for quantifying ROP laser spots. Background Art

[0002] Currently, due to the advocacy and popularization of premature infant fundus screening, more and more children with retinopathy of prematurity (ROP) are receiving early treatment. Researchers have also begun to pay attention to the prognosis of ROP children after laser photocoagulation. Many research results have shown that treatments such as laser will increase the occurrence of refractive errors in ROP children.

[0003] Currently, the prognosis of ROP is mainly judged by ophthalmology professionals with relatively rich experience. However, due to limited and unevenly distributed expert resources, and the instability of subjective judgment of ROP prognosis, it is impossible to efficiently and objectively judge the fundus prognosis and refractive status of children after laser photocoagulation treatment. At present, there is no objective quantification method to evaluate the degree of laser photocoagulation treatment and its impact on the refractive status and fundus development of children. Summary of the Invention

[0004] This application provides a method and device for quantifying ROP laser spots, which can segment and quantify the area of laser spots after laser photocoagulation, and provides an objective quantification method for predicting the fundus prognosis and refractive status of children after ROP laser photocoagulation.

[0005] In a first aspect, an embodiment of this application provides a method for quantifying ROP laser spots, including:

[0006] Obtain multiple laser fundus images, and outline the boundaries of laser spots in each laser fundus image with solid lines;

[0007] Binarize each laser fundus image to obtain a mask corresponding to each laser fundus image;

[0008] Input each batch of laser fundus images into a first network model, and use the mask of each laser fundus image as the true label for training to obtain a trained laser spot segmentation model;

[0009] Obtain the area of the laser spot enclosed by the boundary of the laser spot in each laser fundus image;

[0010] Input each batch of laser fundus images into a second network model, and use the area of the laser spot in each laser fundus image as the true label for training to obtain a trained area prediction model;

[0011] Obtain a to-be-diagnosed image and input it into the laser spot segmentation model to obtain a laser spot segmentation region;

[0012] Input the laser spot segmentation area into the area prediction model to obtain the predicted area of the laser spot;

[0013] Calculate the circular area of the fundus circular area in the image to be diagnosed, and divide the predicted area of the laser spot by the circular area to obtain the laser area index.

[0014] Furthermore, the method further includes:

[0015] Calculate the optic disc area of the image to be diagnosed; compare the predicted area of the laser spot with the optic disc area to obtain the optic disc area percentage.

[0016] Furthermore, the method further includes: before training the first network model, adjust each laser fundus image and its corresponding mask to a preset size, and perform normalization processing on each mask of the preset size.

[0017] Furthermore, the method further includes: before training the second network model, adjust each laser fundus image to a preset size and perform normalization processing so that the pixel values of each laser fundus image are within a preset range.

[0018] Furthermore, the preset size is 800×800, and the preset range is 0-1.

[0019] Furthermore, the first network model is a Unet architecture model.

[0020] Furthermore, the second network model is a Resnet50 model.

[0021] In a second aspect, an ROP laser spot quantification device provided by an embodiment of the present application includes:

[0022] An outlining module for obtaining a plurality of laser fundus images and outlining the laser spot boundaries in each laser fundus image with solid lines;

[0023] A binary module for binarizing each laser fundus image to obtain a mask corresponding to each laser fundus image;

[0024] A first training module for inputting each laser fundus image into the first network model in batches and training with the mask of each laser fundus image as the true label to obtain a trained laser spot segmentation model;

[0025] An area acquisition module for obtaining the area of the laser spot surrounded by the laser spot boundary in each laser fundus image;

[0026] A second training module for inputting each laser fundus image into the second network model in batches and training with the area of the laser spot in each laser fundus image as the true label to obtain a trained area prediction model;

[0027] The first diagnosis module is used to obtain the image to be diagnosed, input it into the laser spot segmentation model, and obtain the laser spot segmentation region;

[0028] The second diagnosis module is used to input the laser spot segmentation region into the area prediction model to obtain the predicted area of the laser spot;

[0029] The exponential module is used to calculate the circular area of the fundus circular region in the image to be diagnosed, divide the predicted area of the laser spot by the circular area, and obtain the laser area index.

[0030] In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it executes the steps of a ROP laser spot quantization method according to any one of the above embodiments.

[0031] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of a ROP laser spot quantization method according to any one of the above embodiments.

[0032] In summary, compared with the prior art, the beneficial effects brought by the technical solutions provided by the embodiments of the present application at least include:

[0033] A ROP laser spot quantization method provided by an embodiment of the present application obtains a mask of the laser fundus image through binarization, and at the same time obtains the area of the laser spot surrounded by the laser spot boundary outlined in the laser fundus image. The mask is used to distinguish the laser spots falling in the circular region of the fundus image, and the mask is used to train the laser spot segmentation model, and the laser spot area is used to train the area prediction model. Therefore, the laser spot segmentation region of the image to be diagnosed and the predicted area of the laser spot corresponding to the laser segmentation region can be obtained through the trained laser spot segmentation model and area prediction model, and the laser area index is obtained by comparing with the circular area of the circular region. The present application realizes the quantization of the laser spot area after laser photocoagulation by using the laser area index, and provides an objective quantization method for predicting the fundus prognosis and refractive state of children after ROP laser photocoagulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flowchart of a ROP laser spot quantization method provided by an exemplary embodiment of the present application.

[0035] Figure 2 It is a schematic structural diagram of a Unet architecture model provided by an exemplary embodiment of the present application.

[0036] Figure 3 It is a schematic structural diagram of a Resnet50 model provided by an exemplary embodiment of the present application.

[0037] Figure 4 The structural diagram of a ROP laser spot quantification device provided for an exemplary embodiment of the present application. Detailed implementation manners

[0038] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0039] All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0040] Please refer to Figure 1 , the embodiments of the present application provide a ROP laser spot quantification method, including:

[0041] Step S1, obtain a plurality of fundus laser images, and outline the boundaries of the laser spots in each fundus laser image with solid lines.

[0042] Among them, the fundus laser images are the fundus images at different stages after laser treatment of ROP children. The fundus laser images are required not to be off-center fundus laser scan images or images that multiple doctors cannot distinguish the laser spots, so as to ensure the accuracy of training.

[0043] Step S2, binarize each fundus laser image to obtain a mask corresponding to each fundus laser image.

[0044] Among them, the mask corresponding to the image is a binary image used to identify different regions in the image. Each pixel point in the mask identifies whether the corresponding position belongs to a certain specific category. The mask has the same size as the original fundus laser image, and the value of each pixel is usually a binary value, indicating the classification of the pixel, that is, two categories: laser spots and non-laser spots. Therefore, the trained laser spot segmentation model can realize the recognition and segmentation of the laser spots falling in the circular area of the fundus image.

[0045] Step S3, input each fundus laser image into the first network model in batches, and use the mask of each fundus laser image as the true label for training to obtain a trained laser spot segmentation model.

[0046] Step S4, obtain the area of the laser spot surrounded by the boundary of the laser spot in each fundus laser image.

[0047] Step S5, input each fundus laser image into the second network model in batches, and use the area of the laser spot in each fundus laser image as the true label for training to obtain a trained area prediction model.

[0048] Step S6: Obtain the image to be diagnosed and input it into the laser spot segmentation model to obtain the laser spot segmentation region.

[0049] Step S7: Input the laser spot segmentation region into the area prediction model to obtain the predicted area of the laser spot.

[0050] The laser spot segmentation region is the part of the laser spot in the image to be diagnosed that falls within the circular region of the fundus, which is recognized and segmented by the laser spot segmentation model. The corresponding predicted area of the laser spot is the area of the laser spot in the image to be diagnosed that falls within the circular region of the fundus.

[0051] Step S7: Calculate the circular area of the circular region of the fundus in the image to be diagnosed, and divide the predicted area of the laser spot by the circular area to obtain the laser area index. In the third edition of the International Classification of Retinopathy of Prematurity (ICROP), there is a concept of "posterior pole zone II". Specifically, the circular region of the fundus refers to the area of the circle drawn with a radius equal to the sum of twice the distance from the center of the optic disc to the fovea centralis and two optic disc diameters, centered on the optic disc, in the ROP zonal definition of the fundus image. Lesions in the posterior pole zone II are more threatening than those in the more peripheral zone II, and are of great value in guiding clinical selection of treatment timing and treatment methods and judging prognosis. Therefore, the circular region of zone I + posterior pole zone II is selected as the cropping standard.

[0052] Specifically, the OpenCV library can be used to read the entire circular region of the fundus, then numpy can be used to calculate the circular area of the entire circular region of the fundus, and then the predicted laser area is divided by the circular area to obtain the laser area index.

[0053] The calculation in Step S7 is expressed by the formula: Laser Area Index LAI = (A laser / A circle ) × 100%; where A laser is the predicted area of the laser spot, and A circle is the circular area of the circular region of the fundus.

[0054] A method for quantifying ROP laser spots provided in the above embodiments obtains a mask of the fundus laser image through binarization, and at the same time obtains the area of the laser spot surrounded by the boundary of the laser spot outlined in the fundus laser image. The mask is used to distinguish the laser spots falling in the circular area of the fundus image, and the mask is used to train a laser spot segmentation model, and the laser spot area is used to train an area prediction model. Therefore, the laser spot segmentation area of the image to be diagnosed and the predicted laser spot area corresponding to the laser segmentation area can be obtained through the trained laser spot segmentation model and area prediction model, and the laser area index is obtained by comparing with the circular area of the circular area. The present application realizes the quantification of the laser spot area after laser photocoagulation by using the laser area index, and provides an objective quantification method for predicting the fundus prognosis and refractive state of children after ROP laser photocoagulation.

[0055] In some embodiments, the method further includes:

[0056] Calculating the optic disc area of the image to be diagnosed; comparing the predicted laser spot area with the optic disc area to obtain the optic disc area percentage.

[0057] After quantifying the laser spots in the above embodiments, further calculating the optic disc area percentage of the optic disc area it occupies helps doctors evaluate the long-term refractive changes of children after laser photocoagulation, and quantify the long-term impact of photocoagulation treatment on refraction, so as to provide clinical doctors with a multi-angle evaluation of the condition and formulate a more accurate refractive treatment plan.

[0058] In some embodiments, the method may further include: before training the first network model, adjusting each fundus laser image and its corresponding mask to a preset size, and normalizing each mask of the preset size.

[0059] Among them, if the image is too large, the computer storage space is not enough, and if the image is too small, too much pixel information will be lost. Therefore, it is more appropriate to set the preset size to 800×800. In the specific implementation process, the preset size also needs to adapt to the input requirements of the first network model.

[0060] In some embodiments, the method may further include: before training the second network model, adjusting each fundus laser image to a preset size and performing normalization processing so that the pixel values of each fundus laser image are within a preset range.

[0061] Among them, the preset range can be 0-1 or specific mean and standard deviation to reduce the computational complexity in model training and improve performance; in the case of limited data volume, data augmentation techniques (such as rotation, scaling, flipping, etc.) can also be applied to improve the adaptability of the model to images under different conditions and enhance the generalization ability of the model.

[0062] In some embodiments, the first network model may be a Unet architecture model. Please refer toFigure 2 This architecture is specifically designed to process complex features in fundus images, thereby achieving precise segmentation of ROP laser spots.

[0063] The main advantage of the Unet architecture lies in its symmetric structure, which includes a contracting path (for capturing background information) and an expanding path (for precise localization). By fusing the high-resolution features of the contracting path in the expanding path, Unet can accurately segment the key features in the image while maintaining background information, which is particularly important for the segmentation of ROP laser spots.

[0064] It should be noted that in this application, a custom DICELossMultiClass loss function is used in the Unet architecture model. This special loss function can calculate the Dice coefficient between the model output and the ground truth mask, and the Dice coefficient is an index commonly used in medical image segmentation. Considering that the mask has a single-channel structure, DICELossMultiClass is designed to only consider the first channel of the output, making it more suitable for the segmentation task of laser spots.

[0065] In some embodiments, the second network model can be a Resnet50 model.

[0066] Please refer to Figure 3 , the deep learning model of Resnet50 is specifically trained to accurately estimate the area of the laser spot region from fundus images. The Resnet50 model is a deep residual network, which solves the problem of vanishing gradients in the training of deep networks by introducing residual connections. Therefore, this application uses the Resnet50 model to extract the features of the ROP laser spot region in fundus images and predicts the area of the laser spot through a linear regression layer.

[0067] In addition, this application uses a standard optimizer configuration for the Resnet50 model for area prediction, such as SGD (Stochastic Gradient Descent) with a high momentum parameter (0.99), which helps the model converge stably during training.

[0068] During the training process, cross-validation is used to adjust and optimize the model parameters to ensure the generalization ability of the model on different datasets. The main evaluation metrics for the segmentation task are the Dice coefficient and the intersection over union (IoU). The Dice coefficient is a set similarity metric function, usually used to calculate the similarity between two samples. IoU is defined as the ratio between the size of the intersection of two sets and the size of the union. After multiple training optimizations, it is determined that the test results of the segmentation model on the image set are relatively good when the average Dice is 0.8633 and the average IoU is 0.7716. The evaluation metric for the Resnet50 linear regression model is R-squared. R-squared is a metric used to evaluate the goodness of fit of a model, which represents the degree of dispersion between the fitted line of the model and the true values. The value of R-squared ranges from 0 to 1, and the closer the value is to 1, the better the model fits, and the closer the predicted values are to the true values. After multiple training optimizations, the prediction of the Resnet50 linear regression model is relatively accurate when the R-squared is 0.993. In the evaluation stage of the Unet architecture model, in addition to using the Dice coefficient and the intersection over union (IoU) as the main evaluation metrics for the segmentation task, this application also particularly focuses on the performance of the custom DICELossMultiClass loss function in evaluating the model performance. In addition, for the Resnet50 model, this application evaluates its accuracy and reliability in area prediction by monitoring the error between the predicted area value and the true area value and analyzing the ability of the model to process laser spots of different sizes.

[0069] Please refer to Figure 4 , Another embodiment of this application provides a ROP laser spot quantification device, including:

[0070] An outlining module 101, configured to obtain a plurality of fundus laser images and outline the boundaries of the laser spots in each fundus laser image with solid lines.

[0071] A binarization module 102, configured to binarize each fundus laser image to obtain a mask corresponding to each fundus laser image.

[0072] A first training module 103, configured to input each fundus laser image into a first network model in batches and train with the mask of each fundus laser image as the true label to obtain a trained laser spot segmentation model.

[0073] An area acquisition module 104, configured to obtain the area of the laser spot surrounded by the boundary of the laser spot in each fundus laser image.

[0074] A second training module 105, configured to input each fundus laser image into a second network model in batches and train with the area of the laser spot in each fundus laser image as the true label to obtain a trained area prediction model.

[0075] The first diagnosis module 106 is configured to obtain a to-be-diagnosed image, input it into a laser spot segmentation model, and obtain a laser spot segmentation region.

[0076] The second diagnosis module 107 is configured to input the laser spot segmentation region into an area prediction model, and obtain a predicted laser spot area.

[0077] The index module 108 is configured to calculate the circular area of the fundus circular region in the to-be-diagnosed image, divide the predicted laser spot area by the circular area, and obtain a laser area index.

[0078] Furthermore, the device further includes a comparison module, configured to calculate the optic disc area of the to-be-diagnosed image; compare the predicted laser spot area with the optic disc area, and obtain an optic disc area percentage.

[0079] Furthermore, the device further includes a first preprocessing module, configured to, before training the first network model, adjust each laser fundus image and its corresponding mask to a preset size, and perform normalization processing on each mask of the preset size.

[0080] Furthermore, the device further includes a second preprocessing module, configured to, before training the second network model, adjust each laser fundus image to a preset size, and perform normalization processing so that the pixel values of each laser fundus image are within a preset range.

[0081] The specific limitations provided in this embodiment for a ROP laser spot quantification device can be referred to the embodiment of a ROP laser spot quantification method in the above text, and will not be elaborated here. Each module in the above ROP laser spot quantification device can be implemented in whole or in part by software, hardware, and their combination.

[0082] The above modules can be embedded in the processor of the computer device in the form of hardware or be independent of the processor, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0083] This application embodiment provides a computer device, which may include a processor, a memory, a network interface, and a database connected through a system bus. Wherein, the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program stored in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the processor is caused to execute the steps of a ROP laser spot quantification method according to any one of the above embodiments.

[0084] For the working process, working details and technical effects of the computer device provided in this embodiment, reference can be made to the embodiment of a ROP laser spot quantization method in the foregoing text, which will not be elaborated herein.

[0085] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of a ROP laser spot quantization method as in any of the foregoing embodiments. Among them, the computer-readable storage medium refers to a carrier for storing data, and may include, but is not limited to, floppy disks, optical discs, hard disks, flash memories, USB flash drives, and / or memory sticks, etc. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. For the working process, working details and technical effects of the computer-readable storage medium provided in this embodiment, reference can be made to the embodiment of a ROP laser spot quantization method in the foregoing text, which will not be elaborated herein.

[0086] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it may include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application may include non-volatile and / or volatile memories. Non-volatile memories may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM).

[0087] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0088] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A ROP laser spot quantification method, characterized in that: include: Acquire multiple laser fundus images, and outline the laser spot boundary in each of the laser fundus images with a solid line; Binarizing each of the laser fundus images to obtain a mask corresponding to each of the laser fundus images; Inputting the laser fundus images into a first network model in batches, and training the model using the masks of the laser fundus images as true labels to obtain a trained laser spot segmentation model; Acquire the laser spot area surrounded by the laser spot boundary in each laser fundus image; Inputting the laser fundus images into a second network model in batches, and training the model using the laser spot area in the laser fundus images as true labels to obtain a trained area prediction model; Acquire the image to be diagnosed and input it into the laser spot segmentation model to obtain the laser spot segmentation area; Inputting the laser spot segmentation area into the area prediction model to obtain the laser spot prediction area; Calculate the circular area of ​​the circular area of ​​the fundus in the image to be diagnosed, and divide the predicted area of ​​the laser spot by the circular area to obtain a laser area index; Calculating the optic disc area of ​​the image to be diagnosed; Comparing the predicted area of ​​the laser spot with the area of ​​the optic disc to obtain a percentage of the optic disc area; The first network model is the Unet architecture model; Use loss function in Unet architecture model; The loss function calculates the Dice coefficient between the Unet architecture model output and the true mask; The loss function only considers the first channel of the output.

2. The ROP laser spot quantification method according to claim 1, characterized in that: Also includes: Before training the first network model, each of the laser fundus images and the corresponding masks are adjusted to a preset size, and each of the masks of the preset size is normalized.

3. The ROP laser spot quantification method according to claim 2, characterized in that: Also includes: Before training the second network model, each of the laser fundus images is adjusted to the preset size and normalized so that the pixel values ​​of each of the laser fundus images are within a preset range.

4. The ROP laser spot quantification method according to claim 3, characterized in that: The preset size is 800×800, and the preset range is 0-1.

5. The ROP laser spot quantification method according to claim 1, characterized in that: The first network model is a Unet architecture model.

6. The ROP laser spot quantification method according to claim 1, characterized in that: The second network model is a Resnet50 model.

7. A ROP laser spot quantification device, characterized in that: include: A delineation module, used to acquire a plurality of laser fundus images and to delineate the laser spot boundary in each of the laser fundus images with a solid line; A binary module, used for binarizing each of the laser fundus images to obtain a mask corresponding to each of the laser fundus images; A first training module, used for inputting the laser fundus images into a first network model in batches, and performing training using the masks of the laser fundus images as true labels to obtain a trained laser spot segmentation model; An area acquisition module, used for acquiring the laser spot area surrounded by the laser spot boundary in each laser fundus image; A second training module is used to input the laser fundus images into a second network model in batches, and use the laser spot area in each laser fundus image as a true label for training to obtain a trained area prediction model; A first diagnosis module is used to obtain an image to be diagnosed and input it into the laser spot segmentation model to obtain a laser spot segmentation area; A second diagnosis module is used to input the laser spot segmentation area into the area prediction model to obtain the laser spot prediction area; An index module, used for calculating the area of ​​a circular fundus region centered on the optic disc in the image to be diagnosed, and dividing the predicted area of ​​the laser spot by the area of ​​the circular region to obtain a laser area index; A comparison module, used for calculating the optic disc area of ​​the image to be diagnosed, and comparing the predicted area of ​​the laser spot with the optic disc area to obtain the optic disc area percentage; The first network model is the Unet architecture model; Use loss function in Unet architecture model; The loss function calculates the Dice coefficient between the Unet architecture model output and the true mask; The loss function only considers the first channel of the output.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the ROP laser spot quantification method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the ROP laser spot quantification method according to any one of claims 1 to 6 are implemented.

Images