Eye lesion recognition method, system, device and method for fundus image

Abstract

The invention provides an eye lesion recognition method, system, device and method for eye fundus images, and the method comprises the steps: obtaining a plurality of normal eye fundus images, and constructing a standard pixel set according to the normal eye fundus images; obtaining a to-be-detected eye fundus image, and performing pixel filtering on the to-be-detected eye fundus image by using the standard pixel set to obtain a filtered image; using the trained YOLOv8 model to identify the filtered image to obtain an identified image; and classifying the identified image by using the trained mapping classification model, and judging the eye lesion type of the fundus image to be detected according to a classification result. According to the invention, the problems in the prior art that a color image is converted into a grey-scale image during image recognition, a part of image features are lost, the subsequent recognition accuracy of a lesion area is influenced, and the judgment accuracy of the lesion type is not high due to the fact that the judgment of the eye lesion type is mostly manual judgment are solved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method, system, device and approach for identifying ocular lesions from fundus images. Background Technology

[0002] Medical image processing and analysis has always been a key and hot research topic in the field of image processing and analysis. With the powerful tools of graphics and image technology, the quality and display methods of medical images have been greatly improved, significantly enhancing diagnostic and treatment capabilities. Image recognition technology has been introduced into ophthalmology for many years. By using image recognition technology to calculate and analyze fundus images, quantitative measurements can be performed on important fundus tissues such as the optic disc, retinal vessels, microaneurysms, and the fovea of ​​the macula. Clear differentiation can be made between normal and abnormal conditions, enabling the early and accurate detection of various ocular lesions and systemic diseases, such as diabetes, hypertension, and arteriosclerosis.

[0003] However, when using image recognition, since fundus images are color images, in order to accurately identify lesion areas in the fundus images, color images are usually converted into grayscale images to avoid the influence of the color of the normal areas around the lesion area on the lesion area. However, converting to grayscale images often results in the loss of some image features, affecting the accuracy of subsequent identification of lesion areas. At the same time, after the lesion area is identified, the judgment of the type of eye lesion is mostly left to doctors to make a human judgment. This is highly dependent on the doctor's professional experience, which not only increases the doctor's workload, but also easily leads to a low accuracy rate in judging the type of lesion. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method, system, device, and approach for identifying ocular lesions from fundus images. It solves the problems of existing image recognition technologies, which convert color images to grayscale images during image recognition, resulting in the loss of some image features and affecting the accuracy of subsequent identification of lesion areas. It also addresses the issue that the determination of ocular lesion types is mostly based on human judgment, leading to low accuracy in lesion type identification.

[0005] According to an embodiment of the present invention, a method for identifying ocular lesions from fundus images includes:

[0006] Acquire multiple normal fundus images and construct a standard pixel set based on the normal fundus images;

[0007] Acquire the fundus image to be tested, and use a standard pixel set to perform pixel filtering on the fundus image to obtain the filtered image;

[0008] The filtered image is identified using a trained YOLOv8 model to obtain the identified image;

[0009] The trained mapping classification model is used to classify the images to be identified, and the type of ocular lesion in the fundus image to be tested is determined based on the classification results.

[0010] Preferably, the method for constructing the standard pixel set includes:

[0011] A coordinate system of the same scale is constructed with the pixel at the center of the optic disc in each normal fundus image as the origin, and each pixel in the image is assigned a corresponding coordinate to generate a corresponding normal pixel set. The normal pixel set includes multiple pixels, and each pixel includes coordinates and RGB values.

[0012] Calculate the average RGB value of pixels with the same coordinates in all normal pixel sets, and then replace the RGB values ​​of the corresponding coordinates in all normal pixel sets with the average RGB value;

[0013] Calculate the union of all normal pixel sets to obtain the standard pixel set.

[0014] Preferably, the method for obtaining a filtered image by pixel filtering of the fundus image to be tested using a standard pixel set includes:

[0015] A coordinate axis is constructed with the pixel at the center of the optic disc in the fundus image to be tested as the origin, and each pixel in the image is assigned a corresponding coordinate to generate a set of pixels to be tested. The set of pixels to be tested includes multiple pixels, and each pixel includes coordinates and RGB values.

[0016] The filtered pixel set is obtained by subtracting the RGB values ​​of the pixels in the pixel set to be tested from the RGB values ​​of the pixels with the same coordinates as the standard pixel set.

[0017] The filtered image is regenerated based on the filtered pixel set.

[0018] Preferably, the YOLOv8 model includes a Backbone section, a Neck section, and a Head section, wherein the Head section includes multiple decoupled detection heads, and each decoupled detection head is connected to an attention module.

[0019] Preferably, the mapping classification model includes a discriminator and a mapping feature extractor;

[0020] Methods for classifying the recognition results using a trained mapping classification model to obtain the corresponding types of eye lesions include:

[0021] Obtain labeled target images corresponding to each type of ocular lesion, and then use a mapping feature extractor to extract discriminative features of the recognition image and all target images;

[0022] The discriminative features of the identified image and all target images are mapped to the same common feature space using the same feature mapping rules;

[0023] Traverse the discriminant features of all target images, use a discriminator to determine whether the discriminant features of the recognition image and the target image in the common feature space are aligned, and take the ocular lesion type corresponding to the target image with feature alignment as the ocular lesion type of the fundus image to be tested.

[0024] Preferably, the mapping feature extractor consists of the backbone network of a trained YOLOv8 model and a detection module. When extracting discriminative features, the recognition image is cropped, and only the image in the detection box is retained.

[0025] Preferably, before use, the discriminator is adversarially trained using multiple target images labeled with different types of eye lesions, so that it can distinguish the types of eye lesions in the target images.

[0026] On the other hand, embodiments of the present invention also provide an ocular lesion recognition system based on fundus images. This system uses the above-described method for recognizing ocular lesions based on fundus images, including:

[0027] Image acquisition module, which is used to acquire normal fundus images and fundus images to be tested;

[0028] A model building module, which is used to build and train YOLOv8 models and mapping classification models;

[0029] A pixel filtering module is used to construct a standard pixel set based on a normal fundus image and to use the standard pixel set to perform pixel filtering on the fundus image to be tested to obtain a filtered image.

[0030] The recognition module is used to recognize the filtered image using a trained YOLOv8 model to obtain a recognized image;

[0031] The classification module is used to classify the recognition image and determine the type of ocular lesion in the fundus image to be tested based on the classification result.

[0032] On the other hand, embodiments of the present invention also provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs an ocular lesion identification method based on fundus images as described above.

[0033] On the other hand, embodiments of the present invention also provide a computer storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the above-described method for identifying ocular lesions from fundus images.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] This invention uses a standard pixel set constructed from normal fundus images to filter out pixels in normal areas of the fundus image to be tested, thereby eliminating the interference of normal area colors on abnormal areas and improving detection accuracy. Then, the YOLOv8 model is used to identify abnormal areas in the filtered image to obtain a recognition image containing abnormal area detection boxes. Finally, a mapping classification model is used to automatically classify the recognition image, replacing human judgment and improving the accuracy of lesion type identification. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the process of identifying eye lesions according to an embodiment of the present invention.

[0037] Figure 2 This is a diagram of the improved YOLOv8 network architecture according to an embodiment of the present invention.

[0038] Figure 3 This is a schematic diagram of the decoupling detection head according to an embodiment of the present invention.

[0039] Figure 4 This is a schematic diagram of feature alignment according to an embodiment of the present invention. Detailed Implementation

[0040] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0041] like Figure 1 As shown in the figure, an embodiment of the present invention proposes a method for identifying ocular lesions from fundus images, including:

[0042] Acquire multiple normal fundus images and construct a standard pixel set based on the normal fundus images;

[0043] Because fundus images are taken from different angles and positions to capture images of all areas of the fundus as comprehensively as possible, and because the direction of blood vessels and nerves varies from person to person, and because fundus images generally need to include the optic disc, in this invention, the optic disc is used as a fixed reference point as an anchor point.

[0044] To avoid the influence of normal areas in fundus images on lesion areas, regions in the fundus image to be tested can be excluded. First, a universally applicable standard normal fundus image should be constructed. However, because the shooting environment (lighting, shooting angle, etc.) differs for each normal fundus image, the pixel values ​​of each pixel in each normal fundus image will also differ. Therefore, this invention uses the average RGB values ​​of each pixel in multiple normal fundus images to construct a standard pixel set.

[0045] (1) Construct a coordinate system of the same scale with the pixel at the center of the optic disc in each normal fundus image as the origin, and assign corresponding coordinates to each pixel in the image to generate the corresponding normal pixel set. The normal pixel set includes multiple pixels, and each pixel includes coordinates and RGB values.

[0046] (2) Calculate the average RGB value of pixels with the same coordinates in all normal pixel sets, and then replace the RGB value of the corresponding coordinates in all normal pixel sets with the average RGB value.

[0047] (3) Calculate the union of all normal pixel sets to obtain the standard pixel set.

[0048] For any normal fundus image, a two-dimensional coordinate system is constructed with the pixel at the center of the optic disc as the origin. The coordinates of each pixel in this system are then determined, and the corresponding set of normal pixels P is generated. u ={S i,j ,R i,j}, where S is the coordinate set and R is the corresponding RGB value. i,j For the pixel with coordinates (i, j), R i,j Let (i, j) be the RGB value of the pixel. Then, calculate the average RGB value of the pixels with the same coordinates in each normal pixel set. Replace the RGB value of the corresponding coordinates in all normal pixel sets with the average RGB value. At this point, the RGB values ​​of the two pixels with the same coordinates in any two normal pixel sets are the same. Then calculate the union of all normal pixel sets to obtain a standard pixel set with a fairly large coverage.

[0049] Acquire the fundus image to be tested, and use a standard pixel set to perform pixel filtering on the fundus image to obtain the filtered image;

[0050] After acquiring the fundus image to be tested, similar to the standard pixel set, a coordinate axis is constructed with the pixel at the center of the optic disc in the fundus image as the origin, and each pixel in the image is assigned a corresponding coordinate, generating the pixel set to be tested Q = {S}. k ,R k Since the normal pixel set consists of multiple normal fundus images, the number of pixels in the normal pixel set is much greater than that in the pixel set to be tested, and the coordinate range is extremely large. Therefore, each pixel in the pixel set to be tested can find a pixel with the same coordinates in the normal pixel set. Thus, by subtracting the RGB values ​​of the pixels with the same coordinates as the pixels in the normal pixel set from the RGB values ​​of the pixels in the pixel set to be tested, we can obtain a filtered pixel set that removes the interference of normal region pixels and leaves only abnormal region pixels. Then, based on the pixels in the filtered pixel set, a filtered image that only contains abnormal regions is regenerated, and the missing pixels in the filtered image are filled with colorless pixels.

[0051] The filtered image is identified using a trained YOLOv8 model to obtain the identified image;

[0052] This invention employs the YOLOv8 model for image recognition and detection. The original YOLOv8 model includes a Backbone, a Neck, and a Head part. The Head part uses a coupled detection head to recognize and detect images, obtaining corresponding recognition images with bounding boxes. However, to better detect lesion areas, this invention makes certain improvements to the original YOLOv8 model. The improved network architecture is as follows: Figure 2 As shown, in the Head section, the original coupling detection head is replaced with a decoupling detection head.

[0053] like Figure 3 As shown, the decoupled detection head uses a 1×1 convolution to reduce the feature channels to 256; then, two 3×3 convolutions are applied to the classification and regression tasks respectively. Specifically, the classification task uses a 1×1 convolution for classification, while the regression task uses a 1×1 convolution for localization and confidence operations. Through these operations, separate detection of classification, localization, and confidence is achieved, improving accuracy.

[0054] In addition, a CA attention module is added before each decoupled detection head to focus on key features and increase detection accuracy.

[0055] Furthermore, YOLOv8 uses CIoU as the loss function. However, since the CIoU loss function only considers the overlap and size difference between the predicted bounding box and the ground truth bounding box, and does not consider the shape difference of the predicted bounding box, it will be affected by scale changes when processing targets of different scales, especially small targets, leading to a decrease in detection accuracy. Therefore, this invention uses the MPDIoU loss function instead of CIoU for bounding box regression when training the YOLOv8 model. The core feature of the MPDIoU loss function replacing the CIoU loss function is that it replaces CIoU loss function, and its shortcomings have also been optimized. The calculation formula is as follows:

[0056]

[0057] c 2 =w 2 +h 2

[0058]

[0059] In the formula, w and h represent the width and height of the input image; (x1, y1) and (x2, y2) are the coordinates of the top-left and bottom-right corners of the prediction box, respectively; similarly, and These are the coordinates of the top-left and bottom-right corners of the ground truth bounding box. When the center points of the predicted bounding box and the actual bounding box do not coincide, the predicted bounding box is effectively guided closer to the position of the ground truth bounding box by minimizing the distance between the top-left and bottom-right points of the predicted bounding box and the ground truth bounding box.

[0060] The trained mapping classification model is used to classify the images to be identified, and the type of ocular lesion in the fundus image to be tested is determined based on the classification results.

[0061] Because some image formats (e.g., JPG) perform lossy compression, image features may be missing. If we directly use SVM classifiers or similar methods to extract image features and calculate the relationships between them to classify images with missing features, it can sometimes lead to inaccurate classification. Therefore, this invention uses feature comparison for classification. It classifies images by comparing the feature similarity between the image to be tested and the image with the defined category on the same dimension, without directly calculating the relationships between features.

[0062] The feature mapping model of this invention includes a discriminator and a mapping feature extractor. The discriminator is built based on a GAN network. Since the mapping feature extractor is not used to generate simulated data, but only to extract features from the data, the pre-trained YOLOv8 model described above is used to construct the mapping feature extractor.

[0063] The mapping feature extractor consists of the backbone network of the YOLOv8 model and a detection module. After the discriminator and the mapping feature extractor are built, the discriminator is trained adversarially. In the adversarial training, the recognition images for determining the type of eye lesion (each lesion type is trained separately) are divided into two parts. One part is used as the source data (the image to be tested), and the other part is used as the target data (the image for determining the type of eye lesion). The mapping feature extractor extracts high-level features from the source data (such as simulation data) and the target data (such as actual detection data) and maps the features to the same common feature space. Then, the discriminator is used to determine whether the features between the source data and the target data are aligned.

[0064] The discriminative features extracted by the mapping feature extractor include:

[0065]

[0066] When extracting discriminative features, the recognition image is cropped, retaining only the image within the detection bounding box. Then, the extracted discriminative features are used for feature mapping. The essence of feature mapping is to map the feature distribution P of the source domain (recognition image). s (X) and the feature distribution P of the target domain (annotated target image corresponding to the type of ocular lesion). t(X) is mapped to a common feature space, making the two similar. At this point, it can be determined that the source data and target data have been feature aligned.

[0067] The feature data generated by the mapping, after feature visualization, is as follows: Figure 4 As shown, if the features of the target data and the source data are not similar after mapping, the generator's model parameters are adjusted and feature extraction is performed again until the discriminator determines that the discriminant features of all source data and target data have been aligned.

[0068] The discriminator's loss:

[0069] Generator loss:

[0070] The ultimate goal is to minimize the joint loss: L total =L det +λ(L G +L D )

[0071] Where L det The loss for YOLO object detection includes classification and regression.

[0072] like Figure 4 As shown, feature alignment can be achieved by mapping the target data features and the source data features. The aligned target data and the aligned source data have approximately the same distribution. At this point, it can be determined that the test image has the same type of ocular lesion as the target image.

[0073] Then, the discriminant features of all target images are traversed. The discriminator is used to determine whether the discriminant features of the recognition image and the target image in the common feature space are aligned. The type of eye lesion corresponding to the target image with aligned features is taken as the type of eye lesion in the fundus image to be tested. In this way, the features of the image within the abnormal region detection box of the recognition model are extracted and mapped. The mapped features are compared with other features that determine the type of eye lesion to be tested, thereby determining the type of eye lesion corresponding to the fundus image to be tested. This eliminates the problem of image missing during feature extraction of images of different formats.

[0074] On the other hand, embodiments of the present invention also provide an ocular lesion recognition system based on fundus images. This system uses the above-described method for recognizing ocular lesions based on fundus images, including:

[0075] The image acquisition module is used to acquire normal fundus images and fundus images of the eye under test.

[0076] The model building module is used to build and train YOLOv8 models and mapping classification models.

[0077] The pixel filtering module is used to construct a standard pixel set based on a normal fundus image and to use the standard pixel set to filter the fundus image under test to obtain a filtered image.

[0078] The recognition module is used to recognize the filtered image using the trained YOLOv8 model to obtain the recognized image.

[0079] The classification module is used to classify the recognition images and determine the type of ocular lesion in the fundus image to be tested based on the classification results.

[0080] On the other hand, embodiments of the present invention also provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs an ocular lesion identification method based on fundus images as described above.

[0081] On the other hand, embodiments of the present invention also provide a computer storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the above-described method for identifying ocular lesions from fundus images.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for identifying ocular lesions from fundus images, characterized in that: include: Acquire multiple normal fundus images and construct a standard pixel set based on the normal fundus images; Acquire the fundus image to be tested, and use a standard pixel set to perform pixel filtering on the fundus image to obtain the filtered image; The filtered image is identified using a trained YOLOv8 model to obtain the identified image; The trained mapping classification model is used to classify the images and the type of ocular lesion in the fundus image is determined based on the classification results.

2. The method for identifying ocular lesions from fundus images as described in claim 1, characterized in that: Methods for constructing a standard pixel set include: A coordinate system of the same scale is constructed with the pixel at the center of the optic disc in each normal fundus image as the origin, and each pixel in the image is assigned a corresponding coordinate to generate a corresponding normal pixel set. The normal pixel set includes multiple pixels, and each pixel includes coordinates and RGB values. Calculate the average RGB value of pixels with the same coordinates in all normal pixel sets, and then replace the RGB values ​​of the corresponding coordinates in all normal pixel sets with the average RGB value; Calculate the union of all normal pixel sets to obtain the standard pixel set.

3. The method for identifying ocular lesions from fundus images as described in claim 1, characterized in that: Methods for obtaining filtered images by pixel filtering of the fundus image under test using a standard pixel set include: A coordinate axis is constructed with the pixel at the center of the optic disc in the fundus image to be tested as the origin, and each pixel in the image is assigned a corresponding coordinate to generate a set of pixels to be tested. The set of pixels to be tested includes multiple pixels, and each pixel includes coordinates and RGB values. The filtered pixel set is obtained by subtracting the RGB values ​​of the pixels in the pixel set to be tested from the RGB values ​​of the pixels with the same coordinates as the standard pixel set. The filtered image is regenerated based on the filtered pixel set.

4. The method for identifying ocular lesions from fundus images as described in claim 1, characterized in that: The YOLOv8 model includes a Backbone, a Neck, and a Head. The Head includes multiple decoupled detection heads, each connected to an attention module.

5. The method for identifying ocular lesions from fundus images as described in claim 1, characterized in that: The mapping classification model includes a discriminator and a mapping feature extractor; Methods for classifying the recognition results using a trained mapping classification model to obtain the corresponding types of eye lesions include: Obtain labeled target images corresponding to each type of ocular lesion, and then use a mapping feature extractor to extract discriminative features of the recognition image and all target images; The discriminative features of the identified image and all target images are mapped to the same common feature space using the same feature mapping rules; Traverse the discriminant features of all target images, use a discriminator to determine whether the discriminant features of the recognition image and the target image in the common feature space are aligned, and take the ocular lesion type corresponding to the target image with feature alignment as the ocular lesion type of the fundus image to be tested.

6. The method for identifying ocular lesions from fundus images as described in claim 5, characterized in that: The mapping feature extractor consists of the backbone network of a trained YOLOv8 model and a detection module. When extracting discriminative features, the recognition image is cropped, and only the image in the detection box is retained.

7. The method for identifying ocular lesions from fundus images as described in claim 5, characterized in that: Before use, the discriminator is adversarially trained using multiple target images labeled with different types of eye lesions, enabling it to distinguish the types of eye lesions in the target images.

8. A system for recognizing ocular lesions from fundus images, characterized in that: The system uses a method for identifying ocular lesions from fundus images as described in any one of claims 1-7, comprising: Image acquisition module, which is used to acquire normal fundus images and fundus images to be tested; A model building module, which is used to build and train YOLOv8 models and mapping classification models; A pixel filtering module is used to construct a standard pixel set based on a normal fundus image and to use the standard pixel set to perform pixel filtering on the fundus image to be tested to obtain a filtered image. The recognition module is used to recognize the filtered image using a trained YOLOv8 model to obtain a recognized image; The classification module is used to classify the recognition image and determine the type of ocular lesion in the fundus image to be tested based on the classification result.

9. A computer device, characterized in that: The device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform a method for identifying ocular lesions from fundus images as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The device contains a computer program that, when executed by a processor, causes the processor to perform a method for identifying ocular lesions from fundus images as described in any one of claims 1 to 7.

Images