A fundus image dataset preprocessing method and a fundus image training model

Abstract

The application provides a preprocessing method of an eye fundus image dataset and an eye fundus image training model, and comprises the following steps: S1, adaptively enhancing the eye fundus image to obtain an initial dataset of the eye fundus image; S2, selectively expanding the initial dataset; and S3, using a generative adversarial network to further expand the initial dataset. Under the condition that only a small amount of eye fundus images with poor quality exist, the small dataset is adaptively enhanced and selectively expanded, so that the eye fundus image screening result with high precision can be obtained through the training of a neural network recognition model.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to an adaptive preprocessing method and a fundus image training model for small fundus image datasets. Background Technology

[0002] In recent years, deep learning algorithms based on convolutional neural networks have been continuously developing in the field of computer vision. Because they alleviate the local minima of traditional training algorithms, they have attracted widespread attention in the machine learning field and are being applied in an increasing number of areas. However, in practical applications, the quantity and quality of the dataset are often a significant factor affecting the algorithm's performance; only a rich and effective dataset can serve as the foundation for researching subsequent algorithms.

[0003] In recent years, medical facilities have been continuously updated and improved with the development of technology. Remote treatment and automated diagnosis have also been widely used in actual diagnosis and treatment. Fundus image screening based on deep learning has greatly saved manpower and material resources and achieved efficient and convenient rapid diagnosis of diseases. However, the number of fundus images is small and difficult to obtain. Moreover, due to various reasons, most of them are of poor quality and far from meeting the good background conditions for computer algorithm simulation, which brings many difficulties to the subsequent training of deep learning. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a preprocessing method for fundus image datasets. Under conditions where only a small amount of fundus images exist, and their quality is poor, adaptive enhancement and selective augmentation are performed on the small dataset, enabling it to be used to train a neural network recognition model and obtain high-precision fundus image screening results. To achieve the above objectives, this invention employs the following specific technical solutions:

[0005] A preprocessing method for a fundus image dataset includes the following steps:

[0006] S1. Adaptive enhancement is performed on the fundus images to optimize and obtain the initial dataset of fundus images;

[0007] Adaptive enhancement includes global adaptive optimization, local adaptive optimization, and local adaptive enhancement;

[0008] Global adaptive optimization is used to distinguish between overexposed and underexposed fundus images;

[0009] Local adaptive optimization is used to make the gray values ​​of fundus images uniformly distributed;

[0010] Local adaptive enhancement is used to filter out fundus images that are far from a preset threshold and to compensate fundus images with local shadows and local bright spots;

[0011] S2. Selectively augment the initial dataset;

[0012] S3. Use a generative adversarial network to further expand the dataset after the expansion in step S2.

[0013] Preferably, the global adaptive optimization includes a global contrast enhancement method based on histogram equalization.

[0014] Preferably, the local adaptive optimization includes a uniform distribution method that uses a cumulative distribution function to uniformly distribute the gray values ​​of the fundus image.

[0015] Preferably, the method of using the cumulative distribution function includes the following steps:

[0016] S111. Calculate the cumulative distribution function of the fundus image;

[0017] S112. Normalize the cumulative distribution function to the gray value range of the fundus image;

[0018] S113. Round the cumulative distribution function to the nearest integer to obtain the grayscale conversion function s. k =T(r) k );

[0019] S114. Using the cumulative distribution function as the transformation function, the gray level r k The point is converted to s k .

[0020] Preferably, local adaptive enhancement includes the following steps:

[0021] S121. Use the Tenengrad function to evaluate the quality of fundus images and filter out fundus images that are far from the preset threshold.

[0022] S122. Using a selective search algorithm, the fundus image is segmented into n segmentation regions. Using the segmentation regions as windows, fundus images that distinguish between local shadows and local bright spots are distinguished based on gray value characteristics.

[0023] S123. Optimize the fundus images of local shadows and local bright spots respectively.

[0024] Preferably, the Tenengrad function uses the Sobel operator to extract gradient values ​​in the horizontal and vertical directions, and its differential convolution kernel template is:

[0025]

[0026]

[0027] Where x and y represent the difference directions;

[0028] A represents the grayscale matrix of the fundus image.

[0029] Preferably, selective data augmentation includes four forms: random horizontal mirroring, translation, random shearing, and random scaling.

[0030] A fundus image training model is provided, which uses the aforementioned fundus image dataset preprocessing method to obtain the training dataset.

[0031] The present invention can achieve the following technical effects:

[0032] 1. This invention adaptively enhances and selectively expands a small dataset when only a small amount of fundus images with poor quality exist, so that a high-precision fundus image screening result can be obtained by training a neural network recognition model. Attached Figure Description

[0033] Figure 1 This is a flowchart of a preprocessing method for a fundus image dataset according to an embodiment of the present invention;

[0034] Figure 2 This is a flowchart of an adaptive enhancement according to an embodiment of the present invention;

[0035] Figure 3 This is a flowchart of a generative adversarial network according to an embodiment of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0037] The purpose of this invention is to provide a preprocessing method for fundus image datasets. The preprocessing method for fundus image datasets provided by this invention will be described in detail below through specific embodiments.

[0038] Figure 1 The flowchart of a preprocessing method for a fundus image dataset according to an embodiment of the present invention is shown, including the following steps:

[0039] S1. Adaptive enhancement is performed on the fundus images to optimize and obtain the initial dataset of fundus images;

[0040] S2. Selectively augment the initial dataset;

[0041] S3. Use generative adversarial networks to further augment the initial dataset.

[0042] This invention addresses the issue of poor-quality fundus images by employing global adaptive optimization, local adaptive optimization, and local adaptive enhancement to adaptively enhance small fundus image datasets, thereby obtaining a high-quality initial dataset. The initial dataset is then selectively augmented to ensure the quality of the augmented images. Finally, a generative adversarial network (GAN) is used to further augment the dataset, resulting in a final high-quality, large-scale fundus image dataset. This approach solves the problem of scarce fundus image data and also compensates for the low recognition accuracy caused by poor initial image quality, providing a high-quality training dataset for building subsequent fundus image training models.

[0043] Reference Figure 2 The flowchart shown illustrates adaptive enhancement, which includes global adaptive optimization, local adaptive optimization, and local adaptive enhancement.

[0044] In a preferred embodiment of the present invention, calculating the global grayscale distribution of an unprocessed, low-quality fundus image can distinguish between images with overexposure and underexposure problems. That is, underexposed images are generally dark and exhibit low grayscale distribution characteristics, while overexposed images are generally too bright and exhibit high grayscale distribution characteristics.

[0045] Therefore, all low-quality fundus images are subjected to global contrast enhancement based on histogram equalization to compensate for overexposed and underexposed images, thereby improving the overall image quality. Furthermore, by adjusting the grayscale distribution of the image, the distribution across the 0-255 grayscale levels is made more even, increasing contrast and thus improving image quality and enhancing image details.

[0046] In a preferred embodiment of the present invention, a cumulative distribution function is used to map the gray points in the fundus image after histogram equalization, so that the gray levels of the overall image conform to a uniform distribution, as follows:

[0047] S111. Calculate the gray probability density function based on the gray level of the fundus image, and calculate the cumulative distribution function of the fundus image based on the gray probability density function.

[0048] S112. Normalize the cumulative distribution function to the grayscale range of the low-quality, unprocessed fundus image.

[0049] S113. Round the cumulative distribution function to the nearest integer to obtain the grayscale conversion function s. k =T(r) k );

[0050] S114. Using the cumulative distribution function as the transformation function, the gray level r k The point is converted to s k .

[0051] For a real number x, the cumulative distribution function is defined as the sum of the probabilities of all values ​​less than or equal to x, and has the following expression:

[0052] F x (x)=P(X≤x)

[0053] Therefore, using the cumulative distribution function of gray levels as the transformation function can make the gray levels of the output image conform to a uniform distribution.

[0054] In a preferred embodiment of the present invention, reference continues to... Figure 2 For images that conform to a uniform grayscale distribution, i.e., images that have undergone global adaptive optimization and local adaptive optimization, classification based on quality parameters is performed. For low-quality images with different types of quality defects, corresponding local adaptive enhancement is performed to form an initial dataset of high-quality fundus images, while retaining the enhancement labels.

[0055] Low-quality images mainly include four types: local shadows, local flares, lens smudges, and large-area ghosting. Before enhancing these four types of images, a high-quality image I needs to be manually selected as a reference. The Tenengrad function is used to evaluate the image quality, judging the quality of the fundus image after global and local adaptive optimization. The specific method is as follows:

[0056] S121. Use the Tenengrad function to evaluate the quality of fundus images and filter out fundus images that are far from the preset threshold.

[0057] S122. Using a selective search algorithm, the fundus image is segmented into n segmentation regions. Using the segmentation regions as windows, fundus images that distinguish between local shadows and local bright spots are distinguished based on gray value characteristics.

[0058] S123. Optimize the fundus images of local shadows and local bright spots respectively.

[0059] In a preferred embodiment of the present invention, the Tenengrad function is a gradient-based function. It is generally believed that images with better focus have sharper edges and therefore larger gradient function values. However, to obtain the optimal quality effect, the preset threshold should be a range rather than a constant. Therefore, it is necessary to select a high-quality image I as the basic reference for all training images.

[0060] The Tenengrad function uses the Sobel operator to extract gradient values ​​in the horizontal and vertical directions. Its differential convolution kernel template is:

[0061]

[0062]

[0063] Where x and y represent the difference directions;

[0064] A represents the grayscale matrix of the fundus image.

[0065] The gradient of image I at point (x, y) is:

[0066] S(x,y)=G x *I(x,y)+G y *I(x, y)

[0067] Define the Tenengrad value of image I as follows:

[0068]

[0069] Where n is the total number of pixels in image I.

[0070] Using the Ten obtained from image I as a reference threshold range, the threshold truncation result divides the image into high-quality and low-quality images. Images with excessively poor quality, such as lens smudges and large-area ghost images, are filtered out. These two types of low quality are quite severe and almost impossible to classify. The resulting low-quality images are those with two types of local quality problems: local shadows and local bright spots. These two types of images require targeted compensation.

[0071] In a preferred embodiment of the present invention, a selective search algorithm is used to segment the lower-quality fundus image obtained above into n segmentation regions, generating a region set R. A greedy strategy is used to make judgments using local small windows. Specifically:

[0072] Calculate the similarity between any two adjacent segmented regions in R:

[0073] S = {s1, s2, ...}

[0074] Each time, the two most similar segmentation regions are merged into a new set. The similarity between these two segmentation regions is removed from S, and the merged set is added to R. The similarity between every two adjacent segmentation regions in the new R is then calculated until S is empty and there are no more adjacent segmentation regions. Finally, n segmentation regions are obtained.

[0075] Using the segmented region as a window, two types of fundus images, namely local shadow and local light spot, are distinguished based on gray value characteristics. The two types of fundus images are then optimized based on gray value increases and decreases to obtain the initial dataset of the final fundus images.

[0076] In step S2, the enhanced labels retained in step S1 are used to selectively augment the initial dataset.

[0077] In a preferred embodiment of the present invention, four data augmentation methods are adopted: random horizontal mirroring, translation, random shearing, and random scaling. Specifically:

[0078] The image changes relatively little after random horizontal mirroring and translation, and features are better preserved. The gray value corresponding to the coordinates (i,j) of any point on the processed image is calculated from the original coordinates (i,j), where the formula for random horizontal mirroring is:

[0079] I = i

[0080] J = N - j + 1

[0081] Where N is the image width.

[0082] The translation formula is:

[0083] I = i + a

[0084] J = j + b

[0085] Where a and b are the translation distances.

[0086] The image after random shearing and random scaling changes relatively significantly, exhibiting more diverse features. The random shearing matrix Q1 with angle α is:

[0087]

[0088] Random scaling is handled in two ways: downsampling and interpolation. In one embodiment of the invention, the downsampling ratio is set to 2:1, and the interpolation uses a bilinear method.

[0089] f(i+u,j+v)=(1-u)(1-v)f(i,j)+(1-u)vf(i,j+1)+u(1-v)f(i+1,j)+uvf(i+1,j+1)

[0090] Where f represents the pixel value at a certain location;

[0091] u and v represent the relative positions of the inserted values.

[0092] Therefore, the corresponding images are obtained by using the enhanced labels retained in step S1. For lower quality images, random horizontal mirroring and translation are used for expansion, while for higher quality images, random shearing and random scaling are used for expansion, thereby ensuring the quality of the expanded images.

[0093] Since the image dataset is limited and the training data is relatively small, the results may be inaccurate. Therefore, step S3 further adopts the GAN augmentation method to increase the number of datasets. Generative adversarial networks are used to augment the initial dataset again to form the final high-quality large fundus image set.

[0094] See Figure 3 The flowchart shown illustrates a Generative Adversarial Network (GAN), which consists of a generator network and a discriminator network. First, the generator is input with a hidden code C containing image information and Gaussian noise Z. The generator network G generates image samples G(Z). Then, random samples are taken from a fake dataset G(Z) and real data X, and input to the discriminator network D for evaluation. This completes the secondary augmentation of the initial dataset, resulting in a large amount of high-quality fundus image data, which serves as the foundation for subsequent algorithmic research.

[0095] A fundus image training model is provided, which uses the aforementioned fundus image dataset preprocessing method to obtain the training dataset.

[0096] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0097] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

[0098] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A preprocessing method for a fundus image dataset, characterized in that, Includes the following steps: S1. Adaptive enhancement is performed on the fundus images to optimize and obtain the initial dataset of the fundus images; The adaptive enhancement includes global adaptive optimization, local adaptive optimization, and local adaptive enhancement; The global adaptive optimization is used to distinguish between overexposed and underexposed fundus images. The global adaptive optimization includes a global contrast enhancement method based on histogram equalization. The global grayscale distribution of the unprocessed, low-quality fundus images is calculated to distinguish images with overexposure and underexposure problems. All low-quality fundus images are subjected to global contrast enhancement based on histogram equalization to compensate for overexposed and underexposed images. The local adaptive optimization is used to make the gray values ​​of the globally adaptive optimized fundus image uniformly distributed; The images after global adaptive optimization and local adaptive optimization are classified based on quality parameters. Corresponding local adaptive enhancement is performed on low-quality images with different types of quality defects. The local adaptive enhancement is used to filter out fundus images that are far from a preset threshold and to compensate fundus images with local shadows and local bright spots. The local adaptive enhancement includes the following steps: S121. Select a high-quality image. I As a reference, the Tenengrad function is used to evaluate the quality of the fundus images and filter out fundus images that are far from a preset threshold. S122. The fundus image is segmented into n segmentation regions using a selective search algorithm. Using the segmentation regions as windows, the fundus image is distinguished from the local shadows and local bright spots based on the gray value characteristics. S123. Optimize the fundus images of the local shadows and local light spots based on the increase or decrease of gray values ​​respectively; S2. Selectively augment the initial dataset; S3. Use a generative adversarial network to further expand the dataset after the expansion in step S2.

2. The preprocessing method for fundus image datasets according to claim 1, characterized in that, The local adaptive optimization includes a uniform distribution method that uses a cumulative distribution function to evenly distribute the gray values ​​of the fundus image.

3. The preprocessing method for fundus image datasets according to claim 2, characterized in that, The method of using the cumulative distribution function includes the following steps: S111. Calculate the cumulative distribution function of the fundus image; S112. Normalize the cumulative distribution function to the gray value range of the fundus image; S113. Round the cumulative distribution function to the nearest integer to obtain the grayscale conversion function. ; S114. Using the cumulative distribution function as a transformation function, the grayscale value is... The point is converted to .

4. The preprocessing method for fundus image datasets according to claim 1, characterized in that, The Tenengrad function uses the Sobel operator to extract gradient values ​​in the horizontal and vertical directions, and its differential convolution kernel template is: in, x , y Indicates the direction of the difference; A This represents the grayscale matrix of the fundus image.

5. The preprocessing method for fundus image datasets according to claim 1, characterized in that, The selective data augmentation includes four forms: random horizontal mirroring, translation, random shearing, and random scaling.

Images