Image pre-processing method and image processing device for fundus image

Abstract

The present invention provides an image pre-processing method and an image processing device for fundus images. A region of interest is obtained from the fundus image to generate a first image. The region of interest is for an eyeball in the fundus image. The first image is smoothed to generate a second image. The numerical difference between a plurality of adjacent pixels in the second image is increased to generate a third image. In this way, features can be enhanced and subsequent image recognition operations are facilitated.

Description

Technical Field

[0001] The present invention relates to an image processing technology, and in particular to an image pre-processing method and an image processing device for fundusscopic images. Background Art

[0002] Medical images are images taken of specific parts of an organism, and these images can be used to assess the risk of disease or the severity of a disease. For example, fundoscopic examinations can be used to detect retinal diseases, glaucoma, macular diseases, or other lesions early. Generally speaking, most doctors manually judge the lesions on medical images. Although medical images can now be evaluated with the help of computers, indicators such as efficiency, complexity, and accuracy still need to be broken through. Summary of the invention

[0003] The embodiment of the present invention is directed to an image pre-processing method and an image processing device for fundus images, which can enhance features and thus improve the accuracy of subsequent identification of lesions or other features.

[0004] According to an embodiment of the present invention, the image pre-processing method includes (but is not limited to) the following steps: obtaining a region of interest from a fundus image to generate a first image. The region of interest is for an eyeball in the fundus image. Smoothing the first image to generate a second image. Increasing the numerical difference between a plurality of adjacent pixels in the second image to generate a third image. The third image is used for image recognition.

[0005] According to an embodiment of the present invention, an image processing device includes (but is not limited to) a memory and a processor. The memory stores program code. The processor is coupled to the memory. The processor loads and executes the program code to be configured to obtain a region of interest from a fundus image to generate a first image, smooth the first image to generate a second image, and increase the numerical difference between multiple adjacent pixels in the second image to generate a third image. The region of interest is for an eyeball in the fundus image. The third image is used for image recognition.

[0006] Based on the above, according to the image pre-processing method and image processing device for fundus images of the embodiments of the present invention, before image recognition, the region of interest is cut out from the initial fundus image, and further smoothed and numerically enhanced. In this way, the features can be enhanced, the noise can be reduced, and the recognition accuracy of subsequent image recognition can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The accompanying drawings are included to provide a further understanding of the present invention and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the present invention and together with the description serve to explain the principles of the present invention.

[0008] Figure 1 is a block diagram of components of an image processing device according to an embodiment of the present invention;

[0009] Figure 2 is a flow chart of an image pre-processing method according to an embodiment of the present invention;

[0010] Figure 3 is a schematic diagram of a first image according to an embodiment of the present invention;

[0011] Figure 4 is a schematic diagram of a second image according to an embodiment of the present invention;

[0012] Figure 5 is a schematic diagram of a third image according to an embodiment of the present invention.

[0013] Description of Figure Numbers

[0014] 100: image processing device;

[0015] 110: memory;

[0016] 130: processor;

[0017] S210~S250: steps. DETAILED DESCRIPTION

[0018] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0019] Figure 1 is a block diagram of components of an image processing apparatus 100 according to an embodiment of the present invention. Figure 1 The image processing device 100 includes (but is not limited to) a memory 110 and a processor 130. The image processing device 100 may be a desktop computer, a notebook computer, a smart phone, a tablet computer, a server, a medical detection instrument or other computing devices.

[0020] The memory 110 may be any type of fixed or removable random access memory (RAM), read only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD) or similar components. In one embodiment, the memory 110 is used to record program codes, software modules, configurations, data (e.g., images, values, reference values, distances, etc.) or files, and its embodiments will be described in detail later.

[0021] The processor 130 is coupled to the memory 110, and the processor 130 may be a central processing unit (CPU), a graphics processing unit (GPU), or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), programmable controller, field programmable gate array (FPGA), application-specific integrated circuit (ASIC), neural network accelerator or other similar components or a combination of the above components. In one embodiment, the processor 130 is used to execute all or part of the operations of the image processing device 100, and can load and execute various program codes, software modules, files and data recorded in the memory 110.

[0022] Hereinafter, the method described in the embodiment of the present invention will be described in conjunction with various devices, components and modules in the image processing device 100. Each process of the method can be adjusted according to the implementation situation, and is not limited thereto.

[0023] Figure 2 is a flow chart of an image pre-processing method according to an embodiment of the present invention. Figure 2 , the processor 130 may obtain a region of interest from a fundus image to generate a first image (step S210). Specifically, a fundus image is an image obtained by performing fundus photography on a human or other organism. The processor 130 may obtain the fundus image through a built-in or external image capture device, or the processor 130 may download the fundus image from a server, a computer, or a storage medium. It is worth noting that the fundus images from different sources may have different shapes or sizes. In order to normalize these fundus images, the processor 130 may first cut out a region of interest that is considered to be important or useful information.

[0024] The region of interest of the embodiment of the present invention is for the eyeball in the fundus image. In one embodiment, the processor 130 can locate the center of the eyeball from the fundus image. For example, the processor 130 can regard the point where the straight lines in the gradient direction of the pixels in the fundus image intersect the most as the position of the eyeball. For another example, the processor 130 can perform a Hough transformation on the fundus image to select the circle that best meets the restriction conditions, and determine the center of the boundary contour of the region of interest accordingly. The processor 13 can further determine the region of interest based on the center of the eyeball. For example, the processor 130 can set a circle that is more consistent with or most consistent with the contour of the eyeball based on the center, and use the contour of this circle as the boundary of the region of interest. For another example, the processor 130 can use the circle obtained by the Hough transformation as the region of interest.

[0025] In another embodiment, the processor 130 may search for the boundary of the eyeball from the outside to the center of the fundus image. For example, the processor 130 scans the four sides of the fundus image toward the center in sequence and determines the brightness of the scanned area. It is worth noting that the brightness value (or brightness, i.e., the brightness of the color) on one side of the boundary of the eyeball is higher than that on the other side. Generally speaking, the brightness value of the area outside the eyeball in the fundus image is low and may be black. When the brightness difference of adjacent pixels on any side is higher than the difference threshold value or the brightness value of one or more pixels is higher than the brightness threshold value (i.e., the brightness value on one side is higher than that on the other side), the processor 130 may determine that the outermost side of the area of ​​interest on this side has been found. The processor 130 may use the outermost sides of the four sides of the fundus image as the boundary, i.e., form a quadrilateral. The processor 130 may use the shortest side length of this quadrilateral as the diameter of a circle (regarded as an eyeball), and use the circle formed by this diameter as the boundary of the eyeball. The center position of the circle is the center of the quadrilateral. In another embodiment, in order to filter out the interference that often appears in the periphery, the processor 130 uses half of the shortest side length of the quadrilateral multiplied by a floating point number greater than 0 and less than 1 as the radius length, and then takes a circle with this radius length. In addition, the center position of the circle is still located at the center of the quadrilateral. Then, the processor 130 can determine the region of interest based on the boundary of the eyeball. That is, the boundary of the eyeball is used as the boundary of the region of interest.

[0026] In one embodiment, the processor 130 may cut out a region of interest from the fundus image. That is, the processor 130 deletes the region of interest in the fundus image. The processor 130 may further add a background color outside the region of interest to form a first image. This background color will be regarded as useless information in subsequent image recognition of the fundus image (for example, identifying lesions, identifying severity, etc.). Useless information may be excluded by feature extraction or its value may be low. For example, the background color is formed by the values ​​of red, green, and blue being 128, 64, or 0, but is not limited to this. In addition, the size, shape, and / or proportion of the first image may be fixed to normalize different fundus images. In some embodiments, the aforementioned circle may be changed to an ellipse or other geometric shape.

[0027] For example, Figure 3 is a schematic diagram of a first image according to an embodiment of the present invention. Figure 3 ,The circular area (i.e., the area of ​​interest) shown in the figure corresponds to the eyeball.

[0028] The processor 130 may perform smoothing on the first image to generate a second image (step S230). Specifically, smoothing is a spatial domain filtering technique that can directly blur pixels in an image and remove noise. For example, the numerical difference (or distance) between adjacent pixels will be shortened.

[0029] In one embodiment, the smoothing process is Gaussian blur. The processor 130 may perform Gaussian blur on the first image. For example, the processor 130 uses a Gaussian convolution kernel to perform a convolution operation with each pixel in the first image, and then sums the convolution results to obtain the second image.

[0030] For example, Figure 4 is a schematic diagram of a second image according to an embodiment of the present invention. Figure 3 and Figure 4 ,and Figure 3 In comparison, the Gaussian blurred Figure 4 Some noise details in the image are blurred, but the edges of blood vessels, macula, veins and / or arteries are preserved.

[0031] In other embodiments, the smoothing process may also be median filtering, mean filtering, box filtering or other processes.

[0032] The processor 130 may increase the numerical difference between a plurality of adjacent pixels in the second image to generate a third image (step S250). Specifically, the smoothing process will bring the numerical difference between adjacent pixels closer. In order to further enhance the feature, in one embodiment, the processor 130 may increase the corresponding numerical difference in proportion to the distance between the numerical difference between those adjacent pixels and the reference value (i.e., update / change the numerical difference). For example, the reference value is 128, and the processor 130 may respectively calculate the original numerical difference between the numerical value of each pixel in the red, green, and blue channels and the adjacent pixels, and compare the distance between the original numerical difference and the reference value. If the distance is farther, the processor 130 may increase the amplitude of the increase in the numerical difference. If the distance is closer, the processor 130 may reduce the amplitude of the increase in the numerical difference. The ratio may be 1, 2, 5, or 10 times. Then, the processor 130 may change the numerical value of the corresponding pixel according to the increased numerical difference (i.e., the updated numerical difference) so that the numerical difference between the two pixels conforms to the updated numerical difference.

[0033] In some embodiments, the changed value has an upper or lower limit specification. For example, the upper limit is 255 and the lower limit is 0. When the changed value exceeds the upper or lower limit, it will be set to a specific value (e.g., an upper limit, a lower limit or other value).

[0034] It should be noted that the mathematical relationship between the original numerical difference and the updated numerical difference is not limited to a proportional relationship. In other embodiments, the processor 130 may also adopt a linear relationship, an exponential relationship or other mathematical relationships according to actual needs.

[0035] For example, Figure 5 is a schematic diagram of a third image according to an embodiment of the present invention. Figure 4 and Figure 5 , Figure 5 The blood vessels, macula, optic disc, veins, arteries, or other objects in the image are more visible.

[0036] It is also worth noting that the third image of the embodiment of the present invention can be used for image recognition. In one embodiment, the processor 130 inputs the third image into a detection model based on a machine learning algorithm (e.g., a deep neural network (DNN), a multi-layer perceptron (MLP), a support vector machine (SVM) or other machine learning models). In one embodiment, the detection model can be used for image recognition. It should be noted that the third image can be used for pre-processing of the training phase and / or the inference phase of the detection model. Generally speaking, the detection model usually does not undergo image pre-processing, and immediately acquires features of the initial fundus image. Through the image pre-processing of the embodiment of the present invention, the image recognition results for lesions such as hemorrhage, exudate, edema, etc. can be made more accurate. Alternatively, the image pre-processing of the embodiment of the present invention can make blood vessels, macula, veins and other parts easier to identify, but is not limited to this.

[0037] In another embodiment, the image recognition may be based on scale-invariant feature transform (SIFT), haar features, Adaboost or other recognition techniques.

[0038] In summary, in the image pre-processing method and image processing device for fundus images of the embodiments of the present invention, the region of interest in the fundus image is determined, the image is smoothed and the numerical difference is increased, thereby enhancing the features and facilitating subsequent image recognition, model training or other image applications.

[0039] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image pre-processing method, characterized in that: include: Obtaining a region of interest from a fundus image to generate a first image, wherein the region of interest is for an eyeball in the fundus image; performing smoothing processing on the first image to generate a second image; Comparing the distance between the numerical difference and the reference value, wherein the numerical difference is the difference between the numerical values ​​of each pixel in the red, green and blue channels and the adjacent pixels; and Increasing the numerical difference according to the distance to generate a third image, wherein the numerical values ​​of the corresponding pixels in the red, green and blue channels are changed according to the increased numerical difference, and the step of increasing the numerical difference according to the distance comprises: The numerical difference is increased in proportion to the distance between the numerical difference and the reference value, wherein If the distance is greater, the magnitude of the numerical difference will be increased; as well as If the distance is closer, the magnitude of the numerical difference will be reduced.

2. The image pre-processing method according to claim 1, characterized in that: The step of obtaining the region of interest from the fundus image comprises: Locating the center of the eyeball from the fundus image; and The region of interest is determined based on the center.

3. The image pre-processing method according to claim 1, characterized in that: The step of obtaining the region of interest from the fundus image comprises: Finding the boundary of the eyeball from the outside of the fundus image, wherein the brightness value on one side of the boundary of the eyeball is higher than that on the other side; and The region of interest is determined according to the boundary of the eyeball.

4. The image pre-processing method according to claim 1, characterized in that: The step of generating the first image comprises: cutting out the region of interest from the fundus image; and A background color is added outside the region of interest to form the first image.

5. The image pre-processing method according to claim 1, characterized in that: The smoothing process is Gaussian blur, and the step of performing the smoothing process on the first image includes: The Gaussian blur is performed on the first image.

6. The image pre-processing method according to claim 1, characterized in that: After the step of generating the third image, the method further includes: The third image is input into a detection model based on a machine learning algorithm.

7. An image processing device, characterized in that: include: Memory, storing program codes; as well as A processor, coupled to the memory, loads and executes the program code and is configured to: Obtaining a region of interest from a fundus image to generate a first image, wherein the region of interest is for an eyeball in the fundus image; performing smoothing processing on the first image to generate a second image; Comparing the distance between the numerical difference and the reference value, wherein the numerical difference is the difference between the numerical values ​​of each pixel in the red, green and blue channels and the adjacent pixels; and The processor is further configured to: increase the value difference according to the distance to generate a third image, wherein the values ​​of the corresponding pixels in the red, green and blue channels are changed according to the increased value difference, and the processor is further configured to: The numerical difference is increased in proportion to the distance between the numerical difference and the reference value, wherein If the distance is greater, the magnitude of the numerical difference will be increased; as well as If the distance is closer, the magnitude of the numerical difference will be reduced.

8. The image processing device according to claim 7, characterized in that: The processor is also configured to: Locating the center of the eyeball from the fundus image; and The region of interest is determined based on the center.

9. The image processing device according to claim 7, characterized in that: The processor is also configured to: Finding the boundary of the eyeball from the center from the outer side of the fundus image, wherein the brightness value on one side of the boundary of the eyeball is higher than that on the other side; and The region of interest is determined according to the boundary of the eyeball.

10. The image processing device according to claim 7, characterized in that: The processor is also configured to: cutting out the region of interest from the fundus image; and A background color is added outside the region of interest to form the first image.

11. The image processing device according to claim 7, characterized in that: The smoothing process is Gaussian blur, and the processor is further configured to: The Gaussian blur is performed on the first image.

12. The image processing device according to claim 7, characterized in that: The processor is also configured to: The third image is input into a detection model based on a machine learning algorithm.

Images