Eye fundus image difference analysis method and related product

Abstract

The invention discloses a method for fundus image difference analysis and a related product. The method comprises the following steps: preprocessing a first fundus image and a second fundus image of the same target in different periods, so that the first fundus image and the second fundus image have the same brightness distribution as a whole; a sliding window is adopted to scan the first eye fundus image and the second eye fundus image at the same time to obtain a first image set and a second image set, and the first image set and the second image set comprise a plurality of sliding window images; calculating a difference value between corresponding sliding window images in the first image set and the second image set; and carrying out fusion processing on all the sliding window images and the calculated difference values to obtain a difference distribution diagram for representing the difference between the first fundus image and the second fundus image. Through the scheme disclosed by the invention, the efficient and accurate analysis of the difference between the eye fundus images can be realized.

Description

Technical Field

[0001] The present disclosure generally relates to the field of image processing technology. More specifically, the present disclosure relates to a method for fundus image difference analysis, an electronic device for executing the aforementioned method, and a computer-readable storage medium. Background Art

[0002] There will be certain differences between two fundus images taken at different times of the same person. At present, the difference analysis between fundus images mainly relies on human observation and judgment, which not only has low analysis efficiency but also low analysis accuracy. The traditional method of automatically comparing image differences is mainly to compare the pixel values ​​of two images one by one. The traditional analysis method is not suitable for images generated by cameras, especially fundus images. Specifically, images taken at different times will be affected by multiple factors such as lighting, camera type and angle. The traditional pixel-by-pixel comparison method has been affected by these factors and cannot effectively extract the true difference between the two images.

[0003] In view of this, there is an urgent need to provide a solution for fundus image difference analysis in order to improve the efficiency and accuracy of image difference analysis. Summary of the invention

[0004] In order to at least solve one or more of the technical problems mentioned above, the present disclosure proposes a solution for fundus image difference analysis in multiple aspects.

[0005] In a first aspect, the present disclosure provides a method for fundus image difference analysis, the method comprising: preprocessing a first fundus image and a second fundus image of the same target at different periods so that the first fundus image and the second fundus image have the same brightness distribution as a whole; using a sliding window to simultaneously scan the first fundus image and the second fundus image to obtain a first image set and a second image set, wherein the first image set and the second image set contain multiple sliding window images; calculating the difference value between the corresponding sliding window images in the first image set and the second image set; and fusing all the sliding window images and the calculated difference values ​​to obtain a difference distribution map for representing the difference between the first fundus image and the second fundus image.

[0006] In some embodiments, preprocessing the first fundus image and the second fundus image of the same target at different time periods includes: performing histogram equalization processing on the first fundus image and the second fundus image.

[0007] In some embodiments, using a sliding window to simultaneously scan the first fundus image and the second fundus image includes: using sliding windows of the same size or multiple sliding windows of different sizes to scan the first fundus image and the second fundus image.

[0008] In some embodiments, the multiple sliding window images include one or more sliding window images under different size windows, and calculating the difference value between the corresponding sliding window images in the first image set and the second image set includes: calculating the difference value between the corresponding sliding window images in the same size window in the first image set and the second image set.

[0009] In some embodiments, calculating the difference value between corresponding sliding window images in the same size window in the first image set and the second image set includes: calculating the histogram distance between corresponding sliding window images in the same size window in the first image set and the second image set; and / or calculating the contrast distance between corresponding sliding window images in the same size window in the first image set and the second image set; and determining the difference value between corresponding sliding window images in the same size window in the first image set and the second image set based on the histogram distance and / or contrast distance between corresponding sliding window images in the same size window in the first image set and the second image set.

[0010] In some embodiments, determining the difference value between corresponding sliding window images in the same size window in the first image set and the second image set includes: summing the histogram distance and the contrast distance between corresponding sliding window images in the same size window in the first image set and the second image set, and determining the summation result as the difference value between corresponding sliding window images in the same size window in the first image set and the second image set.

[0011] In some embodiments, the histogram distance is calculated according to the following formula: in, represents the histogram of the sliding window images in the first image set and the second image set respectively, i represents the number of sliding window images; the contrast distance is calculated according to the following formula: in, Respectively represent the contrast of the sliding window images in the first image set and the second image set.

[0012] In some embodiments, fusing all sliding window images and calculated difference values ​​includes: fusing the sliding window images under each size window and the corresponding difference values; and

[0013] The fusion results under all size windows are superimposed to obtain the difference distribution map.

[0014] In a second aspect, the present disclosure provides an electronic device comprising: a processor; and a memory storing computer instructions for processing ultra-wide-angle fundus images, wherein when the computer instructions are executed by the processor, the electronic device executes the method described in the first aspect of the present disclosure.

[0015] In a third aspect, the present disclosure provides a computer-readable storage medium comprising program instructions for processing ultra-wide-angle fundus images. When the program instructions are executed by a processor, the method described in the first aspect of the present disclosure is implemented.

[0016] Through the scheme for fundus image difference analysis provided above, the disclosed embodiment preprocesses the two fundus images to be analyzed so that the two fundus images have the same brightness distribution as a whole, and then analyzes the detail differences of the two fundus images in combination with a sliding window. It can be seen that the disclosed scheme does not require manual intervention, and can achieve macroscopic regulation through image preprocessing to eliminate the interference of factors such as illumination on image analysis, and then gradually compare the differences in the same areas of the two fundus images in combination with a sliding window, thereby achieving microscopic difference analysis to reduce the interference of factors such as angles on image analysis. Thus, efficient and accurate analysis of the differences between fundus images is achieved. Further, in some embodiments, multiple sliding windows of different sizes can be used to scan two fundus images, and a difference distribution map of the two fundus images can be obtained based on a mixture of difference values ​​under all size windows. Thus, the influence of angle differences can be minimized, and the accuracy of the difference analysis results can be further ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By reading the detailed description below with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0018] Figure 1 An exemplary flow chart showing a method for fundus image difference analysis according to an embodiment of the present disclosure is shown;

[0019] Figure 2 An exemplary flow chart showing a method for fundus image difference analysis according to another embodiment of the present disclosure;

[0020] Figure 3 An exemplary flow chart showing a method for fundus image difference analysis according to yet another embodiment of the present disclosure;

[0021] Figure 4 A comparison diagram of fundus image difference analysis of an embodiment of the present disclosure is shown;

[0022] Figure 5 A comparison diagram showing the effects before and after histogram equalization processing is used in the fundus image difference analysis process of the disclosed embodiment; and

[0023] Figure 6 An exemplary structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0025] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0026] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0027] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0028] Fundus images at different times are easily affected by factors such as shooting angle and lighting during the acquisition process. The traditional method of comparing the pixel values ​​of images one by one is easily affected by the above interference factors and cannot be applied to the difference analysis of fundus images. The inventors have found that the fundus images can be processed macroscopically to reduce the influence of lighting factors, and combined with sliding windows to achieve microscopic difference analysis to reduce the influence of angle factors, so that the true differences between fundus images can be analyzed with reduced accuracy.

[0029] In view of this, the presently disclosed embodiment provides a solution for fundus image difference analysis, which preprocesses two fundus images so that the two images have the same brightness distribution as a whole, thereby reducing the impact of brightness differences on the analysis results, and combines a sliding window to perform block analysis on the two fundus images, thereby reducing the impact of angle differences on the analysis results.

[0030] The specific implementation of the present disclosure is described in detail below with reference to the accompanying drawings.

[0031] Figure 1 An exemplary flow chart of a method 100 for fundus image difference analysis according to an embodiment of the present disclosure is shown.

[0032] like Figure 1 As shown, at step S101, the first fundus image and the second fundus image of the same target at different periods can be preprocessed so that the first fundus image and the second fundus image have the same brightness distribution as a whole. When shooting fundus images of the same target (such as a person) at different periods, the fundus images of different periods will be different due to factors such as the shooting model and light. However, the difference between the images caused by this interference factor is not a real difference, that is, the difference between the images caused by this interference factor has no practical significance. Therefore, in the process of image difference analysis, it is necessary to avoid the influence of the difference caused by the interference factor on the analysis result. Here, the first fundus image and the second fundus image are preprocessed (such as brightness adjustment processing, etc.) so that the first fundus image and the second fundus image have the same brightness distribution as a whole, thereby weakening the brightness difference between the images caused by interference factors such as light.

[0033] Next, at step S102, a sliding window may be used to simultaneously scan the first fundus image and the second fundus image to obtain a first image set and a second image set. The first image set and the second image set include a plurality of sliding window images. Specifically, after preprocessing the fundus images, a sliding window may be used to simultaneously scan the first and second fundus images to provide necessary guarantees for subsequent block analysis of microscopic differences between the two fundus images.

[0034] Next, at step S103, the difference value between the corresponding sliding window images in the first image set and the second image set can be calculated. The difference value between the same area in the two fundus images is analyzed in blocks by sliding window scanning, which can weaken the difference between images caused by interference factors such as angles.

[0035] Finally, at step S104, all sliding window images and the calculated difference values ​​may be fused to obtain a difference distribution map for representing the difference between the first fundus image and the second fundus image.

[0036] Therefore, the above analysis process for fundus image differences does not require manual intervention. Macroscopic regulation can be achieved through image preprocessing to eliminate the interference of factors such as illumination on image analysis, and then the differences in the same area of ​​the two fundus images are gradually compared in combination with the sliding window, thereby achieving microscopic difference analysis to reduce the interference of factors such as angles on image analysis. In this way, efficient and accurate analysis of effective differences between fundus images is achieved.

[0037] Figure 2 FIG. 2 is an exemplary flow chart of a method 200 for fundus image difference analysis according to another embodiment of the present disclosure. It should be noted that: Figure 2 Yes Figure 1 Therefore, the above combined with the method 100 Figure 1 The relevant detailed description also applies to the following.

[0038] like Figure 2 As shown, at step S201, the first fundus image and the second fundus image can be subjected to histogram equalization processing so that the first fundus image and the second fundus image have the same brightness distribution as a whole. Specifically, histogram equalization can transform the histograms of the first fundus image and the second fundus image into a uniformly distributed form, thereby increasing the dynamic range of the grayscale value difference between pixels, thereby achieving the effect of enhancing the overall contrast of the image. Specifically, the grayscale values ​​with a large number of pixels in the image can be widened, and the grayscale values ​​with a small number of pixels can be merged, thereby increasing the contrast, so that the first fundus image and the second fundus image have the same brightness distribution as a whole. It should be noted that the use of histogram equalization to achieve image preprocessing is only an exemplary description, and the scheme disclosed herein is not limited thereto.

[0039] Next, at step S202, a plurality of sliding windows of different sizes may be used to scan the first fundus image and the second fundus image to obtain a first image set and a second image set. Specifically, the number of sliding windows of different sizes is not limited here, and may be adjusted according to application requirements and the resolution of the fundus image. For example, a sliding window of 5×5 size, or a sliding window of 10×10 size, or a sliding window of 20×20 size, etc. may be used. Of course, in some embodiments, only a sliding window of 1 size may be used to scan the first fundus image and the second fundus image. Thus, the first image set and the second image set include multiple sliding window images under 1 or more windows of different sizes.

[0040] Next, for each sliding window of different sizes, the difference value between the corresponding sliding window images in the first image set and the second image set under the same size window can be calculated. For example, in step S203, the histogram distance between the corresponding sliding window images in the first image set and the second image set under the same size window can be calculated. In some embodiments, the difference value can be calculated according to the formula Calculate the histogram distance. Among them, They represent the histograms of the sliding window images in the first image set and the second image set respectively, and i represents the number of sliding window images. It means that the image is histogrammed according to the pixel value to obtain a 1x256 vector. The jth element of the histogram vector represents the image. The number of pixels with pixel value j is obtained by histogramming the images in the same area of ​​the same size window of the two images and then calculating the histogram distance between the two images.

[0041] At step S204, the contrast distance between the corresponding sliding window images in the same size window in the first image set and the second image set can be calculated. In some embodiments, the formula Calculate the contrast distance. Where, Denote the contrast of the sliding window images in the first image set and the second image set respectively. Where contrast(x)=0.212R(x)+0.715G(x)+0.007B(x), which is the weighted average sum of the RGB three channels of the image.

[0042] Next, at step S205, the difference value between the corresponding sliding window images in the same size window in the first image set and the second image set can be determined based on the histogram distance and / or contrast distance between the corresponding sliding window images in the same size window in the first image set and the second image set. It should be noted that, in some embodiments, only step S203 can be performed to calculate the histogram distance between the images, and the difference value between the corresponding sliding window images in the same size window in the first image set and the second image set can be determined based on the histogram distance between the corresponding sliding window images in the same size window in the first image set and the second image set. In other embodiments, it may be necessary to perform steps S203 and S204 to calculate the histogram distance and contrast distance between the images respectively. At this time, it is necessary to determine the difference value between the corresponding sliding window images in the same size window in the first image set and the second image set based on the histogram distance and contrast distance between the corresponding sliding window images in the same size window in the first image set and the second image set.

[0043] Furthermore, in some embodiments, the histogram distance and the contrast distance between the corresponding sliding window images in the same size window in the first image set and the second image set can be summed, and the summation result can be determined as the difference value between the corresponding sliding window images in the same size window in the first image set and the second image set.

[0044] Next, at step S206, the sliding window image under each size window and the corresponding difference value may be fused.

[0045] Finally, in step S207, the fusion results of all size windows are superimposed to obtain a difference distribution map.

[0046] Therefore, the image is first histogram equalized to achieve brightness invariance, and then sliding windows of different sizes are used to calculate the histogram distance of the green, red, and blue three-channel histogram resolution of the blocks in the window. The calculation results are then mixed with the contrast of the two windows. Finally, the results of windows of different scales are fused to ensure that the image difference distribution effect is stable and has a certain degree of robustness to lighting and angles.

[0047] Figure 3 FIG. 3 is an exemplary flow chart of a method 300 for fundus image difference analysis according to another embodiment of the present disclosure. It should be noted that: Figure 3 yes Figure 1 Method 100 and Figure 2 A specific implementation process of method 200. Therefore, the above Figure 1 and Figure 2 The relevant detailed description also applies to the following.

[0048] like Figure 3 As shown, at step S11, the first fundus image of the patient (i.e., image 1) is collected. At step S12, the second fundus image of the same patient at a different time (i.e., image 2) is collected. At S13, the two fundus images are preprocessed using histogram equalization. The two fundus images after histogram equalization will weaken the influence of interference factors such as light on the effective difference. Then, three sliding windows of different sizes are used to scan the two images simultaneously. For example, at S14, a sliding window of size 1 (e.g., 16 pixels) can be used to scan the two images simultaneously. At S15, a sliding window of size 2 (e.g., 32 pixels) can be used to scan the two images simultaneously. At S16, a sliding window of size 3 (e.g., 64 pixels) can be used to scan the two images simultaneously. Among them, the first image can be denoted as X 1 , the second image is denoted as X 2 . Each image can be viewed as a collection of sliding window images, for example: represents the image in the first sliding window of the first image. n represents the number of windows into which the first image is divided. When the window size is larger, n is smaller; when the window size is smaller, n is larger. It should be noted that the description of the number of sliding windows here is only an exemplary description.

[0049] At step S17, the histogram distance between the two corresponding windows in the image can be calculated. At step S18, the contrast distance between the two corresponding windows in the image can be calculated. The histogram distance and contrast distance here can be referred to in the previous text. Figure 2 The calculation is performed using the relevant formulas in the text, which will not be repeated here.

[0050] In step S19, the histogram distance and the contrast distance are superimposed and mixed. Specifically, it can be expressed by the formula To mix the histogram distance and contrast distance, where i = 1, 2, ..., n.

[0051] Finally, in step S20, the difference values ​​under each window and each window size may be fused to obtain a final difference distribution map.

[0052] The following combination Figure 4 The above process is further explained. Figure 4In the figure, image 41 is the fundus image taken for the first time (i.e., image 1), and image 42 is the fundus image taken for the second time after a year (i.e., image 2). Images 41 and 42 are first processed by histogram equalization, and then scanned using three sliding windows of different sizes, where images 44, 45, and 46 respectively show the difference distribution maps obtained by sliding windows of different scales. Among them, image 44 uses a difference distribution map obtained by a sliding window of 16 pixels, image 45 uses a difference distribution map obtained by a sliding window of 32 pixels, and image 46 uses a difference distribution map obtained by a sliding window of 64 pixels. Image 43 is the final difference distribution map that combines three sliding windows of different scales. It can be seen that image 43 can better highlight the effective difference between the two fundus images.

[0053] In the above image difference analysis process, it is preferred to use histogram equalization to pre-process the image to reduce the interference of brightness on abnormal distribution. Figure 5 For explanation. Figure 5 In the figure, image 51 is the fundus image of a patient taken for the first time, and image 52 is the fundus image taken for the second time after a year. It can be seen that there is no difference between the two images. However, without the processing of histogram equalization, the difference algorithm will be affected by brightness and illumination, and a difference distribution diagram as shown in image 53 will be obtained. It can be seen that there is a large area of ​​difference in image 53. Image 54 is the image of image 51 after histogram equalization, image 55 is the image of image 52 after histogram equalization, and image 56 is the difference distribution diagram obtained based on image 54 and image 55. It can be clearly seen that the histogram equalization processing method significantly reduces the interference of brightness on the difference distribution.

[0054] In addition, the difference distribution map generated by the difference analysis of fundus images can facilitate the long-term health management of the same patient process and the establishment of personal fundus health records. It should be noted that the application scenario of the scheme disclosed in this disclosure is not limited to fundus images, but can also be extended to other medical images. Through the scheme disclosed in this disclosure, the interference of factors such as lighting and angle can be effectively reduced, and the accuracy of image difference analysis can be improved.

[0055] The present disclosure also provides an electronic device. Figure 6 As shown, the electronic device 600 may include a processor 601 and a memory 602. The memory 602 stores computer instructions for fundus image difference analysis. When the computer instructions are executed by the processor 601, the electronic device 600 executes the above-mentioned method in combination with the method described above. Figures 1 to 3The method described. For example, in some embodiments, the electronic device 600 can pre-process the first fundus image and the second fundus image of the same target at different periods (for example, perform histogram equalization on the fundus image), use a sliding window to scan the fundus images simultaneously, calculate the difference values ​​between the corresponding sliding window images in different image sets (for example, histogram distance calculation and contrast distance calculation), and determine the final difference distribution map based on all the calculated difference values. Based on this, in the image difference analysis process, the electronic device 600 can achieve macroscopic regulation through image pre-processing to eliminate the interference of factors such as illumination on image analysis without manual intervention, and then gradually compare the differences in the same areas of the two fundus images in combination with the sliding window, thereby achieving microscopic difference analysis to reduce the interference of factors such as angles on image analysis. In this way, efficient and accurate analysis of the differences between fundus images is achieved.

[0056] Although multiple embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art may think of many changes, modifications, and alternatives without departing from the thought and spirit of the present disclosure. It should be understood that in the process of practicing the present disclosure, various alternatives to the embodiments of the present disclosure described herein may be adopted. The attached claims are intended to define the scope of protection of the present disclosure, and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for fundus image difference analysis, characterized in that: The method comprises: Preprocessing a first fundus image and a second fundus image of the same target at different periods so that the first fundus image and the second fundus image have the same brightness distribution as a whole; Scanning the first fundus image and the second fundus image simultaneously using a sliding window to obtain a first image set and a second image set, wherein the first image set and the second image set contain a plurality of sliding window images; Calculating difference values ​​between corresponding sliding window images in the first image set and the second image set; and All sliding window images and the calculated difference values ​​are fused to obtain a difference distribution map for representing the difference between the first fundus image and the second fundus image.

2. The method according to claim 1, characterized in that: Preprocessing the first fundus image and the second fundus image of the same target at different periods includes: A histogram equalization process is performed on the first fundus image and the second fundus image.

3. The method according to claim 1, characterized in that Using a sliding window to simultaneously scan the first fundus image and the second fundus image includes: The first fundus image and the second fundus image are scanned using sliding windows of the same size or multiple sliding windows of different sizes.

4. The method according to claim 3, characterized in that: The multiple sliding window images include one or more sliding window images under windows of different sizes, and calculating the difference value between the corresponding sliding window images in the first image set and the second image set includes: The difference values ​​between corresponding sliding window images under the same size window in the first image set and the second image set are calculated.

5. The method according to claim 4, characterized in that Calculating the difference value between corresponding sliding window images in the same size window in the first image set and the second image set comprises: Calculating the histogram distance between corresponding sliding window images under the same size window in the first image set and the second image set; and / or Calculating the contrast distance between corresponding sliding window images under the same size window in the first image set and the second image set; and Based on the histogram distance and / or contrast distance between the corresponding sliding window images in the same size window in the first image set and the second image set, the difference value between the corresponding sliding window images in the same size window in the first image set and the second image set is determined.

6. The method according to claim 5, characterized in that Determining the difference value between corresponding sliding window images in the same size window in the first image set and the second image set includes: The histogram distance and the contrast distance between the corresponding sliding window images under the same size window in the first image set and the second image set are summed, and the summation result is determined as the difference value between the corresponding sliding window images under the same size window in the first image set and the second image set.

7. The method according to claim 5, characterized in that in, The histogram distance is calculated according to the following formula: in, represent the histograms of the sliding window images in the first image set and the second image set respectively, and i represents the number of sliding window images; The contrast distance is calculated according to the following formula: in, Respectively represent the contrast of the sliding window images in the first image set and the second image set.

8. The method according to any one of claims 4 to 7, characterized in that The fusion process of all sliding window images and calculated difference values ​​includes: Fusing the sliding window image and the corresponding difference value under each size window; and The fusion results under all size windows are superimposed to obtain the difference distribution map.

9. An electronic device, characterized in that: include: processor; as well as A memory storing computer instructions for fundus image difference analysis, wherein when the computer instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The invention comprises program instructions for fundus image difference analysis, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images