Image processing method, fundus image comparison method and density change image application method

Abstract

The invention discloses an image processing method, a fundus image comparison method and a density change chart application method, particularly relates to the technical field of image processing, and aims to assist a doctor in determining fundus change and reduce misdiagnosis or missed diagnosis of the doctor. The image processing method comprises the steps that N digital images corresponding to N to-be-processed fundus images are determined based on leopard spot areas of the N to-be-processed fundus images, and N is larger than or equal to 2; based on the N digital images, M density change diagrams are determined, the density change diagrams are used for representing position distribution and / or value distribution of leopard spot density change of the fundus image to be processed, and M is smaller than or equal to N-1. According to the embodiment of the invention, the M density change diagrams are determined through the N digital images, and the density change diagrams can assist a doctor in knowing the position distribution of the leopard spot density change, so that the doctor can more accurately determine the position distribution of the leopard spot density and the size change of the leopard spot density, know the illness state of the patient and timely discover the change of the fundus of the patient at an early stage.

Description

Technical Field

[0001] The present disclosure belongs to the technical field of image processing, and specifically relates to an image processing method, a fundus image comparison method, and a density change map application method. Background Art

[0002] As people pay more attention to fundus health, usually, when an eye disease is discovered, a long-term observation or treatment phase will be carried out. During the observation or treatment phase, patients will take fundus images at different times according to treatment needs, so that doctors can understand the development of the disease, treat the fundus disease in time, and avoid the development of the disease.

[0003] However, since the fundus images taken by patients during the observation or treatment phase are independent, doctors need to observe and judge the progression of the disease based on their own experience. When the fundus images undergo subtle changes due to a short shooting time, it may be difficult for doctors to detect changes in the disease. In addition, due to the differences in doctors' own medical experience, there may be misdiagnosis due to inability to detect changes in the disease, which may delay the patient's treatment.

[0004] Therefore, there is an urgent need for an image processing method for fundus images to assist doctors in determining changes in fundus images and reduce misdiagnosis or missed diagnosis by doctors. Summary of the invention

[0005] In view of this, the present disclosure provides an image processing method to assist doctors in determining fundus changes, improve the accuracy of doctors in determining fundus changes, and thus reduce the incidence of doctors' misdiagnosis or missed diagnosis.

[0006] In a first aspect, an image processing method provided by an embodiment of the present disclosure includes: based on the leopard spot areas of N fundus images to be processed, determining N digital images corresponding to the N fundus images to be processed, wherein N≥2; based on the N digital images, determining M density change maps, the density change maps are used to characterize the positional distribution and / or numerical distribution of the leopard spot density changes of the fundus images to be processed, wherein M≤N-1.

[0007] In combination with the first aspect, in certain implementations of the first aspect, based on N digital images, M density change maps are determined, including: calculating the difference between the digital image corresponding to the i-th fundus image to be processed and the digital image corresponding to the u-th fundus image to be processed, and determining the difference image corresponding to the i-th fundus image to be processed; determining the M density change maps based on the difference image corresponding to the i-th fundus image to be processed; or, determining the M density change maps based on the difference image corresponding to the i-th fundus image to be processed and a preset interval time, wherein the preset interval time is used to characterize the interval time between the shooting time of the i-th fundus image to be processed and the shooting time of the u-th fundus image to be processed, wherein u<i and 2≤i≤N.

[0008] In combination with the first aspect, in certain implementations of the first aspect, based on the difference image corresponding to the i-th fundus image to be processed, M density change maps are determined, including: based on the M difference images, determining the visualization images corresponding to the M difference images, the visualization images including several color features; based on the visualization images, determining the density change map, the several color features of the visualization image corresponding to different leopard spot density changes in the density change map.

[0009] In combination with the first aspect, in certain implementations of the first aspect, based on the difference image corresponding to the i-th fundus image to be processed and the preset interval time, M density change maps are determined, including: based on the difference image corresponding to the i-th fundus image to be processed and the preset time threshold, determining the unit time change maps corresponding to the M difference images; based on the unit time change maps, determining the M density change maps.

[0010] In combination with the first aspect, in certain implementations of the first aspect, before determining N digital images corresponding to the N fundus images to be processed based on the leopard spot areas of the N fundus images to be processed, the method also includes: determining a reference image among the N fundus images to be processed; based on the reference image, performing image registration on the N-1 fundus images to be processed, and determining a registered image corresponding to the N-1 fundus images to be processed.

[0011] In combination with the first aspect, in certain implementations of the first aspect, before determining a reference image among N fundus images to be processed, the image processing method further includes: preprocessing the N initial fundus images to be processed, and determining a preprocessed fundus image corresponding to the N initial fundus images to be processed, wherein the preprocessing includes at least one of normalization processing, enhancement processing, and denoising processing; and determining N fundus images to be processed based on the preprocessed fundus image.

[0012] In a second aspect, an embodiment of the present disclosure provides a fundus image comparison method, including: determining a comparison reference image among N fundus images to be processed; determining a visualization image corresponding to the comparison reference image based on the leopard spot area of ​​the comparison reference image; using the image processing method mentioned in the first aspect above to process the N fundus images to be processed, and determine M density change maps; based on the visualization image corresponding to the comparison reference image and the M density change maps, determining the comparison results corresponding to N-1 fundus images to be processed, wherein the N-1 fundus images to be processed are images obtained by removing the comparison reference image from the N fundus images to be processed, and the comparison results are used to characterize the image changes of the N fundus images to be processed.

[0013] In a third aspect, an image processing device provided by an embodiment of the present disclosure includes: a determination module for determining N digital images corresponding to N fundus images to be processed based on the leopard spot areas of the N fundus images to be processed, wherein N≥2; a processing module for determining M density change maps based on the N digital images, the density change maps being used to characterize the positional distribution and / or numerical distribution of the leopard spot density changes of the fundus images to be processed, wherein M<N.

[0014] In a fourth aspect, a fundus image comparison device provided by an embodiment of the present disclosure includes: a comparison determination module, used to determine a comparison reference image in N fundus images to be processed; a visualization processing module, used to determine a visualization image corresponding to the comparison reference image based on the leopard spot area of ​​the comparison reference image; a density change determination module, used to process the N fundus images to be processed using the image processing method mentioned in the first aspect above, and determine M density change maps; a comparison module, used to determine a comparison result corresponding to N-1 fundus images to be processed based on the visualization image corresponding to the comparison reference image and the M density change maps, wherein the N-1 fundus images to be processed are images obtained by removing the comparison reference image from the N fundus images to be processed, and the comparison result is used to characterize the image changes of the N fundus images to be processed.

[0015] In a fifth aspect, an electronic device provided by an embodiment of the present disclosure includes: a processor; and a memory for storing instructions executable by the processor, wherein the processor is used to execute the method mentioned in the first aspect and / or the second aspect above.

[0016] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, which stores a computer program, and the computer program is used to execute the method mentioned in the first aspect and / or the second aspect above.

[0017] In the seventh aspect, an embodiment of the present disclosure provides a method for applying a density change map, including: the density change map formed by the image processing method mentioned in the first aspect is applied to ophthalmic diagnosis and analysis reports, medical research fields, image database construction fields, telemedicine fields, clinical trials fields, medical insurance claims fields, image processing algorithm development fields, education and training fields, or intelligent ophthalmic auxiliary diagnosis and treatment systems.

[0018] The image processing method provided by the embodiment of the present disclosure determines M density change maps through N digital images. The density change map represents the position distribution of the leopard spot density change of the i-th fundus image to be processed relative to the u-th fundus image to be processed. The method can assist doctors in understanding the position distribution of the leopard spot density change and the size change of the leopard spot density, thereby enabling doctors to more accurately determine the position change of the leopard spot density, understand the patient's condition, and promptly detect changes in the patient's fundus at an early stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure.

[0020] Figure 1 Shown is a schematic diagram of an application scenario provided by an embodiment of the present disclosure.

[0021] Figure 2 The figure is a flowchart of an image processing method provided by an embodiment of the present disclosure.

[0022] Figure 3 The figure is a schematic diagram of a flow chart of determining M density variation maps based on N digital images provided by an embodiment of the present disclosure.

[0023] Figure 4 It is shown that an embodiment of the present disclosure provides a method for determining M density change maps based on the difference image corresponding to the i-th fundus image to be processed.

[0024] Figure 5 The figure is a schematic diagram of a flow chart of determining M density change maps based on a difference image corresponding to the i-th fundus image to be processed and a preset interval time provided by an embodiment of the present disclosure.

[0025] Figure 6 Shown is a flowchart of an image processing method provided by another embodiment of the present disclosure.

[0026] Figure 7 Shown is a flowchart of an image processing method provided by another embodiment of the present disclosure.

[0027] Figure 8 The figure is a flow chart of a fundus image comparison method provided in one embodiment of the present disclosure.

[0028] Fig. 9 Shown is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure.

[0029] Fig.10 Shown is a schematic structural diagram of a fundus image comparison device provided in one embodiment of the present disclosure.

[0030] Fig.11 Shown is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0032] As people pay more attention to fundus health, usually, when an eye disease is discovered, such as myopia, glaucoma, diabetic retinopathy, etc., the patient will undergo a long-term observation or treatment stage. For example, patients with myopia need to observe the condition for a long time. As the disease progresses, when myopia develops to a certain extent, timely intervention and treatment are required. During the treatment stage, patients will take fundus images at different times according to treatment needs, so that doctors can understand the development of the disease, treat fundus diseases in a timely manner, or adjust the treatment plan of the disease in a timely manner, so as to avoid the development of the disease.

[0033] However, during the observation or treatment stage, patients need to take multiple fundus images within a certain period of time. Fundus images taken at different times are independent, and doctors need to observe and judge the progression of the disease based on their own experience. When the fundus image changes slightly due to a short shooting time, doctors use their naked eyes to subjectively judge the changes in the condition, and it may be difficult for doctors to detect changes in the condition. In addition, due to the differences in doctors' own medical experience, such as in some remote areas or areas with insufficient medical resources, doctors' own lack of experience may also make it impossible to detect changes in the condition, resulting in misdiagnosis or missed diagnosis, which delays the patient's treatment.

[0034] Therefore, there is an urgent need for an image processing method for fundus images to assist doctors in determining changes in fundus images and reduce misdiagnosis or missed diagnosis by doctors.

[0035] Combine the following Figure 1 The application scenario of an embodiment of the present disclosure is briefly introduced.

[0036] Figure 1 FIG. 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure. Figure 1 As shown, the scene is a scene of image processing. Specifically, the scene of image processing includes a server 110 and a user terminal 120 connected to the server 110 for communication. The server 110 is used to execute the method mentioned in the embodiment of the present disclosure.

[0037] For example, in actual application, the user issues an instruction to process an image through the user terminal 120. After receiving the instruction, the server 110 determines N digital images corresponding to the N fundus images to be processed based on the leopard spot areas of the N fundus images to be processed, where N≥2; and determines M density change maps based on the N digital images, where the density change maps are used to characterize the position distribution and / or value distribution of the leopard spot density changes of the fundus images to be processed, where M≤N-1. The server 110 sends the M density change maps to the user terminal 120 so that the user can view the M density change maps through the user terminal 120.

[0038] Exemplarily, the fundus images to be processed mentioned above include but are not limited to all image data stored by the medical institution during the treatment of the same patient, relevant fundus image data of the patient input by the user, and fundus image data taken by the patient as needed during the consultation. The fundus images to be processed can be 45° fundus images, 60° fundus images, wide-angle fundus images, fundus images with other field angles, or even fundus images of other modalities. The fundus images can be taken with the optic disc as the center, or with the macula as the center, or other eye position images. Exemplarily, the server 110 can directly receive the fundus images to be processed, or can obtain them from a data storage device.

[0039] Exemplarily, the above-mentioned users may be doctors, researchers of related diseases or other persons who want to understand the location of changes in fundus structure, and the present disclosure does not further limit the specific types of users.

[0040] Exemplarily, the user terminal 120 mentioned above includes but is not limited to computer terminals such as desktop computers and laptop computers and mobile terminals such as tablet computers and mobile phones.

[0041] Combine the following Figures 2 to 7 The image processing method provided by the present invention is briefly introduced.

[0042] Figure 2 FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present disclosure. Figure 2 As shown, an image processing method provided by an embodiment of the present disclosure includes the following steps.

[0043] Step S210 , determining N digital images corresponding to the N fundus images to be processed based on the leopard spot regions of the N fundus images to be processed.

[0044] Exemplarily, N≥2. Exemplarily, when N is 2, based on the leopard spot area of ​​the two fundus images to be processed, according to the density of the leopard spot area of ​​the fundus images to be processed, the two digital images corresponding to the two fundus images to be processed are determined. The numerical value in the digital image is used to characterize the density of the leopard spot area, and the digital image is a two-dimensional digital image, corresponding to the fundus image to be processed.

[0045] Step S220, determining M density variation maps based on the N digital images.

[0046] The density change map is used to characterize the position distribution and / or value distribution of the leopard spot density change of the fundus image to be processed, wherein M≤N-1, and M and N are both positive integers.

[0047] Exemplarily, by comparing N digital images, M density change maps can be determined. That is, according to the density values ​​of the digital images, a difference operation is performed to determine the M density change maps. It should be understood that in the output result, it can be selected as needed whether to output the N input fundus images to be processed. The doctor can further judge the density changes of different fundus images to be processed by combining the N input fundus images to be processed and the M density change maps, thereby improving the accuracy of diagnosis.

[0048] In some embodiments, the specific implementation method of step S220 is as follows: Figure 3 As shown, no further description is given here.

[0049] The image processing method provided by the embodiment of the present disclosure determines N digital images corresponding to the N fundus images to be processed based on the leopard spot areas of the N fundus images to be processed, and determines M density change maps based on the N digital images. In the embodiment of the present disclosure, M density change maps are determined through N digital images, and the density change maps represent the position distribution and / or numerical distribution of the leopard spot density changes of the fundus images to be processed, which can assist doctors in understanding the position distribution of the leopard spot density changes, more accurately determine the position changes of the leopard spot density, understand the patient's condition, and reduce misjudgment or misdiagnosis. The density change map can be a leopard spot density change map of two fundus images taken at adjacent times, or a leopard spot density change map of two fundus images taken at non-adjacent times, which can be determined according to the operator's choice.

[0050] Figure 3 FIG. 1 is a flow chart of determining M density change maps based on N digital images according to an embodiment of the present disclosure. Figure 3 As shown, the embodiment of the present disclosure provides a method for determining M density change maps based on N digital images, which includes the following steps.

[0051] Step S310 , calculating the difference between the digital image corresponding to the i-th fundus image to be processed and the digital image corresponding to the u-th fundus image to be processed, and determining the difference image corresponding to the i-th fundus image to be processed.

[0052] Exemplarily, according to the difference between the digital image corresponding to the i-th fundus image to be processed and the digital image corresponding to the u-th fundus image to be processed, a difference image is generated according to the difference matching in the corresponding area. Both the difference image and the digital image are two-dimensional images, and the numerical value in the difference image represents the difference between the digital image corresponding to the i-th fundus image to be processed and the digital image corresponding to the u-th fundus image to be processed.

[0053] Exemplarily, according to the requirements, after completing step S310, step S320 or step S330 is continued to be executed, or both steps are executed at the same time. The embodiment of the present disclosure does not further limit the specific circumstances of step S320 and step S330.

[0054] Step S320 , determining M density change maps based on the difference image corresponding to the i-th fundus image to be processed.

[0055] Exemplarily, based on the difference image corresponding to the i-th fundus image to be processed, data in the difference image corresponding to different colors are used to determine M density change maps. In practical applications, the colors can be data in the difference image corresponding to the same color, for example, data in the difference image corresponding to different brightness or contrast is used to determine M density change maps.

[0056] Step S330 , determining M density change maps based on the difference image corresponding to the i-th fundus image to be processed and the preset interval time.

[0057] The preset interval time is used to represent the interval time between the shooting time of the i-th fundus image to be processed and the shooting time of the u-th fundus image to be processed, wherein u<i and 2≤i≤N.

[0058] Exemplarily, based on the ratio of the difference in the difference image corresponding to the i-th fundus image to be processed and the preset interval time, the change of the difference within the preset interval time is determined, that is, the M density change graphs within the preset interval time are determined. Exemplarily, the preset interval time is selected as 6 months or 12 months, and the time can also be selected according to the needs. The embodiment of the present disclosure does not further limit the preset interval time.

[0059] The disclosed embodiment determines M density change maps based on the difference image corresponding to the i-th fundus image to be processed, or determines M density change maps based on the difference image corresponding to the i-th fundus image to be processed and a preset interval time. The disclosed embodiment can determine the density change map in two ways, so that the application scope of the image processing method is wider. In addition, the density change map determined by the difference image can directly characterize the position distribution of the density change, assist the doctor in determining the position distribution of the leopard spot density change, and assist the doctor in improving the accuracy of determining the fundus leopard spot changes, thereby achieving the purpose of timely and correct guidance on the patient's condition.

[0060] Figure 4 FIG. 1 is a flow chart of determining M density change maps based on the difference image corresponding to the i-th fundus image to be processed and the preset interval time provided by an embodiment of the present disclosure. Figure 4 As shown, the embodiment of the present disclosure provides a method for determining M density change maps based on a difference image corresponding to the i-th fundus image to be processed and a preset interval time, including the following steps.

[0061] Step S410: determining a visualized image corresponding to the M difference images based on the M difference images.

[0062] The visualized image includes several color features.

[0063] Exemplarily, the visualized image is a two-dimensional image, and the visualized image corresponding to the M difference images is determined by corresponding a number of color features to values ​​in different difference images.

[0064] Step S420: determining a density variation map based on the visualized image.

[0065] Several color features of the visualized image correspond to different leopard spot density changes in the density change map.

[0066] Exemplarily, based on the visualization image obtained above, a density change map is determined. It should be understood that when N is equal to 2, the number of input fundus images is 2, the value of M is 1, and the output density change map is 1. The different color features of the visualization image can characterize the density change, so that the visualization image is output as a density change map. By analogy, when N is greater than 2, M is a value greater than 1, and the number of output density change maps is M.

[0067] The disclosed embodiment determines the visualization image corresponding to the M-amplitude difference image based on the M-amplitude difference image, and determines the density change map based on the visualization image. The corresponding visualization image is determined based on the difference image, and the visualization image can show the changes in the leopard spot area of ​​different fundus images to be processed. The density change map determined based on the visualization image can make the output result more intuitive. When facing the density change map, doctors or other relevant personnel can intuitively understand the changes in the leopard spot area between different fundus images to be processed, and assist doctors to further improve the accuracy of diagnosis of leopard spot changes.

[0068] Figure 5 FIG. 1 is a flow chart of determining M density change maps based on the difference image corresponding to the i-th fundus image to be processed and the preset interval time provided by an embodiment of the present disclosure. Figure 5 As shown, the method provided in the embodiment of the present disclosure for determining M density change maps based on the difference image corresponding to the i-th fundus image to be processed and the preset interval time includes the following steps.

[0069] Step S510: determining a unit time variation graph corresponding to the M difference images based on the M difference images and a preset time threshold.

[0070] Exemplarily, based on the ratio of the difference of M amplitude difference images and the preset time threshold, the unit time change graph corresponding to the M amplitude difference images is determined. The unit time change graph is a two-dimensional visualization image, which characterizes the change speed distribution of the M amplitude difference images within the preset time interval through different color features. As described above, the selected preset time threshold can be selected according to demand. When the shooting time of the fundus images to be processed differs sufficiently, the selected preset time interval is 6 months or 12 months, and the effect of the unit time change graph is better. It should be understood that when the shooting time of the fundus images to be processed differs by less than 6 months or 12 months, the preset time interval should be selected according to demand, and the embodiments of the present disclosure do not further limit the preset time interval.

[0071] Step S520: Determine M density change graphs based on the unit time change graph.

[0072] Exemplarily, according to the above-obtained unit time variation graph, the selected preset interval time is marked or annotated, so that the unit time variation graph with the annotation or marking is determined as M density variation graphs. It should be understood that the annotation or marking of the preset interval time is optional. Without knowing the interval time, since the unit time variation graph is a two-dimensional visual image, it can be directly determined as a density variation graph based on the unit time variation graph. The present disclosure does not further limit the specific form of the density variation graph.

[0073] The image processing method provided by the embodiment of the present disclosure determines the unit time change graph corresponding to the M difference images based on the M difference images and the preset time threshold, and determines the M density change graphs based on the unit time change graphs. The M density change graphs are determined based on the unit time change graphs, and the density change graphs can represent the change trend over a period of time, and can assist doctors to better understand the density change of the leopard spot area of ​​the fundus image to be processed within the preset time threshold, thereby assisting doctors to improve the accuracy of determining the density change of the leopard spot area of ​​the fundus image to be processed within the preset time threshold.

[0074] Figure 6 FIG. 1 is a flow chart of an image processing method provided by another embodiment of the present disclosure. Figure 2 Based on the embodiment shown, Figure 6 The embodiment shown is described below in detail. Figure 6 The embodiment shown and Figure 2 The differences and similarities of the illustrated embodiments are not described in detail.

[0075] like Figure 6 As shown, another image processing method provided by another embodiment of the present disclosure further includes the following steps before determining N digital images corresponding to the N fundus images to be processed based on the leopard spot areas of the N fundus images to be processed.

[0076] Step S610: determining a reference image among the N fundus images to be processed.

[0077] Exemplarily, according to the time sequence of the fundus images to be processed, the fundus image to be processed with the earliest shooting time is determined as the reference image. Alternatively, according to the requirements, the fundus image to be processed with the best shooting quality is selected to be determined as the reference image.

[0078] Step S620: Based on the reference image, image registration is performed on the N-1 fundus images to be processed, and the registered images corresponding to the N-1 fundus images to be processed are determined.

[0079] Alternatively, based on the reference image, N fundus images to be processed are image registered to determine the registered images corresponding to the N fundus images to be processed. Registering the fundus images first can correct the subtle differences between the fundus images and reduce misjudgment caused by position deviation. Then, the fundus images taken at different times are extracted for feature recognition and comparison. The registered images can be superimposed or displayed side by side, so that doctors can intuitively see the changes in the structure of myopic fundus changes over time, which is convenient for identifying the development trend of the characteristics of myopic fundus changes. This can accurately compare the changes in fundus images at different times, and improve the accuracy of the generation of follow-up reports on myopic fundus changes.

[0080] Exemplarily, based on the reference image, image registration is performed on N-1 fundus images to be processed, using fixed fundus features as the basis for registration, such as the optic disc, blood vessels, vascular arches, or the distance from the optic disc to the macula as the reference to perform image registration on N-1 fundus images to be processed.

[0081] Exemplarily, using image registration technology, multiple fundus images to be processed can be registered so that the multiple fundus images to be processed can be compared with each other, providing a premise for unified comparison, thereby making subsequent image processing results more accurate.

[0082] The image processing method provided by the embodiment of the present disclosure can compare the positional distribution of leopard spot density changes in N fundus images to be processed by performing image registration on N-1 fundus images to be processed, thereby providing a prerequisite for assisting doctors in making comparisons and further improving the accuracy of image processing results.

[0083] Figure 7 FIG. 1 is a flow chart of an image processing method provided by another embodiment of the present disclosure. Figure 6 Based on the embodiment shown, Figure 7 The embodiment shown is described below in detail. Figure 6 The embodiment shown and Figure 7 The differences and similarities of the illustrated embodiments are not described in detail.

[0084] like Figure 7 As shown, another image processing method provided by another embodiment of the present disclosure further includes the following steps before determining a reference image among N fundus images to be processed.

[0085] Step S710 , preprocessing the N initial fundus images to be processed to determine preprocessed fundus images corresponding to the N initial fundus images to be processed.

[0086] The preprocessing includes at least one of normalization processing, enhancement processing and denoising processing.

[0087] Exemplarily, N initial fundus images to be processed are obtained, and the N initial fundus images to be processed are preprocessed to determine the fundus images to be processed. The preprocessing of the initial fundus images to be evaluated includes, but is not limited to, normalization, enhancement, and denoising. Exemplarily, the preprocessing of the initial fundus images to be evaluated may also include extracting a region of interest (ROI) from the initial fundus images to be processed, and performing image enhancement processing on the initial fundus images to be processed, including sharpening, filtering, smoothing, and the like, to finally obtain the fundus images to be processed. Alternatively, after extracting the ROI from the initial processed fundus images, the ROI of the fundus images to be processed is cropped, and then image enhancement, sharpening, filtering, smoothing, and the like are selected to obtain the preprocessed fundus images. The disclosed embodiments do not further limit the manner in which the initial fundus images to be processed are preprocessed.

[0088] Step S720: Determine N fundus images to be processed based on the pre-processed fundus images.

[0089] Exemplarily, based on the pre-processed fundus images obtained above, N pre-processed fundus images are determined as N fundus images to be processed.

[0090] The disclosed embodiment obtains N fundus images to be processed by preprocessing the fundus images to be processed, thereby making the fundus structure of the fundus images to be processed clearer, removing irrelevant information in the fundus images to be processed, determining the real information in the fundus images to be processed, enhancing the detectability of relevant information and simplifying the data to the maximum extent, thereby further ensuring the reliability of subsequent feature extraction, image segmentation, alignment and recognition of the fundus images to be processed.

[0091] Figure 8 FIG. 2 is a flow chart of a fundus image comparison method provided by an embodiment of the present disclosure. Figure 8 As shown, a fundus image comparison method provided by an embodiment of the present disclosure includes the following steps.

[0092] Step S810: determining a comparison reference image among the N fundus images to be processed.

[0093] Exemplarily, the N fundus images to be processed are multiple fundus images to be processed taken by the same patient at different times. The comparison reference images among the N fundus images to be processed are determined according to the order of the shooting time. Alternatively, the comparison reference images among the N fundus images to be processed are selected according to the requirements.

[0094] Step S820: determining a visualized image corresponding to the comparison reference image based on the leopard spot region of the comparison reference image.

[0095] Exemplarily, a leopard spot region of the comparison reference image is extracted, and the leopard spot density is determined according to the leopard spot region area of ​​the comparison reference image and the area of ​​the comparison reference image. Based on the leopard spot density, a visualized image corresponding to the comparison reference image is determined using a digital image.

[0096] Step S830: using an image processing method, process the N fundus images to be processed to determine M density change maps.

[0097] Exemplarily, the above-mentioned image processing method is used to process N fundus images to be processed, and M density change maps are determined.

[0098] Step S840: Determine the comparison results corresponding to the N-1 fundus images to be processed based on the visualized image corresponding to the comparison reference image and the M density change maps.

[0099] The N-1 fundus images to be processed are images obtained by removing the comparison reference image from the N fundus images to be processed, and the comparison result is used to characterize the image changes of the N fundus images to be processed.

[0100] Exemplarily, the M density change maps are compared with the visualized image corresponding to the comparison reference image, so as to determine the comparison results corresponding to the N-1 fundus images to be processed. It should be understood that the image comparison results can include the comparison results corresponding to the N-1 fundus images to be processed and the visualized image corresponding to the comparison reference image as required, so as to facilitate the doctor to compare the changes of the fundus images according to the visualized image corresponding to the comparison reference image, thereby assisting the doctor to determine the change results of the N fundus images to be processed.

[0101] The disclosed embodiment uses the above-mentioned image processing method to determine the comparison results corresponding to N-1 fundus images to be processed based on the visualization image corresponding to the comparison reference image and M density change maps. This can assist doctors in determining the change results of the N fundus images to be processed, so that the fundus images of the patient taken at different times can be compared, thereby assisting doctors in determining the change results of the N fundus images to be processed and reducing errors.

[0102] Fig. 9 FIG. 1 is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure. Fig. 9As shown, the image processing device 900 provided by the embodiment of the present disclosure includes a determination module 901 and a processing module 902. Specifically, the determination module 901 is used to determine N digital images corresponding to the N fundus images to be processed based on the leopard spot areas of the N fundus images to be processed, where N≥2; the processing module 902 is used to determine M density change maps based on the N digital images, and the density change maps are used to characterize the position distribution and / or value distribution of the leopard spot density changes of the fundus images to be processed, where M≤N-1.

[0103] In some embodiments, the processing module 902 is also used to calculate the difference between the digital image corresponding to the i-th fundus image to be processed and the digital image corresponding to the u-th fundus image to be processed, and determine the difference image corresponding to the i-th fundus image to be processed; determine M density change maps based on the difference image corresponding to the i-th fundus image to be processed; or determine M density change maps based on the difference image corresponding to the i-th fundus image to be processed and a preset interval time, wherein the preset interval time is used to represent the interval time between the shooting time of the i-th fundus image to be processed and the shooting time of the u-th fundus image to be processed, wherein u<i and 2≤i≤N.

[0104] In some embodiments, the processing module 902 is also used to determine, based on the M difference images, a visualization image corresponding to the M difference images, where the visualization image includes a number of color features; based on the visualization image, determine a density change map, where the density change map is used to characterize the difference between different densities, and a number of color features of the visualization image correspond to different values ​​in the density change map.

[0105] In some embodiments, the processing module 902 is further used to determine the unit time variation graphs corresponding to the M difference images based on the difference image corresponding to the i-th fundus image to be processed and a preset time threshold; and determine the M density variation graphs based on the unit time variation graphs.

[0106] In some embodiments, before determining the N digital images corresponding to the N fundus images to be processed based on the leopard spot areas of the N fundus images to be processed, the determination module 901 is also used to determine a reference image among the N fundus images to be processed; based on the reference image, image registration is performed on the N-1 fundus images to be processed to determine the registered images corresponding to the N-1 fundus images to be processed.

[0107] In some embodiments, before determining the reference image among N fundus images to be processed, the determination module 901 is also used to preprocess the N initial fundus images to be processed, and determine the preprocessed fundus images corresponding to the N initial fundus images to be processed, wherein the preprocessing includes at least one of normalization processing, enhancement processing, and denoising processing; based on the preprocessed fundus images, determine the N fundus images to be processed.

[0108] Fig.10 FIG. 1 is a schematic diagram of the structure of a fundus image comparison device provided by an embodiment of the present disclosure. Fig.10 As shown, the image processing device 1000 provided by the embodiment of the present disclosure includes a contrast determination module 1001, a visualization processing module 1002, a density change determination module 1003 and a contrast module 1004. Specifically, the contrast determination module 1001 is used to determine a contrast reference image in N fundus images to be processed; the visualization processing module 1002 is used to determine a visualization image corresponding to the contrast reference image based on the leopard spot area of ​​the contrast reference image; the density change determination module 1003 is used to process the N fundus images to be processed using the above-mentioned image processing method to determine M density change maps; the contrast module 1004 is used to determine a contrast result corresponding to the N-1 fundus images to be processed based on the visualization image corresponding to the contrast reference image and the M density change maps, wherein the N-1 fundus images to be processed are images obtained by removing the contrast reference image from the N fundus images to be processed, and the contrast result is used to characterize the image change of the N fundus images to be processed.

[0109] Fig.11 The figure is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure. The electronic device 1100 (the electronic device 1100 may be a computer device) includes a memory 1101, a processor 1102, a communication interface 1103 and a bus 1104. The memory 1101, the processor 1102 and the communication interface 1103 are connected to each other through the bus 1104.

[0110] The memory 1101 may be a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The memory 1101 may store a program. When the program stored in the memory 1101 is executed by the processor 1102, the processor 1102 and the communication interface 1103 are used to execute each step in the image processing method or the fundus image comparison method of the embodiment of the present disclosure.

[0111] The processor 1102 can adopt a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to implement the functions required to be performed by each unit of the image processing device or fundus image comparison device of the embodiment of the present disclosure.

[0112] The processor 1102 may also be an integrated circuit chip having signal processing capabilities. In the implementation process, each step of the image processing method or fundus image comparison method disclosed in the present invention may be completed by an integrated logic circuit of hardware or software instructions in the processor 1102. The above-mentioned processor 1102 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1101, and the processor 1102 reads the information in the memory 1101, and combines its hardware to complete the functions required to be performed by the units included in the image processing device or fundus image comparison device of the embodiment of the present disclosure, or executes the image processing method or fundus image comparison method of the embodiment of the present disclosure.

[0113] The communication interface 1103 uses a transceiver device such as, but not limited to, a transceiver to implement communication between the electronic device 1100 and other devices or a communication network. For example, the fundus image to be processed can be acquired through the communication interface 1103 .

[0114] The bus 1104 may include a path for transmitting information between various components of the electronic device 1100 (eg, the memory 1101 , the processor 1102 , and the communication interface 1103 ).

[0115] It should be noted that although Fig.11The electronic device 1100 shown only shows a memory, a processor, and a communication interface, but in the specific implementation process, those skilled in the art should understand that the electronic device 1100 also includes other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the electronic device 1100 may also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that the electronic device 1100 may also only include the devices necessary for implementing the embodiments of the present disclosure, and does not necessarily include Fig.11 All devices shown in .

[0116] An embodiment of the present disclosure provides a method for applying a density change map, including: applying the density change map formed by the image processing method mentioned above to ophthalmic diagnosis and analysis reports, medical research fields, image database construction fields, telemedicine fields, clinical trials fields, medical insurance claims fields, image processing algorithm development fields, education and training fields, or intelligent ophthalmic auxiliary diagnosis and treatment systems.

[0117] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0118] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0119] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0122] The embodiments of the present disclosure may also be a computer-readable storage medium, on which computer program instructions are stored, and the computer program instructions, when executed by a processor, enable the processor to execute the steps in the method according to various embodiments of the present disclosure described above in this specification. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in various embodiments of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk. The computer-readable storage medium may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.

[0123] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Based on leopard spot areas of N fundus images to be processed, determining N digital images corresponding to the N fundus images to be processed, where N≥2; Based on the N digital images, M density change maps are determined, and the density change maps are used to characterize the position distribution and / or value distribution of the leopard spot density changes of the fundus image to be processed, wherein M≤N-1.

2. The image processing method according to claim 1, characterized in that: The step of determining M density change maps based on the N digital images includes: Calculate the difference between the digital image corresponding to the i-th fundus image to be processed and the digital image corresponding to the u-th fundus image to be processed, and determine the difference image corresponding to the i-th fundus image to be processed; Determine the M density change maps based on the difference image corresponding to the i-th fundus image to be processed; or, Based on the difference image corresponding to the i-th fundus image to be processed and the preset interval time, the M density change maps are determined, wherein the preset interval time is used to represent the interval time between the shooting time of the i-th fundus image to be processed and the shooting time of the u-th fundus image to be processed, wherein u<i and 2≤i≤N.

3. The image processing method according to claim 2, characterized in that: The determining the M density change maps based on the difference image corresponding to the i-th fundus image to be processed includes: Based on the M difference images, determining a visualized image corresponding to the M difference images, wherein the visualized image includes a plurality of color features; Based on the visualized image, the density variation map is determined, and a number of color features of the visualized image correspond to different leopard spot density variations in the density variation map.

4. The image processing method according to claim 2, characterized in that: The determining the M density change maps based on the difference image corresponding to the i-th fundus image to be processed and the preset interval time includes: Determine a unit time variation graph corresponding to M difference images based on the difference image corresponding to the i-th fundus image to be processed and a preset interval time; Based on the unit time variation graph, the M density variation graphs are determined.

5. The image processing method according to any one of claims 1 to 4, characterized in that: Before determining N digital images corresponding to the N fundus images to be processed based on the leopard spot areas of the N fundus images to be processed, the method further includes: Determining a reference image among the N fundus images to be processed; Based on the reference image, image registration is performed on the N-1 fundus images to be processed to determine the registered images corresponding to the N-1 fundus images to be processed.

6. The image processing method according to claim 5, characterized in that: Before determining the reference image among the N fundus images to be processed, the method further includes: Preprocessing N initial fundus images to be processed to determine preprocessed fundus images corresponding to the N initial fundus images to be processed, wherein the preprocessing includes at least one of normalization processing, enhancement processing, and denoising processing; Based on the pre-processed fundus image, the N fundus images to be processed are determined.

7. A fundus image comparison method, characterized in that: include: Determine a comparison reference image among N fundus images to be processed; Based on the leopard spot area of ​​the comparison reference image, determining a visualized image corresponding to the comparison reference image; Using the image processing method described in any one of items 1 to 6 above, the N fundus images to be processed are processed to determine M density change maps; Based on the visualized image corresponding to the comparison reference image and the M density change maps, the comparison results corresponding to N-1 fundus images to be processed are determined, wherein the N-1 fundus images to be processed are images obtained by removing the comparison reference image from the N fundus images to be processed, and the comparison results are used to characterize the image changes of the N fundus images to be processed.

8. An image processing device, characterized in that: include: A determination module, configured to determine N digital images corresponding to the N fundus images to be processed based on leopard spot areas of the N fundus images to be processed, wherein N≥2; The processing module is used to determine M density change maps based on the N digital images, wherein the density change maps are used to characterize the position distribution and / or value distribution of the leopard spot density changes of the fundus image to be processed, wherein M<N.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor, The processor is used to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 7.

11. A density change map application method, characterized in that: include: The density change map formed by the image processing method as described in any one of claims 1 to 6 is applied to ophthalmic diagnosis and analysis reports, medical research fields, image database construction fields, telemedicine fields, clinical trials fields, medical insurance claims fields, image processing algorithm development fields, education and training fields, or intelligent ophthalmic auxiliary diagnosis and treatment systems.

Images