Image processing method and device, storage medium and electronic equipment

Abstract

The invention provides an image processing method and device, a storage medium and electronic equipment, and relates to the technical field of computers. The image processing method comprises the following steps: acquiring a plurality of eye fundus images corresponding to a plurality of periods of a target user; respectively extracting respective blood vessel regions of the plurality of fundus images; determining blood vessel feature information corresponding to respective blood vessel areas of the fundus images; determining blood vessel difference information among the plurality of fundus images based on the blood vessel feature information corresponding to the respective blood vessel areas of the plurality of fundus images; and according to a preset blood vessel difference display rule, generating a blood vessel change visual image corresponding to the target user according to the blood vessel difference information among the fundus images. The change condition of the fundus blood vessel can be clearly displayed by the blood vessel change visual image, so that a doctor can accurately diagnose a disease according to the blood vessel change visual image.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, device, storage medium and electronic device. Background Art

[0002] With the development of medical technology, when diagnosing diseases in patients, doctors often need to collect fundus images of patients and diagnose patients based on the fundus images. Moreover, when the patient comes for a follow-up visit, it is necessary to collect fundus images again and compare fundus images at different times to better judge the patient's condition.

[0003] However, since the changes in the fundus are relatively subtle, especially when the interval between follow-up visits is short, the fundus condition changes little and doctors can easily make misjudgments or misdiagnoses by simply observing the fundus images at different times with the naked eye. Summary of the invention

[0004] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide an image processing method, device, storage medium and electronic device.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, including: acquiring multiple fundus images corresponding to multiple periods of a target user; respectively extracting the vascular regions of the multiple fundus images; determining vascular feature information corresponding to the vascular regions of the multiple fundus images; based on the vascular feature information corresponding to the vascular regions of the multiple fundus images, determining vascular difference information between the multiple fundus images; and generating a vascular change visualization image corresponding to the target user based on the vascular difference information between the multiple fundus images according to preset vascular difference display rules.

[0006] In some embodiments, the vascular characteristic information includes curvature, and determining the vascular characteristic information corresponding to the vascular region of each of the multiple fundus images includes: extracting at least one detection line of the vascular region of each of the multiple fundus images, at least one detection line includes a center line and / or an edge line; determining a first position point on at least one detection line of the vascular region of each of the multiple fundus images; based on the first position point on at least one detection line of the vascular region of each of the multiple fundus images, determining the curvature corresponding to the vascular region of each of the multiple fundus images.

[0007] In some embodiments, based on the vascular feature information corresponding to the vascular areas of each of the multiple fundus images, vascular difference information between the multiple fundus images is determined, including: for the same first position point, comparing the change difference of the curvature determined based on the same first position point in at least two of the multiple fundus images; based on the obtained change difference of the curvature, determining the vascular difference information between the multiple fundus images.

[0008] In some embodiments, the preset vascular difference display rules include mapping rules between vascular curvature change differences and marking colors; according to the preset vascular difference display rules, the vascular difference information between multiple fundus images is used to generate a vascular change visualization image corresponding to the target user, including: based on the mapping rules between vascular curvature change differences and marking colors, determining the marking colors corresponding to each of the obtained change differences; based on the marking colors corresponding to each of the obtained change differences, generating a vascular change visualization image corresponding to the target user.

[0009] In some embodiments, the vascular characteristic information includes the vascular diameter of the target position area, and the vascular difference information between the multiple fundus images is determined based on the vascular characteristic information corresponding to the vascular areas of each of the multiple fundus images, including: for the same target position area, comparing the change difference of the vascular diameter of at least two fundus images in the target position area; based on the obtained change difference of the vascular diameter, determining the vascular difference information between the multiple fundus images.

[0010] In some embodiments, preset vascular difference display rules include mapping rules between vascular diameter change differences and marking methods; according to the preset vascular difference display rules, vascular difference information between multiple fundus images is used to generate a vascular change visualization image corresponding to the target user, including: based on the mapping rules between vascular diameter change differences and marking methods, determining the marking methods corresponding to the obtained vascular diameter change differences; based on the marking methods corresponding to the obtained vascular diameter change differences, generating a vascular change visualization image corresponding to the target user.

[0011] In some embodiments, the marking method includes a marking color, and based on the marking methods corresponding to the obtained differences in the changes in the blood vessel diameters, a visualization image of the blood vessel changes corresponding to the target user is generated, including: in the target position area in any fundus image of at least two fundus images, the marking colors corresponding to the changes in the blood vessel diameters of the target position area are displayed to obtain a visualization image of the blood vessel changes corresponding to the target user.

[0012] In some embodiments, after respectively extracting the vascular areas of multiple fundus images, the method also includes: for two fundus images among the multiple fundus images, displaying the vascular area of ​​one of the two fundus images with a first preset color to obtain a first display image; and displaying the vascular area of ​​the other of the two fundus images with a second preset color on the first display image to obtain a second display image; wherein the first preset color is different from the second preset color.

[0013] In a second aspect, an embodiment of the present application provides an image processing device, comprising: an acquisition module, used to acquire multiple fundus images corresponding to multiple periods of a target user; an extraction module, used to respectively extract the vascular regions of each of the multiple fundus images; a first determination module, used to determine the vascular feature information corresponding to the vascular regions of each of the multiple fundus images; a second determination module, used to determine the vascular difference information between the multiple fundus images based on the vascular feature information corresponding to the vascular regions of each of the multiple fundus images; and a generation module, used to generate a visualization image of vascular changes corresponding to the target user based on the vascular difference information between the multiple fundus images according to a preset vascular difference display rule.

[0014] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is used to execute the method described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; and the processor is used to execute the method described in the first aspect.

[0016] The image processing method proposed in the embodiment of the present application can obtain multiple fundus images corresponding to multiple periods of the target user; extract the vascular regions of each of the multiple fundus images respectively; determine the vascular feature information corresponding to the vascular regions of each of the multiple fundus images; determine the vascular difference information between the multiple fundus images based on the vascular feature information corresponding to the vascular regions of each of the multiple fundus images; and generate a vascular change visualization image corresponding to the target user based on the vascular difference information between the multiple fundus images according to the preset vascular difference display rules. Since the vascular change visualization image can clearly show the changes in the fundus blood vessels, doctors can accurately diagnose diseases based on the vascular change visualization image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 Shown is a schematic diagram of a scenario applicable to an exemplary embodiment of the present application.

[0019] Figure 2 Shown is a flowchart of an image processing method provided by an exemplary embodiment of the present application.

[0020] Figure 3 Shown is a schematic flow chart of a method for determining curvature provided by an exemplary embodiment of the present application.

[0021] Figure 4 Shown is a flow chart of a method for determining blood vessel difference information provided by an exemplary embodiment of the present application.

[0022] Figure 5 Shown is a schematic flow chart of a method for generating a visualization image of vascular changes provided by an exemplary embodiment of the present application.

[0023] Figure 6 Shown is a schematic flow chart of a method for determining blood vessel difference information provided by another exemplary embodiment of the present application.

[0024] Figure 7 Shown is a schematic flow chart of a method for generating a visualization image of vascular changes provided by another exemplary embodiment of the present application.

[0025] Figure 8 Shown is a schematic diagram of a process for generating a visualization image of vascular changes provided by yet another exemplary embodiment of the present application.

[0026] Fig. 9 Shown is a schematic structural diagram of an image processing device provided by an exemplary embodiment of the present application.

[0027] Fig.10 Shown is a schematic structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] Application Overview

[0030] Fundus images are the only images of the human body that can be used to observe capillary changes simply, non-invasively, and clearly. Currently, there are more than 60 fundus diseases or systemic chronic diseases (such as glaucoma, diabetic retinopathy, etc.) that will cause corresponding lesions in the fundus, so fundus images are usually an important reference for clinical diagnosis.

[0031] In the related art, a fundus camera will take a fundus image separately during diagnosis. Then, when the same person is re-examined after a period of time, the fundus image needs to be re-taken. The doctor will then compare the fundus images taken at different times to better judge the patient's condition.

[0032] However, it is difficult for doctors to visually see the difference between some subtle changes in fundus images, or changes in fundus images taken at short intervals (fundus changes are smaller at short intervals). Even if they can see the difference, doctors can easily make misjudgments or misdiagnoses based on their subjective judgment.

[0033] In view of this, the present application proposes an image processing method, which can obtain multiple fundus images corresponding to multiple periods of the target user; extract the vascular regions of each of the multiple fundus images respectively; determine the vascular feature information corresponding to the vascular regions of each of the multiple fundus images; determine the vascular difference information between the multiple fundus images based on the vascular feature information corresponding to the vascular regions of each of the multiple fundus images; and generate a vascular change visualization image corresponding to the target user based on the vascular difference information between the multiple fundus images according to the preset vascular difference display rules. Since the vascular change visualization image can clearly show the changes in the fundus blood vessels, doctors can accurately diagnose diseases based on the vascular change visualization image.

[0034] Example scenarios

[0035] The image processing method proposed in the embodiment of the present application can be executed by an electronic device, which can be a terminal, such as a smart phone, a tablet computer, a desktop computer, etc., or the electronic device can also be a server.

[0036] Figure 1 Shown is a schematic diagram of an application scenario of an image processing method provided by an exemplary embodiment of the present application. Figure 1 The illustrated scenario includes: a server 101 and a user terminal 102, and the server 101 is in communication connection with the user terminal 102. The user terminal 102 includes, but is not limited to: computer terminals such as desktop computers and notebook computers, and mobile terminals such as tablet computers and mobile phones.

[0037] In one application scenario, a user sends an image processing instruction to the server 101 through the user terminal 102. After receiving the image processing instruction, the server 101 can obtain multiple fundus images corresponding to multiple periods of the target user, and generate a visualization image of the vascular changes corresponding to the target user, and then send the visualization image of the vascular changes to the user terminal 102 for display.

[0038] Specifically, when the server 101 obtains multiple fundus images corresponding to multiple periods of the target user, the server 101 may obtain the fundus images from a storage device, and the fundus images stored in the storage device may be taken by a fundus camera and stored in the storage device.

[0039] When generating a visualization image of vascular changes corresponding to a target user, server 101 may perform the following steps: extracting the vascular regions of each of the multiple fundus images respectively; determining vascular feature information corresponding to the vascular regions of each of the multiple fundus images; determining vascular difference information between the multiple fundus images based on the vascular feature information corresponding to the vascular regions of each of the multiple fundus images; and generating a visualization image of vascular changes corresponding to the target user using the vascular difference information between the multiple fundus images according to a preset vascular difference display rule.

[0040] Exemplary Methods

[0041] See also Figure 2 As shown in FIG. 1 , a flow chart of an image processing method provided by an exemplary embodiment of the present application is shown. Figure 2 As shown, the image processing method provided in the embodiment of the present application includes the following steps 201 to 205.

[0042] Step 201: Acquire multiple fundus images corresponding to multiple periods of a target user.

[0043] Among them, the fundus image can be an image of the human eye taken by a fundus camera, such as a color fundus photograph, optical coherence tomography (OCT), and fluorescein fundus angiography (FFA). Fundus images include but are not limited to all image data saved by the medical institution during the treatment of the target user, relevant fundus image data input by the target user, and fundus image data taken by the target user as needed during the medical consultation. In addition, the fundus image can be a 45° fundus image, a 60° fundus image, a wide-angle fundus image, a fundus image with other field of view angles, or even a fundus image of other modalities. It should be noted that the fundus image can be an image taken with the optic disc as the center, or an image taken with the center of the macula as the center.

[0044] Step 202: extract the blood vessel regions of the multiple fundus images respectively.

[0045] Specifically, the vascular regions of each of the multiple fundus images can be extracted using a preset vascular region extraction algorithm or a pre-trained fundus vascular region extraction model. The fundus vascular region extraction model can be trained based on multiple training images with vascular region position markers.

[0046] Step 203: Determine the blood vessel characteristic information corresponding to the blood vessel regions of each of the plurality of fundus images.

[0047] The blood vessel region of any fundus image includes: regions corresponding to multiple blood vessels. The blood vessel characteristic information may exemplarily include: the curvature and / or the blood vessel diameter corresponding to the blood vessel region.

[0048] Step 204 : determining the vascular difference information between the multiple fundus images based on the vascular feature information corresponding to the vascular regions of the multiple fundus images.

[0049] The blood vessel difference information represents: difference data of blood vessel characteristic information corresponding to blood vessel regions in fundus images of multiple periods.

[0050] In some embodiments, after executing step 201 and before executing step 204, multiple fundus images may be registered first, so that when executing step 204, the difference data between corresponding areas, key points and lines in multiple fundus images can be accurately determined. Based on the inherent fundus features of each fundus image of multiple periods, image registration is performed on fundus images of multiple periods. The inherent fundus features include one or more of the following features: optic disc, macula, blood vessels, vascular arches, and the distance from the optic disc to the macula. The inherent fundus features may include anatomical structures or markers that exist naturally and are relatively stable in the fundus. The above-listed features are obvious and recognizable features in the fundus, and therefore can be used as a reference for image registration. The specific registration steps can be adjusted by those skilled in the art according to the registration method in the prior art, and this embodiment will not be repeated. Registering the fundus images first can correct the subtle differences between the images and reduce misjudgment caused by position deviation. The fundus image features of each period can then be extracted for comparison. The registered images can be superimposed or displayed side by side, allowing doctors to intuitively see the changes in the fundus structure over time and facilitate the identification of the development trend of fundus features. This makes it possible to accurately compare the changes in fundus images at different periods, thereby improving the accuracy of fundus follow-up reports.

[0051] Step 205 : Generate a vascular change visualization image corresponding to the target user based on the vascular difference information between the multiple fundus images according to the preset vascular difference display rules.

[0052] Among them, the visualization image of vascular changes can intuitively and accurately display the characteristic changes of vascular areas in fundus images of multiple periods, so that doctors can make more accurate diagnoses based on the visualization image of vascular changes.

[0053] Figure 3 Shown is a schematic flow chart of a method for determining curvature provided by an exemplary embodiment of the present application.

[0054] In some embodiments, Figure 3As shown, the blood vessel characteristic information includes curvature. Determining the blood vessel characteristic information corresponding to the blood vessel regions of each of the plurality of fundus images includes the following steps 301 to 303 .

[0055] Step 301: extract at least one detection line of a blood vessel region of each of a plurality of fundus images.

[0056] Wherein, at least one detection line includes a center line (i.e., a line segment representing the center line of a blood vessel) and / or an edge line (i.e., a line segment representing the edge of a blood vessel). Specifically, when determining the center line of any blood vessel region, the center line of the blood vessel region may be determined based on any blood vessel center line extraction algorithm or blood vessel center line extraction model, and when determining the edge line of any blood vessel region, it may be determined based on edge pixel points of the blood vessel region. The embodiment of the present application does not limit the method for determining the center line and / or edge line.

[0057] Step 302: Determine a first position point on at least one detection line of a blood vessel region of each of the plurality of fundus images.

[0058] The first position point may be each pixel point on the detection line; or the first position point may be selected from a plurality of pixel points on the detection line. For example, a pixel point every preset number of pixel points is selected as the first position point, such as selecting a pixel point every 5 pixel points as the first position point.

[0059] Step 303 : Determine the curvature corresponding to the blood vessel regions of the plurality of fundus images based on the first position point on at least one detection line of the blood vessel regions of the plurality of fundus images.

[0060] In some embodiments, when determining the curvature corresponding to the vascular region of each of the multiple fundus images based on the first position point on at least one detection line of the vascular region of each of the multiple fundus images, the second position point corresponding to the vascular region of each of the multiple fundus images can be first determined based on the first position point on at least one detection line of the vascular region of each of the multiple fundus images, and then the curvature of the second position point corresponding to the vascular region of each of the multiple fundus images is determined based on the second position point corresponding to the vascular region of each of the multiple fundus images and a preset position point curvature algorithm, and the curvature of the second position point corresponding to the vascular region of each of the multiple fundus images is used as the curvature corresponding to the vascular region of each of the multiple fundus images.

[0061] In some embodiments, when determining the second position point corresponding to the vascular region of each of the multiple fundus images based on the first position point on at least one detection line of the vascular region of each of the multiple fundus images, for any vascular region, when there is only one detection line, the first position point on the detection line can be used as the second position point corresponding to the vascular region; when there are at least two detection lines (such as at least two detection lines including a center line and two edge lines), for any first position point on any detection line, the first position point corresponding to the first position point on other detection lines of the at least two detection lines except the detection line can be determined, and then based on the first position point and the first position point corresponding to the first position point, the second position point corresponding to the vascular region is determined by a preset second position point algorithm (such as an algorithm for calculating the average, median, maximum or minimum value).

[0062] In other embodiments, when determining the curvature corresponding to the vascular region of each of the multiple fundus images based on the first position point on at least one detection line of the vascular region of each of the multiple fundus images, the curvature of the first position point on at least one detection line of the vascular region of each of the multiple fundus images can be first determined based on a preset position point curvature algorithm, and then the curvature corresponding to the vascular region of each of the multiple fundus images can be determined based on the curvature of the first position point on at least one detection line of the vascular region of each of the multiple fundus images.

[0063] Specifically, when determining the curvature corresponding to the blood vessel regions of the respective fundus images based on the curvature of the first position point on at least one detection line of the blood vessel regions of the respective fundus images, for any blood vessel region, in the case of only one detection line, the curvature of the first position point on the detection line can be used as the curvature corresponding to the blood vessel region. In the case of at least two detection lines, for any first position point on any detection line, a first position point corresponding to the first position point on other detection lines other than the detection line among at least two detection lines can be determined, and the first curvature of the first position point and the second curvature of the first position point corresponding to the first position point can be determined based on a preset position point curvature algorithm, and then based on the first curvature of the first position point and the second curvature of the first position point corresponding to the first position point, a third curvature is determined by a preset blood vessel region curvature algorithm (such as an algorithm for calculating the average, median, maximum or minimum value, etc.), and then multiple third curvatures calculated based on the multiple first position points on the detection line are used as the curvature corresponding to the blood vessel region, and each third curvature is used to indicate the curvature degree of a partial area of ​​the blood vessel region.

[0064] In the embodiment of the present application, since the center line and / or edge line of the blood vessel area can characterize the direction of the blood vessel, by determining the curvature corresponding to the blood vessel area, the curvature change of the target user's fundus blood vessels can be accurately determined based on the curvature, thereby assisting doctors in diagnosis.

[0065] Figure 4 Shown is a flow chart of a method for determining blood vessel difference information provided by an exemplary embodiment of the present application.

[0066] In some embodiments, Figure 4 As shown, based on the vascular feature information corresponding to the vascular regions of each of the multiple fundus images, the vascular difference information between the multiple fundus images is determined, including the following steps 401-402.

[0067] Step 401 : for a same first position point, comparing the change difference of curvature determined based on the same first position point in at least two fundus images among a plurality of fundus images.

[0068] The same first position point is used to indicate a position point at the same blood vessel site in different fundus images.

[0069] Exemplarily, when determining the difference in change of curvature determined based on the same first position point in at least two fundus images, the curvatures determined based on the same first position point in at least two fundus images can be subtracted to obtain the difference in change of curvature; or, the percentage difference in curvature determined based on the same first position point in at least two fundus images can be used as the difference in change of curvature, wherein the percentage difference is the ratio of the difference in curvature determined based on the same first position point in at least two fundus images to the curvature determined based on the first position point in one of the fundus images.

[0070] Step 402: Determine blood vessel difference information between multiple fundus images based on the obtained curvature change difference.

[0071] The blood vessel difference information between the multiple fundus images may include: the obtained curvature change difference value, and / or the difference analysis data calculated based on the obtained curvature change difference value.

[0072] In some embodiments, the preset blood vessel difference display rules include a mapping rule between the blood vessel curvature change difference and the marking color. The mapping rule may exemplarily include a mapping relationship between multiple blood vessel curvature change difference ranges and multiple color values, such as -1 to 1 corresponds to green, 2 to 3 corresponds to orange, and 3 to 4 corresponds to red.

[0073] Figure 5 Shown is a schematic flow chart of a method for generating a visualization image of vascular changes provided by an exemplary embodiment of the present application.

[0074] In some embodiments, Figure 5 As shown, when the vascular difference information between multiple fundus images is used to generate a vascular change visualization image corresponding to a target user according to a preset vascular difference display rule, the following steps 501 to 502 are included.

[0075] Step 501: Based on the mapping rule between the vascular tortuosity change difference and the mark color, determine the mark color corresponding to each of the obtained change difference.

[0076] Specifically, the vascular curvature change difference range to which the vascular curvature change difference belongs can be determined first, and then the mark color corresponding to the vascular curvature change difference range can be determined according to the mapping rule between the vascular curvature change difference and the mark color. The mark color can be illustratively RGB (Red Green Blue) color, HSV (Hue Saturation Value) color, etc.

[0077] Exemplarily, if Kn represents the curvature change difference (where n is a positive integer), the mapping rules between the blood vessel curvature change difference and the marking color include: K1~K2 correspond to color 1, K2~K3 correspond to color 2, K3~K4 correspond to color 3, and if the obtained change difference belongs to K2~K3, the marking color corresponding to the obtained change difference is color 2.

[0078] Step 502: Generate a blood vessel change visualization image corresponding to the target user based on the mark colors corresponding to the obtained change differences.

[0079] Specifically, for any first position point in any vascular area, the mark color corresponding to the difference in curvature change determined based on the first position point can be displayed on the area corresponding to the first position point in the vascular area in the fundus image (such as the latest fundus image), and so on. The multiple mark colors in step 501 are displayed in the corresponding areas of the fundus image according to this method, thereby generating a visualization image of vascular changes.

[0080] The vascular change visualization image can accurately and intuitively display the changes in the curvature of each area in the blood vessel, thereby assisting doctors in making diagnoses and improving the accuracy and efficiency of doctors' diagnoses. In a specific example, the larger the change difference, the darker the corresponding mark color. In this way, the changes in the curvature of the blood vessel area can be seen based on the color depth of each area in the vascular change visualization image.

[0081] In some embodiments, the obtained change difference may also be displayed on the above-mentioned vascular change visualization image.

[0082] Figure 6 Shown is a schematic flow chart of a method for determining blood vessel difference information provided by another exemplary embodiment of the present application.

[0083] In some embodiments, the blood vessel characteristic information includes the blood vessel diameter in the target location area; Figure 6 As shown, based on the vascular feature information corresponding to the vascular regions of each of the multiple fundus images, the vascular difference information between the multiple fundus images is determined, including the following steps 601 to 602.

[0084] Step 601 : for the same target position area, comparing the difference in the change of the blood vessel diameters in the target position area of ​​at least two fundus images among the plurality of fundus images.

[0085] Specifically, the target location region may represent the region where any blood vessel is located, or may be a local region of any blood vessel. When determining the vessel diameter of the target location region, the vessel diameter of the target location region may be, for example, the average value of the lengths of line segments in multiple target location regions that are perpendicular to the extending direction of the blood vessel and respectively pass through multiple target location points (such as each pixel point) in the extending direction of the blood vessel.

[0086] The difference in change of the blood vessel diameters of at least two fundus images in the target position area may be: the difference (obtained by subtracting) of the blood vessel diameters of at least two fundus images in the target position area, or may be the percentage difference of the blood vessel diameters of at least two fundus images in the target position area (i.e., the ratio of the difference in the blood vessel diameters of at least two fundus images in the target position area to the blood vessel diameter of any fundus image in the target position area).

[0087] Step 602: Determine blood vessel difference information between multiple fundus images based on the obtained blood vessel diameter change difference.

[0088] The blood vessel difference information between the multiple fundus images may include: the obtained blood vessel diameter change difference value, and / or the difference analysis data calculated based on the obtained blood vessel diameter change difference value.

[0089] In some embodiments, the preset vascular difference display rules include a mapping rule between vascular diameter change differences and marking methods. The mapping rules may exemplarily include a mapping relationship between multiple vascular diameter change difference ranges and multiple marking methods. The marking method may include at least one of marking color, marking font, marking symbol, and marking size. Exemplarily, -3 to -1 correspond to vascular diameter change difference marks with green marking color and green font, -1 to 1 correspond to vascular diameter change difference marks with yellow marking color and yellow font, and 1 to 3 correspond to vascular diameter change difference marks with red marking color and red font.

[0090] Figure 7 Shown is a schematic flow chart of a method for generating a visualization image of vascular changes provided by another exemplary embodiment of the present application.

[0091] In some embodiments, Figure 7 As shown, according to the preset vascular difference display rules, the vascular difference information between multiple fundus images is used to generate a vascular change visualization image corresponding to the target user, including the following steps 701 to 702.

[0092] Step 701: Based on the mapping rule between the blood vessel diameter change difference and the marking method, determine the marking methods corresponding to the obtained blood vessel diameter change difference.

[0093] Specifically, the vascular diameter change difference range to which the obtained vascular diameter change difference belongs can be determined first, and then the marking method corresponding to the vascular diameter change difference range can be determined according to the mapping rule between the vascular diameter change difference and the marking method.

[0094] Exemplarily, if Dn represents the curvature change difference (where n is a positive integer), the mapping rules between the blood vessel diameter change difference and the marking method include: D1~D2 correspond to marking method 1, D2~D3 correspond to marking method 2, and D3~D4 correspond to marking method 3. If the obtained change difference belongs to D3~D4, the marking method corresponding to the obtained blood vessel diameter change difference is marking method 3.

[0095] Step 702: Generate a visualization image of the blood vessel changes corresponding to the target user based on the labeling methods corresponding to the obtained differences in the changes in the blood vessel diameters.

[0096] The vascular change visualization image can accurately and intuitively display the changes in the diameter of the blood vessels, thereby assisting doctors in making diagnoses and improving the accuracy and efficiency of doctors' diagnoses. In a specific example, the larger the difference in the change in the vascular diameter obtained, the darker the corresponding mark color. In this way, the changes in the diameter of the vascular area can be seen based on the color depth of each area in the vascular change visualization image.

[0097] Figure 8 Shown is a schematic diagram of a process for generating a visualization image of vascular changes provided by yet another exemplary embodiment of the present application.

[0098] In some embodiments, Figure 8 As shown, the marking method includes a marking color, and based on the marking methods corresponding to the obtained blood vessel diameter change differences, a blood vessel change visualization image corresponding to the target user is generated, including the following steps 801.

[0099] Step 801: In a target position area in any one of at least two fundus images, the marking colors corresponding to the difference in the change of the blood vessel diameter in the target position area are displayed to obtain a visualized image of the blood vessel change corresponding to the target user.

[0100] The marking color may be RGB color, HSV color, etc., for example.

[0101] For example, area 1 corresponds to marking color 1, area 2 corresponds to marking color 2, and area 3 corresponds to marking color 3. Then marking color 1, marking color 2, and marking color 3 can be displayed in area 1, area 2, and area 3 respectively.

[0102] Exemplarily, the target position area may include an area with increased diameter where the difference in blood vessel diameter change is greater than 0 and an area with decreased diameter where the difference in blood vessel diameter change is less than 0. The area with increased diameter and the area with decreased diameter may be marked with different colors, such as blue and red respectively.

[0103] Through this method, the changes in blood vessel diameter can be clearly displayed, thereby assisting doctors in making diagnoses and improving the accuracy and efficiency of doctors' diagnoses.

[0104] Furthermore, the method of the present application can be used to mark the change in blood vessel curvature with a first color, and the change in blood vessel diameter with a second color. Different transparencies or different brightness of the same color represent the degree of change. This method can provide a comprehensive and visual display of the changes in fundus blood vessels, making it easier for doctors and patients to clearly understand the changes in the fundus.

[0105] As an example, obtain any pixel point on the blood vessel of the registered fundus image, obtain the original grayscale value corresponding to the pixel point: determine the curvature change value and diameter change value corresponding to the pixel point; replace the original grayscale value based on the curvature change value and diameter change value corresponding to the pixel point to generate a replacement result; render the blood vessel based on the replacement result of each pixel point on the blood vessel to generate a blood vessel follow-up change image; perform visualization based on the registered fundus image blood vessel area and blood vessel follow-up change image, and generate a blood vessel bend follow-up change heat map according to the preset mapping rule, that is, the decrease in blood vessel curvature change corresponds to the first color, the increase in blood vessel curvature change corresponds to the second color, the increase in blood vessel diameter corresponds to the third color, and the decrease in blood vessel diameter corresponds to the fourth color. This method can not only observe the original appearance of the blood vessel, but also see the follow-up changes of the blood vessel, so that doctors and patients can understand clearly and intuitively understand the progress of the disease.

[0106] In some embodiments, after extracting the vascular regions of the multiple fundus images respectively, the vascular region of one of the two fundus images may be displayed in a first preset color for two fundus images among the multiple fundus images to obtain a first display image; and then the vascular region of the other of the two fundus images may be displayed in a second preset color on the first display image to obtain a second display image. The first preset color is different from the second preset color.

[0107] Exemplarily, if at least two fundus images include a first fundus image taken earlier and a second fundus image taken later, the vascular area of ​​the first fundus image can be first displayed in a first preset color to obtain a first display image, and then the vascular area of ​​the second fundus image can be displayed in a second preset color on the first display image to obtain a second display image, wherein at least part of the vascular area of ​​the first fundus image displayed in the first preset color (i.e., the overlapping area of ​​the vascular areas of the two fundus images) will be covered by the vascular area of ​​the second fundus image displayed in the second preset color, and the uncovered part of the vascular area of ​​the first fundus image uses the first preset color which is different from the second preset color, thereby clearly displaying the vascular narrowing of the target user.

[0108] Through this method, the narrowing of blood vessels can be clearly demonstrated, thereby assisting doctors in diagnosis and improving the accuracy and efficiency of doctors' diagnosis.

[0109] In the above embodiment, multiple fundus images corresponding to multiple periods of the target user can be obtained; the vascular regions of each of the multiple fundus images can be extracted respectively; the vascular feature information corresponding to the vascular regions of each of the multiple fundus images can be determined; based on the vascular feature information corresponding to the vascular regions of each of the multiple fundus images, the vascular difference information between the multiple fundus images can be determined; according to the preset vascular difference display rules, the vascular difference information between the multiple fundus images can be used to generate a vascular change visualization image corresponding to the target user. Since the vascular change visualization image can clearly show the changes in the fundus blood vessels, doctors can accurately diagnose diseases based on the vascular change visualization image.

[0110] Exemplary Devices

[0111] Combination of the above Figures 2 to 8 , describes the method embodiment of the present application in detail, and the following is combined with Fig. 9 , describes the device embodiment of the present application in detail. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the parts not described in detail can refer to the previous method embodiment.

[0112] Fig. 9FIG. 1 is a schematic diagram of the structure of an image processing device provided by an exemplary embodiment of the present application. Fig. 9 As shown, the image processing device 900 provided in the embodiment of the present application includes: an acquisition module 901, used to acquire multiple fundus images corresponding to multiple periods of the target user; an extraction module 902, used to respectively extract the vascular regions of the multiple fundus images; a first determination module 903, used to determine the vascular feature information corresponding to the vascular regions of the multiple fundus images; a second determination module 904, used to determine the vascular difference information between the multiple fundus images based on the vascular feature information corresponding to the vascular regions of the multiple fundus images; a generation module 905, used to generate a vascular change visualization image corresponding to the target user based on the vascular difference information between the multiple fundus images according to a preset vascular difference display rule.

[0113] In some embodiments, the vascular feature information includes curvature, and the first determination module 903 is further used to: respectively extract at least one detection line of the vascular region of each of the multiple fundus images, at least one detection line including a center line and / or an edge line; respectively determine a first position point on at least one detection line of the vascular region of each of the multiple fundus images; based on the first position point on at least one detection line of the vascular region of each of the multiple fundus images, determine the curvature corresponding to the vascular region of each of the multiple fundus images.

[0114] In some embodiments, the second determination module 904 is also used to: for the same first position point, compare the change difference of curvature determined based on the same first position point in at least two fundus images among the multiple fundus images; and determine the vascular difference information between the multiple fundus images based on the obtained change difference of curvature.

[0115] In some embodiments, the preset blood vessel difference display rules include mapping rules between blood vessel curvature change differences and marking colors, and the generation module 905 is further used to: determine the marking colors corresponding to the obtained change differences based on the mapping rules between blood vessel curvature change differences and marking colors; and generate a blood vessel change visualization image corresponding to the target user based on the marking colors corresponding to the obtained change differences.

[0116] In some embodiments, the vascular feature information includes the vascular diameter of the target position area, and the second determination module 904 is further used to: for the same target position area, compare the change difference of the vascular diameter in the target position area of ​​at least two fundus images among the multiple fundus images; based on the obtained change difference of the vascular diameter, determine the vascular difference information between the multiple fundus images.

[0117] In some embodiments, the preset blood vessel difference display rules include mapping rules between blood vessel diameter change differences and marking methods, and the generation module 905 is also used to: determine the marking methods corresponding to the obtained blood vessel diameter change differences based on the mapping rules between the blood vessel diameter change differences and the marking methods; and generate a blood vessel change visualization image corresponding to the target user based on the marking methods corresponding to the obtained blood vessel diameter change differences.

[0118] In some embodiments, the marking method includes marking color, and the generation module 905 is also used to: display the marking color corresponding to the change difference of the blood vessel diameter in the target position area in any fundus image of at least two fundus images, and obtain a visualization image of the blood vessel changes corresponding to the target user.

[0119] In some embodiments, after respectively extracting the vascular areas of each of the multiple fundus images, the image processing device 900 is also used to: for two fundus images among the multiple fundus images, display the vascular area of ​​one of the two fundus images with a first preset color to obtain a first display image; and display the vascular area of ​​the other of the two fundus images with a second preset color on the first display image to obtain a second display image; wherein the first preset color is different from the second preset color.

[0120] Below, reference Fig.10 To describe an electronic device according to an embodiment of the present application. Fig.10 Shown is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application.

[0121] like Fig.10 As shown, the electronic device 1000 includes one or more processors 1001 and a memory 1002 .

[0122] The processor 1001 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1000 to perform desired functions.

[0123] The memory 1002 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1001 may run the program instructions to implement the image processing methods of the various embodiments of the present application described above and / or other desired functions. Various contents such as fundus images, vascular regions, vascular feature information, vascular difference information, and vascular change visualization images may also be stored in the computer-readable storage medium.

[0124] In one example, the electronic device 1000 may further include: an input device 1003 and an output device 1004 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0125] The input device 1003 may include, for example, a keyboard, a mouse, etc.

[0126] The output device 1004 can output various information to the outside, including fundus images, vascular regions, vascular feature information, vascular difference information, vascular change visualization images, etc. The output device 1004 can include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0127] Of course, to simplify, Fig.10 Only some of the components related to the present application in the electronic device 1000 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 1000 may also include any other appropriate components.

[0128] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the image processing method according to various embodiments of the present application described above in this specification.

[0129] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0130] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the image processing method according to various embodiments of the present application described above in this specification.

[0131] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0132] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0133] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0134] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0135] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0136] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. An image processing method, characterized in that: include: Acquire multiple fundus images corresponding to multiple periods of the target user; respectively extracting blood vessel regions from the plurality of fundus images; Determining blood vessel characteristic information corresponding to the blood vessel regions of each of the plurality of fundus images; Determining vascular difference information between the multiple fundus images based on vascular feature information corresponding to the vascular regions of the multiple fundus images; According to a preset vascular difference display rule, the vascular difference information between the multiple fundus images is used to generate a vascular change visualization image corresponding to the target user.

2. The image processing method according to claim 1, characterized in that: The blood vessel characteristic information includes curvature, and the determining of the blood vessel characteristic information corresponding to the blood vessel regions of the plurality of fundus images includes: Respectively extract at least one detection line of the blood vessel area of ​​each of the plurality of fundus images, wherein at least one of the detection lines includes a center line and / or an edge line; respectively determine a first position point on at least one of the detection lines of the blood vessel regions of the plurality of fundus images; Based on at least one first position point on the detection line of the blood vessel region of each of the plurality of fundus images, the curvature corresponding to the blood vessel region of each of the plurality of fundus images is determined.

3. The image processing method according to claim 2, characterized in that: The determining of the vascular difference information between the multiple fundus images based on the vascular feature information corresponding to the vascular regions of the multiple fundus images includes: For the same first position point, comparing the change difference of the curvature determined based on the same first position point in at least two of the plurality of fundus images; Based on the obtained difference in curvature, blood vessel difference information between the plurality of fundus images is determined.

4. The image processing method according to claim 3, characterized in that: The preset blood vessel difference display rule includes a mapping rule between a blood vessel curvature change difference and a marking color; The generating a vascular change visualization image corresponding to the target user from the vascular difference information between the multiple fundus images according to the preset vascular difference display rule includes: Determining the mark colors corresponding to the obtained change differences based on the mapping rule between the blood vessel tortuosity change differences and the mark colors; Based on the marking colors corresponding to the obtained change differences, a blood vessel change visualization image corresponding to the target user is generated.

5. The image processing method according to any one of claims 1 to 4, characterized in that: The blood vessel characteristic information includes the blood vessel diameter of the target position area, and determining the blood vessel difference information between the multiple fundus images based on the blood vessel characteristic information corresponding to the blood vessel areas of the multiple fundus images includes: For the same target location area, comparing the difference in the change of the blood vessel diameters of at least two of the multiple fundus images in the target location area; Based on the obtained difference in the change in the blood vessel diameter, the blood vessel difference information between the multiple fundus images is determined.

6. The image processing method according to claim 5, characterized in that: The preset blood vessel difference display rules include mapping rules between blood vessel diameter change differences and marking methods; The generating a vascular change visualization image corresponding to the target user from the vascular difference information between the multiple fundus images according to the preset vascular difference display rule includes: Determining the marking methods corresponding to the obtained blood vessel diameter change differences based on the mapping rules between the blood vessel diameter change differences and the marking methods; Based on the labeling methods corresponding to the obtained blood vessel diameter change differences, a blood vessel change visualization image corresponding to the target user is generated.

7. The image processing method according to claim 6, characterized in that: The marking method includes a marking color, and the generating of a visualization image of the blood vessel change corresponding to the target user based on the marking methods corresponding to the obtained blood vessel diameter change differences includes: In the target position area in any one of the at least two fundus images, the marking colors corresponding to the change differences of the blood vessel diameters in the target position area are displayed to obtain a visualized image of the blood vessel changes corresponding to the target user.

8. The image processing method according to any one of claims 1 to 4, characterized in that: After respectively extracting the blood vessel regions of the plurality of fundus images, the method further includes: For two fundus images among the plurality of fundus images, displaying a blood vessel region of one of the two fundus images in a first preset color to obtain a first display image; Displaying the blood vessel area of ​​the other fundus image of the two fundus images in a second preset color on the first display image to obtain a second display image; The first preset color is different from the second preset color.

9. An image processing device, characterized in that: include: An acquisition module, used to acquire multiple fundus images corresponding to multiple periods of a target user; An extraction module, used to extract the blood vessel regions of each of the plurality of fundus images respectively; A first determination module is used to determine the blood vessel characteristic information corresponding to the blood vessel regions of each of the plurality of fundus images; A second determination module, configured to determine vascular difference information between the plurality of fundus images based on vascular feature information corresponding to the vascular regions of the plurality of fundus images; The generating module is used to generate a vascular change visualization image corresponding to the target user based on the vascular difference information between the multiple fundus images according to a preset vascular difference display rule.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the image processing method according to any one of claims 1 to 8.

11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to execute the image processing method described in any one of claims 1 to 8.

Images