Fundus color photo synthesis method, electronic device and computer-readable medium

Abstract

This application discloses a fundus color photograph synthesis method, electronic device, and computer-readable medium. The method extracts an infrared vascular feature image from an infrared fundus image, and a visible vascular feature image from a visible light fundus image; aligns the infrared fundus image with the visible light fundus image based on the infrared and visible vascular feature images to obtain a registered infrared fundus image; generates an RGB fused image based on the visible light fundus image and the registered infrared fundus image; and adjusts the color information of the RGB fused image based on a standard fundus image to obtain a target fundus color photograph. The method can provide sufficient diagnostic information to facilitate accurate identification and analysis of fundus diseases by doctors, and also facilitates understanding of the images, allowing doctors to quickly provide fundus disease identification and analysis results.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for synthesizing fundus color photographs, an electronic device, and a computer-readable medium. Background Art

[0002] Fundus color photography refers to color photographs of the eye's fundus, taken with specialized equipment such as a fundus camera. These images clearly show fundus structures such as the retina, optic disc, and macula. Through these images, doctors can observe the morphology and changes of blood vessels, nerves, and pigments within the fundus. Fundus color photography can be used to detect and assess a variety of fundus diseases, such as diabetic retinopathy, age-related macular degeneration, and retinal vein occlusion. These images are crucial in ophthalmological clinical diagnosis, disease monitoring, and research, providing a valuable basis for diagnosis and treatment.

[0003] However, the images obtained by current fundus image processing technology cannot provide sufficient diagnostic information, thus limiting doctors' accurate identification and analysis of fundus diseases. Or, although it can provide sufficient diagnostic information, the lack of true color results limits doctors' understanding of the images.

[0004] Therefore, a better method for synthesizing fundus color photographs is urgently needed. Summary of the Invention

[0005] The present application aims to solve one of the technical problems in the related art to a certain extent. To this end, the present application provides a method for synthesizing fundus color photographs, an electronic device, and a computer-readable medium.

[0006] As a first aspect of the present application, a method for synthesizing a fundus color photograph is provided, wherein the method comprises:

[0007] Extracting an infrared light blood vessel feature image from the infrared light captured fundus image, and extracting a visible light blood vessel feature image from the visible light captured fundus image;

[0008] Based on the infrared light blood vessel characteristic image and the visible light blood vessel characteristic image, aligning the infrared light captured eye fundus image with the visible light captured eye fundus image to obtain a registered infrared light fundus image;

[0009] generating an RGB fused image based on the visible light fundus image and the registered infrared fundus image;

[0010] Based on the standard fundus image, the color information of the RGB fusion image is adjusted to obtain a target fundus color photograph.

[0011] Optionally, generating an RGB fused image based on the visible light fundus image and the registered infrared fundus image includes:

[0012] Based on a preset minimum radius and a preset maximum radius, the visible light collected fundus image and the registered infrared light fundus image are divided into a central area, a transition area and a peripheral area;

[0013] generating a color transition area image according to the transition area of ​​the visible light fundus image and the transition area of ​​the registered infrared light fundus image;

[0014] Mapping the pixel values ​​of the central area of ​​the visible light fundus image to the RGB three channels to obtain a color central area image; and mapping the pixel values ​​of the peripheral area of ​​the registered infrared light fundus image to the RGB three channels to obtain a color peripheral area image;

[0015] The color center area image, the color transition area image, and the color peripheral area image are spliced ​​to obtain an RGB fused image.

[0016] Optionally, generating a color transition area image according to the transition area of ​​the visible light fundus image and the transition area of ​​the registered infrared light fundus image includes:

[0017] For two pixels having identical pixel coordinates in a transition region between the visible light fundus image and the registered infrared fundus image, corresponding RGB three-channel values ​​are determined for the pixel coordinates based on weight parameters of the two pixels and their respective pixel values; wherein the weight parameters linearly transform with the distance between the pixel coordinates and the image center point;

[0018] A color transition area image is generated according to the pixel coordinates of the transition area and its corresponding RGB three-channel values.

[0019] Optionally, the aligning the infrared-light-captured fundus image with the visible-light-captured fundus image based on the infrared-light blood vessel feature image and the visible-light blood vessel feature image to obtain a registered infrared-light fundus image includes:

[0020] Optimizing the parameters of the initial affine matrix according to the infrared light blood vessel characteristic image and the visible light blood vessel characteristic image to obtain an optimized target affine matrix;

[0021] The target affine matrix is ​​applied to the infrared light collected fundus image to obtain a registered infrared light fundus image.

[0022] Optionally, optimizing parameters of an initial affine matrix according to the infrared light blood vessel characteristic image and the visible light blood vessel characteristic image to obtain an optimized target affine matrix includes:

[0023] Applying an initial affine matrix to the infrared blood vessel characteristic image to obtain a projected characteristic image;

[0024] According to the gradient descent algorithm, the parameters of the initial affine matrix are optimized until the optimization loss is minimized, thereby obtaining a target affine matrix; wherein the optimization loss is determined by the overlap rate between the projected feature image and the visible light vascular feature image.

[0025] Optionally, extracting a vascular feature image from the acquired fundus image includes:

[0026] Adjusting the brightness of the collected fundus image;

[0027] Optimize the vascular network pixels in the brightness-adjusted image;

[0028] Extracting an upper segmented sub-image and a lower segmented sub-image from the optimized image according to a preset segmentation ratio; wherein the preset segmentation ratio can be a value of [2 / 3, 1];

[0029] The largest sub-vascular network is extracted from the upper segmented sub-image and the lower segmented sub-image respectively and superimposed to obtain a vascular feature image.

[0030] Optionally, the optimizing the blood vessel network pixels in the brightness-adjusted image includes:

[0031] Using a ridge filter to identify ridge feature pixels from the brightness-adjusted image, and determining the ridge feature pixels having a confidence level higher than a confidence threshold as blood vessel pixels; wherein the confidence threshold is determined by a global mean and standard deviation of pixel values ​​in the brightness-adjusted image;

[0032] The brightness-adjusted image is expanded and contracted to fill in the discontinuous area of ​​the blood vessel pixels.

[0033] Optionally, adjusting the color information of the RGB fused image based on the standard fundus image to obtain a target fundus color photograph includes:

[0034] Determining the global mean and standard deviation of the RGB three channels of the standard fundus image to linearly scale the RGB three channel values ​​of the RGB fused image;

[0035] Converting the linearly scaled RGB fused image into a first Lab image, and converting the standard fundus image into a second Lab image;

[0036] Determine the global mean of the L channel of the second Lab image to linearly scale the L channel value of the first Lab image, and linearly scale the histograms of the a channel and the b channel of the first Lab image according to the histograms of the a channel and the b channel of the second Lab image to obtain a target fundus color photograph.

[0037] As a second aspect of the present application, an electronic device is provided, wherein the electronic device includes:

[0038] one or more processors;

[0039] A memory having one or more computer programs stored thereon, wherein when the one or more computer programs are executed by the one or more processors, the one or more processors implement the fundus color photograph synthesis method according to the first aspect of the present application.

[0040] As a third aspect of the present application, a computer-readable medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, the method for synthesizing fundus color photographs according to the first aspect of the present application is implemented.

[0041] The fundus color photograph synthesis method provided in the embodiments of the present application extracts an infrared vascular feature image from an infrared fundus image, and extracts a visible vascular feature image from a visible light fundus image. Based on the infrared vascular feature image and the visible vascular feature image, the infrared fundus image is aligned with the visible light fundus image to obtain a registered infrared fundus image. An RGB fused image is generated based on the visible light fundus image and the registered infrared fundus image. The color information of the RGB fused image is adjusted based on a standard fundus image to obtain a target fundus color photograph. Based on spatial fusion and color conversion of multi-band light source images, a high-quality fundus color photograph is synthesized, which can provide sufficient diagnostic information to facilitate doctors' accurate identification and analysis of fundus diseases, and facilitate doctors' understanding of the images and thus quickly provide fundus disease identification and analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present application will be further described below with reference to the accompanying drawings:

[0043] Figure 1 This is a flow chart of an implementation method of the fundus color photograph synthesis method provided in the examples of the present application;

[0044] Figure 2 This is a flowchart of an implementation method for generating an RGB fused image provided in an embodiment of the present application;

[0045] Figure 3This is a flowchart of an implementation method of generating a color transition area image provided in an embodiment of the present application;

[0046] Figure 4 This is a flow chart of an implementation method for obtaining a registered infrared fundus image provided in an embodiment of the present application;

[0047] Figure 5 This is a flowchart of an implementation method for obtaining an optimized target affine matrix provided in an embodiment of the present application;

[0048] Figure 6 This is a flowchart of an implementation method of extracting a vascular feature image from a captured fundus image provided by an embodiment of the present application;

[0049] Figure 7 This is a flowchart of an implementation method of optimizing vascular network pixels in an image after brightness adjustment provided by an embodiment of the present application;

[0050] Figure 8 This is a flowchart of an implementation method of obtaining a target fundus color photograph based on an RGB fusion image provided in an embodiment of the present application;

[0051] Figure 9 Schematic diagram of a specific implementation of the method for synthesizing fundus color photographs provided in the examples of the present application;

[0052] Figure 10 This is a module diagram of an implementation of an electronic device provided in an embodiment of the present application;

[0053] Figure 11 It is a schematic diagram of the computer-readable medium provided in an embodiment of the present application.

[0054] Description of Reference Numerals

[0055] 101: Processor 102: Memory

[0056] 103: I / O interface 104: bus DETAILED DESCRIPTION

[0057] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described in the embodiments are intended to be used to explain the present application and are not to be construed as limiting the present application.

[0058] References in this specification to "one embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with the embodiment itself can be included in at least one embodiment disclosed herein. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0059] Fundus color photography refers to color photographs of the eye's fundus, taken with specialized equipment such as a fundus camera. These images clearly show fundus structures such as the retina, optic disc, and macula. Through these images, doctors can observe the morphology and changes of blood vessels, nerves, and pigments within the fundus. Fundus color photography can be used to detect and assess a variety of fundus diseases, such as diabetic retinopathy, age-related macular degeneration, and retinal vein occlusion. These images are crucial in ophthalmological clinical diagnosis, disease monitoring, and research, providing a valuable basis for diagnosis and treatment.

[0060] However, the images obtained by current fundus image processing technology cannot provide sufficient diagnostic information, thus limiting doctors' accurate identification and analysis of fundus diseases. Or, although it can provide sufficient diagnostic information, the lack of true color results limits doctors' understanding of the images.

[0061] Therefore, a better method for synthesizing fundus color photographs is urgently needed.

[0062] In this regard, the applicant of this application proposes that the fusion and color space conversion of fundus images collected by infrared light and fundus images collected by visible light can achieve high-quality color image synthesis, and is particularly suitable for the synthesis of fundus color photographs.

[0063] As a first aspect of the embodiment of the present application, a method for synthesizing fundus color photographs is provided, such as Figure 1 As shown, the method includes:

[0064] Step S110, extracting an infrared light blood vessel feature image from the infrared light captured fundus image, and extracting a visible light blood vessel feature image from the visible light captured fundus image;

[0065] Step S120, based on the infrared light blood vessel characteristic image and the visible light blood vessel characteristic image, aligning the infrared light captured eye fundus image with the visible light captured eye fundus image to obtain a registered infrared light fundus image;

[0066] Step S130, generating an RGB fused image based on the visible light fundus image and the registered infrared fundus image;

[0067] Step S140 : Based on the standard fundus image, the color information of the RGB fused image is adjusted to obtain a target fundus color photograph.

[0068] The infrared fundus image and the visible light fundus image are images obtained by capturing the fundus of the same patient under an infrared light source environment and a visible light light source environment, respectively. The embodiments of this application do not specifically limit the wavelengths of infrared light and visible light. For example, the wavelengths of infrared light and visible light can be 785nm and 525nm, respectively.

[0069] The embodiment of the present application does not impose any special limitation on the execution order between the two sub-steps in step S110. Either sub-step may be executed first and then the other sub-step, or both sub-steps may be executed simultaneously.

[0070] Among them, the embodiment of the present application extracts the vascular feature image to align the infrared light captured fundus image toward the visible light captured fundus image, which can eliminate the spatial deviation of the multimodal fundus image, integrate the complementary information of infrared light and visible light, and maximize the effect of multimodal imaging.

[0071] Among them, the embodiment of the present application spatially fuses and color encodes the fundus image collected by visible light and the registered infrared fundus image, integrating the "surface color characteristics" of the fundus under the stimulation of visible light and the "deep grayscale characteristics" of the fundus under the stimulation of infrared light into an RGB fused image. It not only retains the intuitiveness of traditional fundus color photos, but also supplements the penetrating advantage of infrared light, so that the RGB fused image contains higher information content.

[0072] Among them, the embodiment of the present application obtains the target fundus color photograph by adjusting the color information of the RGB fused image based on the standard fundus image, and converts the RGB fused image that may have color distortion, color overload and other problems due to light source differences into the target fundus color photograph, thereby realizing the key transformation of multimodal imaging from technical implementation to clinical application. By establishing a unified color diagnostic language, it solves the problems of color deviation, equipment differences, visual cognitive conflicts, etc. in cross-modal imaging, so that the target fundus color photograph finally obtained can not only retain the deep information of infrared light, but also conform to the doctor's visual habits of traditional fundus color photographs, which is ultimately conducive to improving the accuracy and repeatability of fundus disease diagnosis.

[0073] The fundus color photograph synthesis method provided in the embodiments of the present application extracts an infrared vascular feature image from an infrared fundus image, and extracts a visible vascular feature image from a visible light fundus image. Based on the infrared vascular feature image and the visible vascular feature image, the infrared fundus image is aligned with the visible light fundus image to obtain a registered infrared fundus image. An RGB fused image is generated based on the visible light fundus image and the registered infrared fundus image. The color information of the RGB fused image is adjusted based on a standard fundus image to obtain a target fundus color photograph. Based on spatial fusion and color conversion of multi-band light source images, a high-quality fundus color photograph is synthesized, which can provide sufficient diagnostic information to facilitate doctors' accurate identification and analysis of fundus diseases, and facilitate doctors' understanding of the images and thus quickly provide fundus disease identification and analysis results.

[0074] In some embodiments, the RGB fusion image is generated based on the visible light fundus image and the registered infrared fundus image (ie, the step S130 involved), as shown in FIG. Figure 2 Shown, including:

[0075] Step S210, dividing the visible light fundus image and the registered infrared fundus image into a central area, a transition area, and a peripheral area based on a preset minimum radius and a preset maximum radius;

[0076] Step S220, generating a color transition area image according to the transition area of ​​the visible light fundus image and the transition area of ​​the registered infrared light fundus image;

[0077] Step S230, mapping the pixel values ​​of the central area of ​​the visible light fundus image to the RGB three channels to obtain a color central area image; and mapping the pixel values ​​of the peripheral area of ​​the registered infrared fundus image to the RGB three channels to obtain a color peripheral area image;

[0078] In step S240 , the color center area image, the color transition area image, and the color peripheral area image are spliced ​​to obtain an RGB fused image.

[0079] The present embodiment does not impose any specific limitations on the values ​​of the preset minimum radius r and the preset maximum radius R. For example, they can be determined based on the actual size of the fundus image collected by infrared light or the fundus image collected by visible light. It is also understood that the value of the preset minimum radius r is smaller than the value of the preset maximum radius R.

[0080] The central area refers to the part that is no more than r away from the center point of the image, the transition area refers to the part that is more than r but no more than R away from the center point of the image, and the peripheral area refers to the part that is more than R away from the center point of the image.

[0081] Among them, it can be understood that the RGB fusion image obtained in the embodiment of the present application also includes a central area, a transition area and a peripheral area, that is, a color central area image, a color transition area image and a color peripheral area image. The color central area image is the part of the RGB fusion image that is no more than r away from the center point of the image, and is generated based on the central area of ​​the fundus image captured by visible light. The color transition area image is the part of the RGB fusion image that is more than r but not more than R away from the center point of the image, and is generated based on the transition area of ​​the fundus image captured by visible light and the transition area of ​​the registered infrared fundus image. The color peripheral area image is the part of the RGB fusion image that is more than R away from the center point of the image, and is generated based on the peripheral area of ​​the registered infrared fundus image.

[0082] Among them, the embodiment of the present application does not specifically limit how to map pixel values ​​to the RGB three channels. For example, it can be copied to the R, G, and B channels, or it can be allocated as needed.

[0083] In some embodiments, the color transition area image (ie, the one involved in step S220) is generated based on the transition area of ​​the visible light fundus image and the transition area of ​​the registered infrared fundus image. Figure 3 Shown, including:

[0084] Step S310: For two pixels having identical pixel coordinates in a transition region between the visible light fundus image and the registered infrared fundus image, corresponding RGB three-channel values ​​are determined for the pixel coordinates based on weight parameters of the two pixels and their respective pixel values; wherein the weight parameters linearly transform with the distance between the pixel coordinates and the image center point;

[0085] Step S320 : generating a color transition area image according to the coordinates of each pixel in the transition area and its corresponding RGB three-channel values.

[0086] Among them, the embodiment of the present application realizes linear superposition and gradient processing of pixels in two transition areas by setting weight parameters for each pixel coordinate in the transition area.

[0087] Among them, the embodiment of the present application does not specifically limit how the weight parameter is linearly transformed with the distance between the pixel coordinates and the image center point. For example, for any pixel in the transition area, its distance to the image center is d (r≤d≤R). For the transition area of ​​the fundus image collected by visible light, the weight parameter With distance Increases and decreases linearly from 1 (when d=r) to 0 (when d=R). For the transition area of ​​the registered infrared fundus image, the weight parameter With distance Increase linearly.

[0088] In some embodiments, based on the infrared light blood vessel feature image and the visible light blood vessel feature image, the infrared light captured fundus image is aligned with the visible light captured fundus image to obtain a registered infrared light fundus image (i.e., step S120 involved), as shown in FIG. Figure 4 Shown, including:

[0089] Step S410 , optimizing the parameters of the initial affine matrix based on the infrared light blood vessel characteristic image and the visible light blood vessel characteristic image to obtain an optimized target affine matrix;

[0090] Step S420 : Apply the target affine matrix to the infrared fundus image to obtain a registered infrared fundus image.

[0091] The applicants of this application note that the challenge in registering fundus images lies in the fact that blood vessels at the outermost periphery of the field of view are essentially straight, lacking bifurcation points or corners, making them unsuitable for keypoint registration. Therefore, the applicants of this application propose iteratively optimizing the initial affine matrix and then registering the infrared fundus image with the visible light fundus image using the target affine matrix.

[0092] The embodiment of the present application does not specifically limit the initial affine matrix. For example, a two-dimensional affine matrix (Affine Matrix) can be used, which can be expressed as a 3×3 matrix, that is, , the optimization target is the six parameters a, b, c, d, e, and f of the Affine Matrix.

[0093] Among them, applying the affine matrix to the image means using the parameters of the affine matrix to transform the initial pixel coordinates (x, y) to obtain the transformed pixel coordinates ( , For example, by applying a two-dimensional affine matrix to an image, the coordinates (x, y) of the pixel can be transformed using the parameters a, b, and c to obtain the transformed coordinates , use parameters d, e, f to transform the pixel coordinates (x, y) to obtain the transformed coordinates .

[0094] In some embodiments, the parameters of the initial affine matrix are optimized based on the infrared light blood vessel characteristic image and the visible light blood vessel characteristic image to obtain the optimized target affine matrix (i.e., the one involved in step S410), such as Figure 5 Shown, including:

[0095] Step S510, applying the initial affine matrix to the infrared blood vessel characteristic image to obtain a projected characteristic image;

[0096] Step S510, optimizing the parameters of the initial affine matrix according to the gradient descent algorithm until the optimization loss is minimized, thereby obtaining a target affine matrix; wherein the optimization loss is determined by the overlap ratio between the projected feature image and the visible light vascular feature image.

[0097] It is understood that obtaining the target affine matrix with the minimum optimization loss may require multiple gradient descent iterations. The optimization loss function is the overlap ratio between the projected feature image and the visible light vascular feature image. Minimizing the overlap ratio indicates a minimum optimization loss function and a maximum optimization loss. Conversely, maximizing the overlap ratio indicates a maximum optimization loss function and a minimum optimization loss.

[0098] In some embodiments, a vascular feature image is extracted from the acquired fundus image (ie, the image involved in step S110), such as Figure 6 Shown, including:

[0099] Step S610, adjusting the brightness of the collected fundus image;

[0100] Step S620, optimizing the blood vessel network pixels in the brightness-adjusted image;

[0101] Step S630: extracting an upper segmented sub-image and a lower segmented sub-image from the optimized image according to a preset segmentation ratio; wherein the preset segmentation ratio may be a value of [2 / 3, 1];

[0102] Step S640 , extracting the largest sub-vessel network from the upper segmented sub-image and the lower segmented sub-image respectively and superimposing them to obtain a vessel feature image.

[0103] It can be understood that the above-mentioned step S110 involves two sub-steps, namely, extracting a visible light vascular feature image from the fundus image captured by visible light, and extracting an infrared light vascular feature image from the fundus image captured by infrared light, but the execution principles of the two sub-steps can be consistent, that is, both are extracted through steps S610-S640.

[0104] The embodiment of the present application does not specifically limit how to adjust the brightness of the collected fundus image. For example, the brightness can be adjusted by performing contrast-limited adaptive histogram equalization (CLAHE) processing on the collected fundus image.

[0105] Among them, vascular network pixels refer to pixels that constitute the vascular network. Optimizing the vascular network pixels in the image after brightness adjustment involves both identifying the vascular network pixels in the image after brightness adjustment and filling in discontinuous areas of the vascular network.

[0106] The applicant of this application proposes that, considering that the discontinuity of blood vessels at the middle optic cup may cause the largest sub-vascular network and the second largest sub-vascular network in an image to be the upper half of the blood vessels and the lower half of the blood vessels, respectively, by first dividing the optimized image into two sub-images according to a preset segmentation ratio and then extracting the largest sub-vascular network, such a situation can be effectively avoided, thereby obtaining a clear vascular network and thus a clearer and more accurate vascular feature image.

[0107] The embodiment of the present application does not impose any specific restrictions on the value of the preset split ratio, as long as the value is within the interval [2 / 3, 1). As an optimal implementation, the value of the preset split ratio can be 2 / 3.

[0108] The upper segmented sub-image is the sub-image with the larger area obtained by segmenting from the top to the bottom of the image when the preset segmentation ratio is reached (for example, the upper 2 / 3 of the image when the preset segmentation ratio is 2 / 3). The same applies to the lower segmented sub-image.

[0109] In some embodiments, the optimization of the vascular network pixels in the brightness-adjusted image (ie, step S620 involved) is as follows: Figure 7 Shown, including:

[0110] Step S710, using a ridge filter to identify ridge feature pixels from the brightness-adjusted image, and determining ridge feature pixels having a confidence level higher than a confidence threshold as blood vessel pixels; wherein the confidence threshold is determined by the global mean and standard deviation of the pixel values ​​of the brightness-adjusted image;

[0111] Step S720 , performing expansion and contraction processing on the brightness-adjusted image to fill in the region where the blood vessel pixels are discontinuous.

[0112] Among them, the embodiment of the present application first uses a ridge filter to perform pattern recognition on the network of blood vessel area features. For the identified ridge feature pixel area, the area with a confidence level higher than the confidence threshold is identified as a blood vessel. Then, the image after brightness adjustment is post-processed by expansion and contraction to fill the area where the blood vessel pixels are discontinuous.

[0113] The embodiment of the present application does not specifically limit how to determine the confidence threshold based on the global mean and standard deviation of the pixel values ​​of the image after brightness adjustment. For example, the confidence threshold can be set to ,in, represents the global mean, Represents standard deviation.

[0114] It is understandable that, before performing expansion and contraction processing on an image, the image is usually first subjected to binarization processing.

[0115] In some embodiments, the color information of the RGB fused image is adjusted based on the standard fundus image to obtain a target fundus color photograph (ie, the one involved in step S140), such as Figure 8 Shown, including:

[0116] Step S810, determining the global mean and standard deviation of the RGB three channels of the standard fundus image to linearly scale the RGB three channel values ​​of the RGB fused image;

[0117] Step S820, converting the linearly scaled RGB fused image into a first Lab image, and converting the standard fundus image into a second Lab image;

[0118] Step S830: Determine the global mean of the L channel of the second Lab image to linearly scale the L channel value of the first Lab image, and linearly scale the histograms of the a channel and the b channel of the first Lab image according to the histograms of the a channel and the b channel of the second Lab image to obtain a color fundus photograph of the target.

[0119] Among them, the standard fundus image is the template fundus image. The embodiment of the present application first obtains the global mean and standard deviation of the RGB three channels of the standard fundus image, linearly scales the RGB three channel values ​​of the RGB fused image to the global mean and standard deviation of the RGB three channels of the standard fundus image, and realizes preliminary color adjustment of the RGB fused image.

[0120] Furthermore, the embodiment of the present application converts both the RGB fusion image and the standard fundus image into the Lab color space, uses the first Lab image and the second Lab image to represent the conversion results respectively, obtains the global mean of the L channel of the second Lab image, and linearly scales the L channel value of the first Lab image to the global mean of the L channel of the second Lab image, which can ensure the consistency of the overall brightness of the target fundus color photograph finally obtained.

[0121] The embodiment of the present application also linearly scales and fits the histograms of the a channel and the b channel of the first Lab image to the histograms of the a channel and the b channel of the second Lab image, which can ensure that the color distribution of the target fundus color photograph finally obtained is highly consistent with the color distribution of the standard fundus image.

[0122] In addition, the applicant of this application also proposed that after step S140, the completely unsuperimposed areas at the edge of the target fundus color photograph can be cropped to ensure that the image structure of the target fundus color photograph finally obtained is complete and the visual effect is better.

[0123] The following reference Figure 9 As shown in FIG, and in combination with a specific embodiment, the fundus color photo synthesis method provided in the embodiment of the present application is further described. Figure 9 As shown in the figure, ridge feature extraction, binarization, and maximum sub-image extraction are used to extract vascular feature images from infrared fundus images (IR images) and visible light fundus images (VL images). , infrared vascular feature images, and visible light vascular feature images. The infrared fundus image is aligned with the visible light fundus image to obtain a registered infrared fundus image. Furthermore, an RGB fused image is generated based on the visible light fundus image (VL image) and the registered infrared fundus image (IR image). The RGB fused image is then converted to a standard fundus image using Lab color conversion to obtain a target fundus color photograph. The edges of the target fundus color photograph are then cropped.

[0124] As a second aspect of the embodiments of the present application, an electronic device is provided, wherein, Figure 10 As shown, the electronic device includes:

[0125] One or more processors 101;

[0126] The memory 102 stores one or more computer programs. When the one or more computer programs are executed by the one or more processors 101, the one or more processors 101 implement the fundus color photograph synthesis method provided in the first aspect of the embodiment of the present application.

[0127] The electronic device may further include one or more I / O interfaces 103 connected between the processor 101 and the memory 102 and configured to implement information exchange between the processor 101 and the memory 102 .

[0128] Among them, the processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically such as SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read-write interface) is connected between the processor and the memory, and can realize information exchange between the processor and the memory, including but not limited to a data bus (Bus), etc.

[0129] In some embodiments, the processor 101 , the memory 102 , and the I / O interface 103 are connected to each other via a bus 104 , and further connected to other components of the computing device.

[0130] As a third aspect of the embodiment of this application, Figure 11 As shown, a computer-readable medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, the method for synthesizing fundus color photographs provided in the first aspect of the embodiment of the present application is implemented.

[0131] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. Accordingly, the computer program can be stored in a non-volatile computer-readable storage medium, and when the computer program is executed, it can implement the method of any of the above-mentioned embodiments. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the embodiments of the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0132] The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Those skilled in the art should understand that the present application includes but is not limited to the contents described in the drawings and the above specific embodiments. Any modifications that do not deviate from the functional and structural principles of the present application are included within the scope of the claims.

Claims

1. A method for synthesizing fundus color photographs, characterized in that: The method comprises: Extracting an infrared light blood vessel feature image from the infrared light captured fundus image, and extracting a visible light blood vessel feature image from the visible light captured fundus image; Based on the infrared light blood vessel characteristic image and the visible light blood vessel characteristic image, aligning the infrared light captured eye fundus image with the visible light captured eye fundus image to obtain a registered infrared light fundus image; generating an RGB fused image based on the visible light fundus image and the registered infrared fundus image; Based on the standard fundus image, adjusting the color information of the RGB fusion image to obtain a target fundus color photograph; The step of generating an RGB fused image based on the visible light fundus image and the registered infrared fundus image includes: Based on a preset minimum radius and a preset maximum radius, the visible light collected fundus image and the registered infrared light fundus image are divided into a central area, a transition area and a peripheral area; generating a color transition area image according to the transition area of ​​the visible light fundus image and the transition area of ​​the registered infrared light fundus image; Mapping the pixel values ​​of the central area of ​​the visible light fundus image to the RGB three channels to obtain a color central area image; and mapping the pixel values ​​of the peripheral area of ​​the registered infrared light fundus image to the RGB three channels to obtain a color peripheral area image; The color center area image, the color transition area image, and the color peripheral area image are spliced ​​to obtain an RGB fused image.

2. The method according to claim 1, characterized in that The step of generating a color transition area image according to the transition area of ​​the visible light fundus image and the transition area of ​​the registered infrared light fundus image includes: For two pixels having identical pixel coordinates in a transition region between the visible light fundus image and the registered infrared fundus image, corresponding RGB three-channel values ​​are determined for the pixel coordinates based on weight parameters of the two pixels and their respective pixel values; wherein the weight parameters linearly transform with the distance between the pixel coordinates and the image center point; A color transition area image is generated according to the pixel coordinates of the transition area and its corresponding RGB three-channel values.

3. The method according to claim 1, characterized in that The step of aligning the infrared light captured fundus image with the visible light captured fundus image based on the infrared light blood vessel characteristic image and the visible light blood vessel characteristic image to obtain a registered infrared light fundus image includes: Optimizing the parameters of the initial affine matrix according to the infrared light blood vessel characteristic image and the visible light blood vessel characteristic image to obtain an optimized target affine matrix; The target affine matrix is ​​applied to the infrared light collected fundus image to obtain a registered infrared light fundus image.

4. The method according to claim 3, characterized in that Optimizing the parameters of the initial affine matrix according to the infrared light blood vessel characteristic image and the visible light blood vessel characteristic image to obtain an optimized target affine matrix includes: Applying an initial affine matrix to the infrared blood vessel characteristic image to obtain a projected characteristic image; According to the gradient descent algorithm, the parameters of the initial affine matrix are optimized until the optimization loss is minimized, thereby obtaining a target affine matrix; wherein the optimization loss is determined by the overlap rate between the projected feature image and the visible light vascular feature image.

5. The method according to claim 1, characterized in that Extracting vascular feature images from the acquired fundus images, including: Adjusting the brightness of the collected fundus image; Optimize the vascular network pixels in the brightness-adjusted image; Extracting an upper segmentation sub-image and a lower segmentation sub-image from the optimized image according to a preset segmentation ratio; wherein the value of the preset segmentation ratio includes [2 / 3, 1); The largest sub-vascular network is extracted from the upper segmented sub-image and the lower segmented sub-image respectively and superimposed to obtain a vascular feature image.

6. The method according to claim 5, characterized in that The optimizing of the blood vessel network pixels in the brightness-adjusted image includes: Using a ridge filter to identify ridge feature pixels from the brightness-adjusted image, and determining the ridge feature pixels having a confidence level higher than a confidence threshold as blood vessel pixels; wherein the confidence threshold is determined by a global mean and standard deviation of pixel values ​​in the brightness-adjusted image; The brightness-adjusted image is expanded and contracted to fill in the discontinuous area of ​​the blood vessel pixels.

7. The method according to claim 1, characterized in that The method of adjusting the color information of the RGB fused image based on the standard fundus image to obtain a target fundus color photograph includes: Determining the global mean and standard deviation of the RGB three channels of the standard fundus image to linearly scale the RGB three channel values ​​of the RGB fused image; Converting the linearly scaled RGB fused image into a first Lab image, and converting the standard fundus image into a second Lab image; Determine the global mean of the L channel of the second Lab image to linearly scale the L channel value of the first Lab image, and linearly scale the histograms of the a channel and the b channel of the first Lab image according to the histograms of the a channel and the b channel of the second Lab image to obtain a target fundus color photograph.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory having one or more computer programs stored thereon, wherein when the one or more computer programs are executed by the one or more processors, the one or more processors implement the fundus color photograph synthesis method according to any one of claims 1 to 7.

9. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for synthesizing fundus color photographs according to any one of claims 1 to 7 is implemented.

Images