Method and apparatus for processing fundus images

Abstract

The present invention provides a method and apparatus for processing fundus images. Among them, the method includes: collecting a plurality of fundus strip images through a fundus camera; taking adjacent fundus strip images as a group of images to be processed; determining the overlapping area and non-overlapping area of each group of images to be processed; determining the pixel values of the target position in the overlapping area of each group of images to be processed in each fundus strip image, determining the pixel value of the target position in the overlapping area according to the pixel values in each fundus strip image, and obtaining the target overlapping area according to the determined pixel values; splicing each group of non-overlapping areas and the target overlapping area according to the scanning information to obtain a fundus image. In this way, effective luminance information can be fully utilized, thereby eliminating or significantly reducing the splicing stripes in the horizontal direction.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and apparatus for processing fundus images. Background Art

[0002] The number of patients with fundus diseases is huge, with a very high risk of blindness and irreversibility. Therefore, it is very necessary to conduct large-scale and rapid screening in the field of ophthalmology. A fundus camera is an important instrument for the diagnosis and screening of ophthalmic diseases, capable of observing and recording the structures at the back of the eye, including the retina, choroid, and optic disc. Fundus examination is crucial for the early detection and treatment of ophthalmic diseases.

[0003] With the technological iteration, fundus cameras have gradually evolved from traditional optical fundus cameras to wide-angle fundus cameras and ultra-wide-angle fundus cameras. Due to the large imaging field of view of ultra-wide-angle fundus cameras, segmented scanning and then stitching methods are usually adopted to obtain a complete fundus image. However, due to the optical system, obvious horizontal stitching stripes appear in the stitched image, and artifacts usually appear at the center of the field of view. The stitching stripes will reduce the quality of the fundus image, thus affecting clinical diagnosis.

[0004] For eliminating the stitching stripes, the traditional methods are to increase the light intensity or increase the number of segments. Increasing the light intensity can weaken the stitching stripes, but strong light irradiation will make the patient feel uncomfortable. Increasing the number of segments can make more parts of the fundus be in the bright band, thus reducing the influence of the stitching stripes, but this will prolong the scanning time, increase the risk of eye movement, and reduce the scanning success rate. At the same time, increasing the scanning time will also increase the total light intensity and reduce the comfort of the patient. For eliminating artifacts, the common method is to design the optical system to avoid artifacts as much as possible. But this method cannot completely eliminate artifacts. Therefore, most of the ultra-wide-angle fundus cameras on the market currently have stitching marks and artifacts, and how to effectively optimize the stitching and artifact removal algorithms becomes necessary.

[0005] In the prior art, generally, the "hard stitching" method is used for stitching fundus images, that is: multiple fundus strip images are stitched in sequence to obtain a fundus image. However, when shooting two adjacent fundus strip images, the brightness distribution of the same area on the object being photographed is different in the two fundus strip images. Therefore, at the stitching boundary, obvious brightness discontinuity will occur, with one side being darker than the other, presenting as horizontal stitching stripes. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a method and apparatus for processing fundus images, so as to make full use of the effective brightness information, thereby eliminating or significantly weakening the horizontal stitching stripes.

[0007] In a first aspect, an embodiment of the present invention provides a method for processing fundus images. The method includes: acquiring a plurality of fundus strip images through a fundus camera; taking adjacent fundus strip images as a group of images to be processed; determining the overlapping area and non-overlapping area of each group of images to be processed; determining the pixel values of the target position in the overlapping area of each group of images to be processed in each fundus strip image, determining the pixel value of the target position in the overlapping area according to the pixel values in each fundus strip image, and obtaining the target overlapping area according to the determined pixel values; splicing each group of non-overlapping areas and the target overlapping area according to the scanning information to obtain a fundus image.

[0008] In an alternative embodiment of the present application, the step of determining the pixel value of the target position in the overlapping area according to the pixel values in each fundus strip image includes: taking the larger value of the pixel values in each fundus strip image as the pixel value of the target position in the overlapping area.

[0009] In an alternative embodiment of the present application, after the step of acquiring a plurality of fundus strip images through the fundus camera, the method further includes: cropping the fundus strip images to determine new fundus strip images, and the new fundus strip images are used to form the images to be processed. Among them, the cropping range of the fundus strip images is determined based on the brightness distribution of the fundus strip images.

[0010] In an alternative embodiment of the present application, the method further includes: acquiring a plurality of background strip images through the fundus camera; where the background strip images do not include the object being photographed; determining a target background strip image from the plurality of background strip images; determining an artifact mask from each target background strip image; expanding the artifact mask and then performing an inversion operation on the artifact mask to obtain a non-artifact mask; determining the non-artifact area of the corresponding fundus strip image based on each non-artifact mask to obtain the fundus strip image after artifact removal processing.

[0011] In an alternative embodiment of the present application, the step of determining a target background strip image from the plurality of background strip images includes: determining a fundus strip image with artifacts as a target fundus strip image, and determining the background strip image corresponding to the target fundus strip image from the plurality of background strip images as the target background strip image.

[0012] In an alternative embodiment of the present application, the step of determining the background strip image corresponding to the target fundus strip image from the plurality of background strip images as the target background strip image includes: determining the background strip image with the same shooting area as the target fundus strip image from the plurality of background strip images as the target background strip image.

[0013] In an alternative embodiment of the present application, the step of acquiring a plurality of background strip images by the fundus camera includes: acquiring background strip images corresponding to a plurality of preset diopters by the fundus camera; each of the preset diopters corresponding to a plurality of the background strip images; determining a plurality of the background strip images corresponding to the preset diopter that is closest to the target diopter corresponding to the target fundus strip image.

[0014] In an alternative embodiment of the present application, the step of determining a target background strip image from a plurality of the background strip images and determining an artifact mask from each of the target background strip images includes: determining, as a target background strip image with an artifact, a background strip image among a plurality of the background strip images that has a pixel value greater than a preset pixel threshold; determining an artifact mask according to a region in each of the target background strip images where the pixel value is greater than the pixel threshold.

[0015] In an alternative embodiment of the present application, the pixel threshold is determined based on the light intensity and exposure time when the fundus camera acquires the background strip image.

[0016] In a second aspect, an embodiment of the present invention further provides a processing device for fundus images. The device includes: a fundus strip image acquisition module, configured to acquire a plurality of fundus strip images by a fundus camera; a to-be-processed image determination module, configured to use adjacent fundus strip images as a group of to-be-processed images; a to-be-processed image division module, configured to determine an overlapping region and a non-overlapping region of each group of to-be-processed images; a pixel value determination module, configured to determine pixel values of a target position in the overlapping region of each group of to-be-processed images in each fundus strip image of the group of to-be-processed images, determine a pixel value of the target position in the overlapping region according to the pixel values in each fundus strip image, and obtain a target overlapping region according to the determined pixel values; a to-be-processed image stitching module, configured to stitch each group of non-overlapping regions and the target overlapping region according to scanning information to obtain a fundus image.

[0017] The embodiments of the present invention bring the following beneficial effects:

[0018] An embodiment of the present invention provides a method and device for processing fundus images. Multiple fundus strip images are collected by a fundus camera; adjacent fundus strip images are used as a group of images to be processed; the overlapping area and non-overlapping area of each group of images to be processed are determined; the pixel values of the target position in the overlapping area of each group of images to be processed in each fundus strip image are determined, and the pixel value of the target position in the overlapping area is determined according to the pixel values in each fundus strip image. The target overlapping area is obtained according to the determined pixel values; each group of non-overlapping areas and the target overlapping area are stitched according to the scanning information to obtain a fundus image. In this way, the method of determining the pixel value of the target position in the overlapping area by using the pixel values in the fundus strip images of each group of images to be processed can make full use of effective brightness information, thereby eliminating or significantly reducing the stitching stripes in the horizontal direction.

[0019] Other features and advantages of the present disclosure will be described in the following description, or some features and advantages can be inferred from the description or determined without doubt, or can be obtained by implementing the above technologies of the present disclosure.

[0020] To make the above objects, features, and advantages of the present disclosure more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, details are described as follows. Description of the Drawings

[0021] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 Schematic diagram of a fundus strip image strip1 provided by an embodiment of the present invention;

[0023] Figure 2 Schematic diagram of a fundus strip image strip2 provided by an embodiment of the present invention;

[0024] Figure 3 Schematic diagram of a puzzle image2 provided by an embodiment of the present invention;

[0025] Figure 4 Flowchart of a method for processing fundus images provided by an embodiment of the present invention;

[0026] Figure 5 Schematic diagram of a fundus strip image with artifacts provided by an embodiment of the present invention;

[0027] Figure 6 Schematic diagram of the splicing result of strip1 and strip2 provided by an embodiment of the present invention;

[0028] Figure 7 Schematic diagram of a fundus image spliced by the "hard splicing" method provided by an embodiment of the present invention;

[0029] Figure 8 Schematic diagram of a fundus image spliced by the "take max" method provided by an embodiment of the present invention;

[0030] Figure 9 Flowchart of another method for processing fundus images provided by an embodiment of the present invention;

[0031] Figure 10 Schematic diagram of a background strip image with artifacts provided by an embodiment of the present invention;

[0032] Figure 11 Schematic diagram of another background strip image with artifacts provided by an embodiment of the present invention;

[0033] Figure 12 Schematic diagram of a non-artifact mask provided by an embodiment of the present invention;

[0034] Figure 13 Schematic diagram of a fundus strip image after artifact removal processing provided by an embodiment of the present invention;

[0035] Figure 14 Schematic diagram of a specific algorithm for artifact removal processing provided by an embodiment of the present invention;

[0036] Figure 15 Schematic diagram of the structure of a fundus image processing device provided by an embodiment of the present invention;

[0037] Figure 16 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0039] Currently, the fundus images are generally stitched by the "hard stitching" method, where multiple fundus strip images are stitched in sequence to obtain the fundus image. Suppose the multiple fundus strip images sequentially captured by the fundus camera are strip1, strip2... strip N , as can be seen in Figure 1 a schematic diagram of a fundus strip image strip1 and Figure 2 a schematic diagram of a fundus strip image strip2 shown in Figure 1 is strip1, Figure 2 is strip2, and the number of rows in the overlapping area is overlap_rows. Then, crop_bottom_rows rows at the bottom of strip1 and crop_top_rows rows at the top of strip2 can be cropped. Among them, crop_top_rows and crop_bottom_rows can be different for each strip, but it is necessary to ensure that crop_bottom_rows + crop_top_rows = overlap_rows. After the above cropping, the two fundus strip images exactly have no overlapping area and can be seamlessly stitched without misalignment to obtain the puzzle image2, as can be seen in Figure 3 a schematic diagram of a puzzle image2 shown in N .

[0040] However, this "hard stitching" method has some problems: When shooting two adjacent fundus strip images, the brightness distribution of the same area on the object being photographed is different in the two fundus strip images. Therefore, at the stitching boundary, there will be obvious brightness discontinuity, with one side being darker than the other, presenting as horizontal stitching stripes.

[0041] Based on this, a fundus image processing method and apparatus provided in an embodiment of the present invention specifically provide a fundus image stitching method and a fundus image artifact removal method, which can make the processed fundus image more natural, without stitching traces, without artifacts, can avoid stitching stripes, have a significant artifact removal effect, and have the advantage of fast processing speed.

[0042] For the convenience of understanding this embodiment, first, a fundus image processing method disclosed in an embodiment of the present invention will be introduced in detail.

[0043] Embodiment 1:

[0044] An embodiment of the present invention provides a method for processing fundus images. This embodiment focuses on describing a method for stitching fundus images. All methods used in this embodiment are single-channel processing unless otherwise specified. After the three RGB (Red Green Blue) channels are processed separately, they are then fused into a color image, which will not be elaborated further hereinafter.

[0045] Based on the above description, refer to Figure 4 the flowchart of a method for processing fundus images shown in

[0046] Step S402, collect multiple fundus strip images through a fundus camera.

[0047] Among them, adjacent fundus strip images may have an overlapping area in the number of rows, and the fundus strip images may have artifacts. In this embodiment, multiple fundus strip images can be collected through a fundus camera: Assume the vertical direction is the Y direction and the horizontal direction is the X direction. The fundus camera sequentially collects N rectangular fundus strip images with a width of w and a height of h in the Y direction from top to bottom (or from bottom to top). As an optional item, w = 4608 and h = 256.

[0048] As Figure 1 and Figure 2 shown, Figure 1 and Figure 2 show two adjacent fundus strip images ( Figure 1 is strip1, Figure 2 is strip2).

[0049] Among them, the fundus strip images have the following characteristics: (1) Two adjacent fundus strip images have a certain number of overlapping rows in the Y direction; (2) The fundus strip images are bright in the middle and dark on both sides in the Y direction; (3) Due to the optical system, artifacts usually appear in the center of the field of view, and the corresponding fundus strip images will contain artifacts. Refer to Figure 5 the schematic diagram of a fundus strip image with artifacts shown in

[0050] Step S404, regard adjacent fundus strip images as a group of images to be processed.

[0051] In this embodiment, two adjacent fundus strip images are sequentially selected as a group of images to be processed in the order from top to bottom or from bottom to top. For example: The fundus strip images sequentially collected by the fundus camera are strip1, strip2... strip N Then, it is possible to first obtain Figure 1 the strip1 shown in Figure 2For the strip2 shown, splice the bottom of strip1 and the top of strip2.

[0052] In some embodiments, the fundus strip image can also be cropped to determine a new fundus strip image, and the new fundus strip image is used to form an image to be processed. Among them, the cropping range of the fundus strip image is determined based on the brightness distribution of the fundus strip image.

[0053] Before splicing two adjacent fundus strip images, in this embodiment, the above two adjacent fundus strip images can also be appropriately cropped to obtain a new fundus strip image. Thus, before splicing two adjacent fundus strip images, a certain number of overlapping rows are still retained in the two adjacent fundus strip images. Among them, the specific number of cropped rows can be determined according to the brightness distribution of each fundus strip image.

[0054] For most fundus strip images, the typical brightness distribution in the Y direction is "bright in the middle, dark at the top and bottom", presenting a bright band (fundus area) in the middle and a black band at each of the top and bottom (almost no information), such as Figure 1 strip1 and Figure 2 strip2, and no cropping needs to be done on the above fundus strip images.

[0055] However, for the fundus strip images at the top and bottom (for example, if there are a total of 100 fundus strip images, then the fundus strip images at the top and bottom may be 1-10 and 91-100), due to optical design reasons, stray light will appear in these fundus strip images, and the presented effect is that on the basis of the typical brightness distribution, stray light will appear in the places that should be black bands. If these areas are not cropped and the processing method of this embodiment is adopted, this stray light will be introduced into the final image. Therefore, appropriate cropping needs to be done on the above fundus strip images, and the specific number of cropped rows can be determined according to the brightness distribution of each fundus strip image, and just try to crop all the rows where the stray light is located.

[0056] Step S406, determine the overlapping area and non-overlapping area of each group of images to be processed.

[0057] After determining the images to be processed, the overlapping area and non-overlapping area of each group of images to be processed can be determined first. For example: a group of images to be processed is Figure 1 strip1 shown and Figure 2 strip2 shown. Among them, the bottom of strip1 overlaps with the top of strip2. Therefore, the bottom of strip1 and the top of strip2 can be considered as the overlapping area, and the top of strip1 and the bottom of strip2 can be considered as the non-overlapping area.

[0058] In step S408, determine the pixel values of the target position of the overlapping region of each group of images to be processed in each fundus strip image of the group of images to be processed. Determine the pixel values of the target position of the overlapping region according to the pixel values in each fundus strip image, and obtain the target overlapping region according to the determined pixel values.

[0059] In this embodiment, the pixel values of the target position of the overlapping region of each group of images to be processed in each fundus strip image of the group can be determined, and then the pixel values of the target position of the overlapping region can be determined according to the pixel values in each fundus strip image.

[0060] In some embodiments, the larger value of the pixel values in each of the fundus strip images can be used as the pixel value of the target position of the overlapping region.

[0061] In this embodiment, the larger value of the pixel values (which can be simply referred to as candidate pixel values) in each fundus strip image is used as the pixel value of the target position of the overlapping region.

[0062] Taking Figure 1 strip1 and Figure 2 strip2 as an example, this embodiment can retain the non-overlapping regions of strip1 and strip2. For each pixel in the overlapping region of strip1 and strip2, this embodiment can take the larger of the two candidate pixel values at the target positions of strip1 and strip2 as the output, simply referred to as "taking the max". The result can be seen in Figure 6 a schematic diagram of a splicing result of a certain strip1 and strip2 shown.

[0063] In step S410, splice each group of non-overlapping regions and target overlapping regions according to the scanning information to obtain a fundus image.

[0064] Among them, the above scanning information may include: scanning position, scanning order, scanning time, etc. After completing the splicing of a group of images to be processed, this embodiment can sequentially re-select two adjacent fundus strip images as a new group of images to be processed for splicing in the previous order. After splicing all the fundus strip images, a complete fundus image can be obtained.

[0065] Referring to Figure 7 a schematic diagram of a fundus image spliced by the "hard splicing method" shown and Figure 8 a schematic diagram of a fundus image spliced by the "taking the max" method shown, Figure 8 the fundus image of Figure 7 compared with

[0066] For any position in the overlapping region of two fundus strip images, the method of splicing by "taking the maximum" provided in the embodiments of the present invention can take the larger of the two candidate pixel values as the output, so as to make full use of the effective luminance information and eliminate or greatly weaken the splicing stripes in the horizontal direction.

[0067] The embodiments of the present invention provide a method for processing fundus images, which includes collecting a plurality of fundus strip images through a fundus camera; taking adjacent fundus strip images as a group of images to be processed; determining the overlapping region and non-overlapping region of each group of images to be processed; determining the pixel values of the target position in the overlapping region of each group of images to be processed in each fundus strip image, determining the pixel value of the target position in the overlapping region according to the pixel values in each fundus strip image, and obtaining the target overlapping region according to the determined pixel values; splicing each group of non-overlapping regions and the target overlapping region according to the scanning information to obtain a fundus image.

[0068] Embodiment 2:

[0069] The present embodiment provides another method for processing fundus images, which is implemented on the basis of the above embodiments. The present embodiment focuses on describing a method for removing artifacts from fundus images. All methods adopted in the present embodiment are single-channel processing, unless otherwise specified. After the RGB three channels are processed separately, they are fused into a color image, which will not be elaborated hereinafter.

[0070] There are still some problems with the aforementioned "hard splicing" method: for fundus strip images containing artifacts, if the artifacts are not cropped, abnormal high-brightness regions will appear in the final large image.

[0071] Therefore, in the present embodiment, the fundus strip image with artifacts is called the target fundus strip image, and the target fundus strip image needs to be processed to remove artifacts first and then the image splicing in the aforementioned embodiments is performed.

[0072] Based on the above description, refer to Figure 9 the flowchart of another method for processing fundus images shown in

[0073] Step S902, collecting a plurality of background strip images through a fundus camera; wherein, the background strip images do not include the object being photographed.

[0074] The method for removing artifacts provided in the present embodiment depends on the pre-collected background strip images. In the present embodiment, a plurality of background strip images that do not include the object being photographed can be collected through a fundus camera. Among them, light-shielding paper can be placed on the lens cover of the fundus camera; a plurality of background strip images are collected through the fundus camera with the light-shielding paper covered.

[0075] Step S904, determine a target background strip image from multiple said background strip images.

[0076] In this embodiment, an artifact-containing background strip image can be directly determined from multiple background strip images as the target background strip image.

[0077] In some embodiments, an artifact-containing fundus strip image can be determined as the target fundus strip image, and a background strip image corresponding to the target fundus strip image is determined from multiple said background strip images as the target background strip image.

[0078] In addition to directly determining an artifact-containing background strip image from multiple said background strip images as the target background strip image as described above; in this embodiment, an artifact-containing fundus strip image can also be first determined from fundus strip images as the target fundus strip image, and then a background strip image corresponding to the target fundus strip image is determined from multiple said background strip images as the target background strip image. The target background strip image determined in this way is also an artifact-containing background strip image.

[0079] In some embodiments, a background strip image with the same shooting area as the target fundus strip image can be determined from multiple said background strip images as the target background strip image.

[0080] That is, whether directly determining an artifact-containing background strip image from multiple said background strip images as the target background strip image; or first determining an artifact-containing fundus strip image from fundus strip images as the target fundus strip image, and then determining a background strip image corresponding to the target fundus strip image from multiple said background strip images as the target background strip image. In this embodiment, the target fundus strip image and the target background strip image need to shoot the same area before the target background strip image can be used to remove artifacts from the target fundus strip image.

[0081] In some embodiments, a background strip image with a pixel value greater than a preset pixel threshold can be determined from multiple said background strip images as the target background strip image with artifacts; an artifact mask is determined according to the area where the pixel value in each said target background strip image is greater than the pixel threshold.

[0082] In this embodiment, a pixel threshold can be set. An area in the background strip image greater than the above pixel threshold is considered an artifact area, and the background strip image with an artifact area is the target background strip image. In addition, an artifact mask can be determined from the area where the pixel value in the target background strip image is greater than the pixel threshold.

[0083] In some embodiments, the above pixel threshold is determined based on the light intensity and exposure time of the background strip image collected by the fundus camera.

[0084] The pixel threshold in this embodiment can be strongly correlated with factors such as light intensity and exposure time. For the RGB channels, different thresholds need to be set. Here, reasonable thresholds can be determined according to experiments to ensure that there are no abnormalities in the processed image.

[0085] Among them, the above pixel threshold can be a number between 0 and 255. Different thresholds need to be set for the RGB channels. It can be simply understood that: the light intensities of the RGB three channels are different, so the intensities of the artifacts presented are also different. In the hardware configuration, the red light is very strong and the blue light is very weak. Then, the artifacts in the red channel (R channel) must be stronger than those in the blue channel (B channel). Therefore, the pixel threshold of the red channel should be higher. Therefore, for the RGB three channels, 3 pixel thresholds can be set respectively.

[0086] In some embodiments, the fundus camera can be used to collect background strip images corresponding to multiple preset diopters; each of the preset diopters corresponds to multiple background strip images; and multiple background strip images corresponding to the preset diopter closest to the target diopter corresponding to the target fundus strip image are determined.

[0087] The fundus strip image is collected by a doctor during actual use. When collecting the human eye, the doctor will select a diopter according to the situation of the human eye, and at this diopter, the image is the clearest. For example, 1D. Then, for the entire fundus image or each fundus strip image, regardless of the position, the diopter is 1D. Among them, for different fundus positions of the same eye, the diopter is the same.

[0088] The background strip image is usually not collected on-site by the doctor and can be considered to be pre-collected and stored in the device. Many groups of background strip images can be pre-collected, such as background strip images corresponding to 10 diopters (1D, 2D... 9D, 10D) respectively. In the background strip images collected at a certain diopter (such as 1D), for the entire background image or each background strip image, regardless of the position, the diopter is the same (both 1D).

[0089] The shapes and brightnesses of the artifacts at different diopters are different. Therefore, it is necessary to collect and save the background strip images at intervals of a certain diopter (such as 1D) to establish a background strip image database.

[0090] See Figure 10 A schematic diagram of a background strip image with artifacts is shown. At the same diopter, Figure 5 the artifacts of the fundus strip image in Figure 10The artifacts in the background strip image are consistent; when the diopter difference is large, the artifacts in the fundus strip image are very different from those in the background strip image. In actual use, it is best to use a background strip image with the same diopter as the fundus strip image for artifact removal. If there is no background strip image with the same diopter as the fundus strip image, then use the background strip image with the closest diopter to that of the fundus strip image.

[0091] In addition, the artifact removal method proposed in this embodiment has a certain diopter tolerance. When the background with the same diopter cannot be used, artifact removal can also be achieved. Refer to Figure 11 the schematic diagram of another background strip image with artifacts shown in Figure 11 The background strip image shown is collected at a diopter of -10D, Figure 5 The fundus strip image shown is collected at a diopter of -5D. In this embodiment, the background strip image shown in Figure 11 can be used for artifact removal of the fundus strip image shown in Figure 5

[0092] For example: when the doctor collects the fundus strip image corresponding to the target eye with a diopter of 1D, the background strip image collected at 1D diopter is automatically used for "artifact removal" processing, which is the most ideal situation. However, in practice, the situation that may occur is, for example, the collected target eye has a diopter of 1.3D, and there is no pre-collected background strip image with a diopter of 1.3D in the background library. At this time, the background strip image with the closest diopter, that is, the background strip image corresponding to 1D, is used for "artifact removal" processing; another example is that if the fundus strip image corresponding to the target eye with a diopter of 1.7D is collected, then the background strip image corresponding to 2D diopter is used for "artifact removal" processing.

[0093] Step S906, determine an artifact mask from each of the target background strip images; after dilating the artifact mask, perform a negation operation on the artifact mask to obtain a non-artifact mask.

[0094] In this embodiment, the artifact mask can be dilated first, and then the artifact mask is negated to obtain a non-artifact mask. Refer to Figure 12 the schematic diagram of a non-artifact mask shown in

[0095] The purpose of dilating the artifact mask is to ensure that all the highlighted artifacts are removed. If the artifact mask is not dilated, it may not be possible to remove all the highlighted artifacts, and the subsequent "take max" stitching will definitely expose the highlighted artifacts, resulting in a poor stitching effect.

[0096] ​In addition, in this embodiment, the non-artifact mask can also be edge-smoothed. As an optional preference, in this embodiment, the edges of the non-artifact mask can also be edge-smoothed to achieve a smooth transition between the artifact region and the non-artifact region. For example, image smoothing can be performed by filtering to perform edge smoothing.

[0097] Among them, edge smoothing is an image processing method. By adjusting the parameters of the smoothed edge, the shape edge can transition more smoothly, thereby reducing the distortion of the image.

[0098] Step S908: Based on each of the non-artifact masks, determine the non-artifact region of the corresponding fundus strip image, and obtain the fundus strip image after artifact removal processing.

[0099] After obtaining the non-artifact mask, in this embodiment, the non-artifact region of the fundus strip image can be extracted based on the non-artifact mask, and the fundus strip image after artifact removal processing can be obtained, thereby completing the artifact removal processing. Among them, the actual operation process of determining the non-artifact region of the target fundus strip image can be to multiply the non-artifact mask by the non-artifact region of the target fundus strip image.

[0100] Reference can be made to Figure 13 the schematic diagram of a fundus strip image after artifact removal processing shown in Figure 13 The fundus strip image shown after artifact removal processing is obtained by extracting the non-artifact region from the fundus strip image shown in Figure 5 using the non-artifact mask shown in Figure 12 shown.

[0101] After completing the artifact removal processing, in this embodiment, through the "take max" image stitching method provided by the foregoing embodiment, the fundus strip image after artifact removal processing can also be compensated.

[0102] In the artifact removal processing of this embodiment, the artifact region in the fundus strip image will be discarded. Therefore, when performing the "take max" image stitching provided by the embodiment after artifact removal, if there is a fundus strip image, there is a target region in the fundus strip image that has the same position as the artifact region in other fundus strip images, and the above target region of the fundus strip image is not covered by the artifact, then the above target region can be extracted from the fundus strip image, and the above target region image can be added to the fundus strip image after artifact removal processing, so as to obtain the compensated fundus strip image and complete the compensation for the fundus strip image after artifact removal processing during the image stitching process.

[0103] For example, for the same position in the fundus, it may appear in different fundus strip images, that is, the same position is photographed more than once. For instance, in the first photograph, strip1 is obtained, and there is an artifact at position (x1, y1). In the second photograph, strip2 is obtained, and (x2, y2) is in strip2. However, in this photograph, there may be no artifact at position (x2, y2). (x1, y1) and (x2, y2) correspond to the same position in the fundus. If there is no artifact at position (x2, y2), when splicing strip1 and strip2, the pixel value at the corresponding position of (x1, y1) and (x2, y2) in the spliced fundus image is the pixel value corresponding to the real pixel value (x2, y2), achieving the purpose of compensating the fundus strip image after artifact removal processing.

[0104] In addition, when the hardware configuration (including optics, system, etc.) is fixed, the positions where artifacts appear are also fixed. Therefore, it is possible to know in advance which fundus strip images contain artifacts, and only perform artifact removal processing on these fundus strip images. After that, perform the mosaic processing, and a fundus image without artifacts can be obtained. For the specific algorithm of the artifact removal processing in this embodiment, reference can be made to Figure 14 the schematic diagram of a specific algorithm for artifact removal processing shown.

[0105] For example: The height of the fundus area is 50 rows of pixels, and the height of the fundus strip image is 20 rows of pixels. The shooting process can be simply understood as scanning from top to bottom once. strip1: Shoot the 1st - 20th rows of the fundus area; strip2: Shoot the 11th - 30th rows of the fundus area; strip3: Shoot the 21st - 40th rows of the fundus area; strip4: Shoot the 31st - 50th rows of the fundus area. This shooting process is precisely controlled by mechanical and optical designs, so the relative positions of each fundus strip image can be accurately known. For example, the fundus area photographed by the 11th - 20th rows of pixels in strip1 must overlap with the fundus area photographed by the 1st - 10th rows of pixels in strip2.

[0106] The above - mentioned method for artifact removal processing provided by the embodiments of the present invention uses the background strip image, sets the pixel threshold, can detect the artifact mask to obtain the non - artifact mask, and the non - artifact mask extracts the non - artifact area of the fundus strip image, thus completing the artifact removal processing.

[0107] The above - mentioned method provided by the embodiments of the present invention optimizes the splicing and artifact removal methods of the fundus strip images of the fundus camera through the above - mentioned splicing method of "taking the maximum" and the above - mentioned method for artifact removal processing. The spliced fundus image is natural, has no splicing trace, and has no artifacts.

[0108] Embodiment Three:

[0109] Corresponding to the above method embodiments, an embodiment of the present invention provides a processing device for fundus images. Refer to Figure 15 the structural schematic diagram of a processing device for fundus images shown in

[0110] A fundus strip image acquisition module 1501, configured to acquire a plurality of fundus strip images through a fundus camera;

[0111] A to-be-processed image determination module 1502, configured to use adjacent fundus strip images as a group of to-be-processed images;

[0112] A to-be-processed image division module 1503, configured to determine the overlapping area and non-overlapping area of each group of to-be-processed images;

[0113] A pixel value determination module 1504, configured to determine the pixel values of the target position in the overlapping area of each group of to-be-processed images in each fundus strip image of the group of to-be-processed images, determine the pixel value of the target position in the overlapping area according to the pixel values in each fundus strip image, and obtain the target overlapping area according to the determined pixel values;

[0114] A to-be-processed image stitching module 1505, configured to stitch each group of non-overlapping areas and target overlapping areas according to the scanning information to obtain a fundus image.

[0115] An embodiment of the present invention provides a processing device for fundus images, which acquires a plurality of fundus strip images through a fundus camera; uses adjacent fundus strip images as a group of to-be-processed images; determines the overlapping area and non-overlapping area of each group of to-be-processed images; determines the pixel values of the target position in the overlapping area of each group of to-be-processed images in each fundus strip image of the group of to-be-processed images, determines the pixel value of the target position in the overlapping area according to the pixel values in each fundus strip image, and obtains the target overlapping area according to the determined pixel values; stitches each group of non-overlapping areas and target overlapping areas according to the scanning information to obtain a fundus image. In this way, the method of determining the pixel value of the target position in the overlapping area by using the pixel values in the fundus strip images of each group of to-be-processed images can make full use of the effective brightness information, thereby eliminating or significantly weakening the stitching stripes in the horizontal direction.

[0116] The above device further includes: a pixel value processing module, configured to use the larger value of the pixel values in each of the fundus strip images as the pixel value of the target position in the overlapping area.

[0117] The above device further includes: a fundus strip image cropping module, configured to crop the fundus strip image to determine a new fundus strip image, and the new fundus strip image is used to form the to-be-processed image, wherein the cropping range of the fundus strip image is determined based on the brightness distribution of the fundus strip image.

[0118] The above device further includes an artifact removal processing module, configured to collect a plurality of background strip images through the fundus camera; wherein, the background strip images do not include the object to be photographed; determine a target background strip image from the plurality of background strip images; determine an artifact mask from each of the target background strip images; expand the artifact mask and then perform a negation operation on the artifact mask to obtain a non-artifact mask; determine the non-artifact region of the corresponding fundus strip image based on each non-artifact mask, so as to obtain the fundus strip image after artifact removal processing.

[0119] The above artifact removal processing module is configured to determine a fundus strip image with artifacts as a target fundus strip image, and determine a background strip image corresponding to the target fundus strip image from the plurality of background strip images as a target background strip image.

[0120] The above artifact removal processing module is configured to determine a background strip image with the same shooting area as the target fundus strip image from the plurality of background strip images as a target background strip image.

[0121] The above artifact removal processing module is configured to collect a plurality of background strip images corresponding to a plurality of preset diopters through the fundus camera; each preset diopter corresponds to a plurality of background strip images; determine a plurality of background strip images corresponding to the preset diopter closest to the target diopter corresponding to the target fundus strip image.

[0122] The above artifact removal processing module is configured to determine a background strip image with a pixel value greater than a preset pixel threshold from the plurality of background strip images as a target background strip image with artifacts; determine an artifact mask according to the region where the pixel value is greater than the pixel threshold in each target background strip image.

[0123] The above pixel threshold is determined based on the light intensity and exposure time of the background strip image collected by the fundus camera.

[0124] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working process of the above-described fundus image processing device can refer to the corresponding process in the embodiment of the above-described fundus image processing method, which will not be repeated here.

[0125] Embodiment 4:

[0126] The embodiment of the present invention further provides an electronic device for running the above fundus image processing method; see Figure 16Schematic diagram of the structure of an electronic device shown. The electronic device includes a memory 100 and a processor 101. Among them, the memory 100 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor 101 to implement the above-mentioned fundus image processing method.

[0127] Furthermore, Figure 16 The electronic device shown further includes a bus 102 and a communication interface 103. The processor 101, the communication interface 103, and the memory 100 are connected through the bus 102.

[0128] Among them, the memory 100 may include a high-speed random access memory (RAM, Random Access Memory), and may also include a non-volatile memory, such as at least one disk memory. Through at least one communication interface 103 (which can be wired or wireless), a communication connection is established between this system network element and at least one other network element, and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 102 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 16 only a bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0129] The processor 101 may be an integrated circuit chip with the ability to process signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 101 or the instructions in the form of software. The above-mentioned processor 101 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 100, and the processor 101 reads the information in the memory 100 and combines its hardware to complete the steps of the method in the foregoing embodiments.

[0130] An embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions cause the processor to implement the above-mentioned method for processing fundus images. For specific implementation, reference may be made to the method embodiment, which will not be elaborated here.

[0131] The computer program product of the method and device for processing fundus images provided by the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method in the foregoing method embodiments. For specific implementation, reference may be made to the method embodiment, which will not be elaborated here.

[0132] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems and / or devices described above can refer to the corresponding processes in the foregoing method embodiments, which will not be elaborated here.

[0133] In addition, in the description of the embodiments of the present invention, unless otherwise clearly defined and limited, the terms "install", "connect", and "couple" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0134] If the above-mentioned functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0135] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0136] Finally, it should be noted that the above-mentioned embodiments are only specific implementation manners of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting them. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions described in the foregoing embodiments or easily conceive of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for processing fundus images, characterized in that, The method includes: Collecting a plurality of fundus strip images by a fundus camera; Regarding adjacent fundus strip images as a group of images to be processed; Determining the overlapping area and non-overlapping area of each group of the images to be processed; Determining the pixel values of the target position of the overlapping area of each group of the images to be processed in each of the fundus strip images of the group, determining the pixel value of the target position of the overlapping area according to the pixel values in each of the fundus strip images, and obtaining a target overlapping area according to the determined pixel values; Stitching each group of the non-overlapping areas and the target overlapping area according to scanning information to obtain a fundus image; The step of determining the pixel value of the target position of the overlapping area according to the pixel values in each of the fundus strip images includes: taking the larger value of the pixel values in each of the fundus strip images as the pixel value of the target position of the overlapping area; the target position includes each pixel of the overlapping area; The method further includes: collecting a plurality of background strip images by the fundus camera; wherein, the background strip images do not include the object to be photographed; determining a target background strip image from the plurality of background strip images; determining an artifact mask from each of the target background strip images; expanding the artifact mask and then performing an inversion operation on the artifact mask to obtain a non-artifact mask; determining a non-artifact area of the corresponding fundus strip image based on each of the non-artifact masks to obtain the fundus strip image after artifact removal processing.

2. The method according to claim 1, wherein After the step of collecting a plurality of fundus strip images by the fundus camera, the method further includes: Cropping the fundus strip images to determine new fundus strip images, and the new fundus strip images are used to form the images to be processed, wherein the cropping range of the fundus strip images is determined based on the brightness distribution of the fundus strip images.

3. The method according to claim 1, characterized in that The step of determining a target background strip image from the plurality of background strip images includes: Determining a fundus strip image with artifacts as a target fundus strip image, and determining a background strip image corresponding to the target fundus strip image from the plurality of background strip images as the target background strip image.

4. The method according to claim 3, characterized in that, The step of determining a background strip image corresponding to the target fundus strip image from the plurality of background strip images as the target background strip image includes: Determining a background strip image with the same shooting area as the target fundus strip image from the plurality of background strip images as the target background strip image.

5. The method according to claim 3, characterized in that, The step of collecting a plurality of background strip images by the fundus camera includes: Collecting a plurality of background strip images corresponding to a plurality of preset diopters by the fundus camera; each of the preset diopters corresponds to a plurality of the background strip images; Determining a plurality of the background strip images corresponding to the preset diopter closest to the target diopter corresponding to the target fundus strip image.

6. The method according to claim 1, wherein Determining a target background strip image from the plurality of background strip images; The step of determining an artifact mask from each of the target background strip images includes: Determine, from multiple background strip images, a background strip image with a pixel value greater than a preset pixel threshold as a target background strip image with artifacts; Determine an artifact mask based on the area with a pixel value greater than the pixel threshold in each target background strip image.

7. The method according to claim 6, wherein The pixel threshold is determined based on the light intensity and exposure time of the fundus camera for collecting the background strip image.

8. An apparatus for processing fundus images, characterized in that, The device includes: A fundus strip image acquisition module for acquiring multiple fundus strip images through a fundus camera; A to-be-processed image determination module for taking adjacent fundus strip images as a group of to-be-processed images; A to-be-processed image division module for determining the overlapping area and non-overlapping area of each group of the to-be-processed images; A pixel value determination module for determining the pixel values of the target positions in the overlapping area of each group of the to-be-processed images in each fundus strip image of the group, determining the pixel values of the target positions in the overlapping area according to the pixel values in each fundus strip image, and obtaining a target overlapping area based on the determined pixel values; A to-be-processed image stitching module for stitching each group of the non-overlapping areas and the target overlapping area according to the scanning information to obtain a fundus image; The pixel value determination module is used to take the larger value of the pixel values in each fundus strip image as the pixel value of the target position in the overlapping area; the target position includes each pixel in the overlapping area; The device further includes: an artifact removal processing module for acquiring multiple background strip images through the fundus camera; wherein, the background strip image does not include the object to be photographed; determining a target background strip image from multiple background strip images; determining an artifact mask from each target background strip image; performing an inversion operation on the artifact mask after expanding the artifact mask to obtain a non-artifact mask; determining a non-artifact area of the corresponding fundus strip image based on each non-artifact mask to obtain the fundus strip image after artifact removal processing.

Images