Eye fundus image processing method and device

Abstract

The invention provides an eye fundus image processing method and device. The method comprises the following steps: acquiring a plurality of fundus strip-shaped images through a fundus camera; taking adjacent fundus strip-shaped images as a group of images to be processed; determining an overlapping region and a non-overlapping region of each group of images to be processed; determining a pixel value of a target position of an overlapping region of each group of images to be processed in each fundus strip-shaped image in the group of images to be processed, determining a pixel value of the target position of the overlapping region according to the pixel value in each fundus strip-shaped image, and obtaining a target overlapping region according to the determined pixel value; and splicing each group of non-overlapping regions and the target overlapping region according to the scanning information to obtain an eye fundus image. In the mode, effective brightness information can be fully utilized, so that splicing stripes in the horizontal direction are eliminated or greatly weakened.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for processing fundus images. Background Art

[0002] There are a large number of patients with fundus diseases, which have a high risk of blindness and are irreversible, so large-scale rapid screening in the field of ophthalmology is very necessary. Fundus cameras are important instruments for the diagnosis and screening of ophthalmic diseases. They can observe and record the structures at the back of the eye, including the retina, choroid and optic disc. Fundus examination is essential for the early detection and treatment of ophthalmic diseases.

[0003] With the iteration of technology, fundus cameras have gradually developed from traditional optical fundus cameras to wide-angle fundus cameras and ultra-wide-angle fundus cameras. Since ultra-wide-angle fundus cameras have a large imaging field of view, segmented scanning and then stitching are usually used to obtain a complete fundus image. However, due to the optical system, the stitched image has obvious horizontal stitching stripes, and artifacts usually appear in the center of the field of view. Stitching stripes will reduce the quality of fundus images, thereby affecting clinical diagnosis.

[0004] To eliminate stitching stripes, the traditional method is to increase the light intensity or increase the number of segments. Increasing the light intensity can weaken the stitching stripes, but strong light exposure can make patients feel uncomfortable. Increasing the number of segments can make more parts of the fundus in the bright band, thereby reducing the impact of stitching stripes, but this will prolong the scanning time, increase the risk of eye movement, and reduce the success rate of the scan. At the same time, increasing the scanning time will also increase the total light intensity and reduce the patient's comfort. To eliminate artifacts, a common method is to avoid artifacts as much as possible through optical system design. However, this method cannot completely eliminate artifacts. Therefore, most of the ultra-wide-angle fundus cameras on the market currently have stitching marks and artifacts, and how to effectively optimize the stitching and artifact removal algorithms has become a necessity.

[0005] In the prior art, fundus images are generally stitched together by a "hard stitching" method, that is, multiple fundus strip images are stitched together in sequence to obtain a fundus image. However, when shooting two adjacent fundus strip images, the brightness distribution of the same area on the object being photographed is different in the two fundus strip images, so there will be obvious brightness discontinuity at the stitching boundary, with one side darker than the other, appearing as stitching stripes in the horizontal direction. Summary of the invention

[0006] In view of this, an object of the present invention is to provide a method and device for processing fundus images, so as to make full use of effective brightness information, thereby eliminating or significantly reducing the splicing stripes in the horizontal direction.

[0007] In a first aspect, an embodiment of the present invention provides a method for processing fundus images, the method comprising: acquiring multiple fundus strip images through a fundus camera; taking adjacent fundus strip images as a group of images to be processed; determining overlapping areas and non-overlapping areas of each group of images to be processed; determining the pixel value of the target position of the overlapping area of ​​each group of images to be processed in each fundus strip image in the group of images to be processed, determining the pixel value of the target position of the overlapping area according to the pixel value in each fundus strip image, and obtaining the target overlapping area according to the determined pixel value; splicing each group of non-overlapping areas and the target overlapping area according to scanning information to obtain a fundus image.

[0008] In an optional embodiment of the present application, the above-mentioned step of determining the pixel value of the target position of the overlapping area according to the pixel value in each of the fundus strip images includes: taking the larger value of the pixel value in each of the fundus strip images as the pixel value of the target position of the overlapping area.

[0009] In an optional embodiment of the present application, after the above-mentioned step of acquiring multiple fundus strip images through a fundus camera, the method further includes: cropping the fundus strip images to determine new fundus strip images, and the new fundus strip images are used to compose the image to be processed, wherein the range of cropping the fundus strip images is determined based on the brightness distribution of the fundus strip images.

[0010] In an optional embodiment of the present application, the above method also includes: collecting multiple background strip images through the fundus camera; wherein the background strip image does not include the photographed object; determining the target background strip image from the multiple background strip images; determining an artifact mask from each of the target background strip images; after expanding the artifact mask, performing an inversion operation on the artifact mask to obtain a non-artifact mask; based on each of the non-artifact masks, determining the corresponding non-artifact area of ​​the fundus strip image to obtain the fundus strip image after artifact removal processing.

[0011] In an optional embodiment of the present application, the above-mentioned step of determining the target background strip image from the multiple background strip images includes: determining a fundus strip image with artifacts as the target fundus strip image, and determining a background strip image corresponding to the target fundus strip image from the multiple background strip images as the target background strip image.

[0012] In an optional embodiment of the present application, the above-mentioned step of determining a background strip image corresponding to the target fundus strip image from a plurality of the background strip images as the target background strip image includes: determining the background strip image having the same shooting area as the target fundus strip image from a plurality of the background strip images as the target background strip image.

[0013] In an optional embodiment of the present application, the above-mentioned step of collecting multiple background strip images through the fundus camera includes: collecting multiple background strip images corresponding to preset refractive powers through the fundus camera; each of the preset refractive powers corresponds to multiple background strip images; and determining the multiple background strip images corresponding to the preset refractive power that is closest to the target refractive power corresponding to the target fundus strip image.

[0014] In an optional embodiment of the present application, the above-mentioned step of determining the target background strip image from the multiple background strip images; determining the artifact mask from each of the target background strip images, includes: determining the background strip image having a pixel value greater than a preset pixel threshold from the multiple background strip images as the target background strip image with artifacts; and determining the artifact mask based on the area in each of the target background strip images having a pixel value greater than the pixel threshold.

[0015] In an optional embodiment of the present application, the above pixel threshold is determined based on the light intensity and exposure time of the background strip image collected by the fundus camera.

[0016] In a second aspect, an embodiment of the present invention further provides a fundus image processing device, the device comprising: a fundus strip image acquisition module, used to acquire multiple fundus strip images through a fundus camera; a to-be-processed image determination module, used to treat adjacent fundus strip images as a group of images to be processed; a to-be-processed image division module, used to determine overlapping areas and non-overlapping areas of each group of images to be processed; a pixel value determination module, used to determine the pixel value of the target position of the overlapping area of ​​each group of images to be processed in each fundus strip image in the group of images to be processed, determine the pixel value of the target position of the overlapping area according to the pixel value in each fundus strip image, and obtain the target overlapping area according to the determined pixel value; and a to-be-processed image stitching module, used to stitch each group of non-overlapping areas and the target overlapping area according to scanning information to obtain a fundus image.

[0017] The embodiments of the present invention bring the following beneficial effects: The embodiment of the present invention provides a method and device for processing fundus images, which collects multiple fundus strip images through a fundus camera; takes adjacent fundus strip images as a group of images to be processed; determines the overlapping area and non-overlapping area of ​​each group of images to be processed; determines the pixel value of the target position of the overlapping area of ​​each group of images to be processed in each fundus strip image in the group of images to be processed, determines the pixel value of the target position of the overlapping area according to the pixel value in each fundus strip image, and obtains the target overlapping area according to the determined pixel value; splices each group of non-overlapping areas and the target overlapping area according to scanning information to obtain the fundus image. In this method, the method of determining the pixel value of the target position of the overlapping area by the pixel value in the fundus strip image of each group of images to be processed can make full use of the effective brightness information, thereby eliminating or greatly reducing the splicing stripes in the horizontal direction.

[0018] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.

[0019] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A schematic diagram of a fundus strip image strip1 provided by an embodiment of the present invention; Figure 2 A schematic diagram of a fundus strip image strip2 provided by an embodiment of the present invention; Figure 3 A schematic diagram of a puzzle image2 provided by an embodiment of the present invention; Figure 4 A flowchart of a method for processing fundus images provided by an embodiment of the present invention; Figure 5 A schematic diagram of a fundus strip image with artifacts provided by an embodiment of the present invention; Figure 6 A schematic diagram of a splicing result of strip1 and strip2 provided in an embodiment of the present invention; Figure 7A schematic diagram of a fundus image obtained by stitching using a "hard stitching" method provided by an embodiment of the present invention; Figure 8 A schematic diagram of a fundus image obtained by splicing using the "max" method provided in an embodiment of the present invention; Fig. 9 A flowchart of another method for processing fundus images provided by an embodiment of the present invention; Fig.10 A schematic diagram of a background strip image with artifacts provided by an embodiment of the present invention; Fig.11 A schematic diagram of another background strip image with artifacts provided by an embodiment of the present invention; Fig.12 A schematic diagram of a non-artifact mask provided by an embodiment of the present invention; Fig.13 A schematic diagram of a fundus strip image after artifact removal provided by an embodiment of the present invention; Fig.14 A schematic diagram of a specific algorithm for artifact removal processing provided by an embodiment of the present invention; Fig.15 A schematic diagram of the structure of a fundus image processing device provided by an embodiment of the present invention; Fig.16 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] At present, the fundus images are generally stitched by the "hard stitching" method, and multiple fundus strip images are stitched in sequence to obtain the fundus image. Assume that the multiple fundus strip images collected by the fundus camera in sequence are strip1, strip2...strip N , see Figure 1 A schematic diagram of a fundus strip image strip1 is shown and Figure 2 FIG. 1 is a schematic diagram of a fundus strip image strip2 shown in FIG. Figure 1 For strip1, Figure 2If the number of rows in the overlapping area is overlap_rows, then crop_bottom_rows rows at the bottom of strip1 and crop_top_rows rows at the top of strip2 can be cropped. Among them, crop_top_rows and crop_bottom_rows can be different for each strip, but it is necessary to ensure that crop_bottom_rows+crop_top_rows=overlap_rows. After the above cropping, the two fundus strip images have no overlapping areas and can be seamlessly spliced ​​without misalignment to obtain the puzzle image2, which can be seen in Figure 3 The schematic diagram of a puzzle image2 is shown in FIG. The same hard splicing process is performed on puzzle image2 and strip3 to obtain puzzle image3. Similarly, the same process is performed on all fundus strip images to obtain a complete fundus image image N .

[0024] However, this "hard stitching" method has some problems: when shooting two adjacent fundus strip images, the brightness distribution of the same area on the object is different in the two fundus strip images, so there will be obvious brightness discontinuity at the stitching boundary, with one side darker than the other, appearing as horizontal stitching stripes.

[0025] Based on this, an embodiment of the present invention provides a method and device for processing fundus images, which specifically provide a fundus image stitching method and a fundus image artifact removal method, which can make the processed fundus image more natural, without stitching traces and artifacts, avoid stitching stripes, have a significant artifact removal effect, and have the advantage of fast processing speed.

[0026] To facilitate understanding of this embodiment, a method for processing a fundus image disclosed in an embodiment of the present invention is first introduced in detail.

[0027] Embodiment 1: The embodiment of the present invention provides a method for processing fundus images. This embodiment focuses on describing a method for stitching fundus images. All methods used in this embodiment, unless otherwise specified, are single-channel processing. After the three channels of RGB (Red GreenBlue) are processed separately, they are fused into a color image, which will not be described in detail later.

[0028] Based on the above description, see Figure 4 A flowchart of a method for processing a fundus image is shown, and the method for processing a fundus image comprises the following steps: Step S402: collecting a plurality of fundus strip images by using a fundus camera.

[0029] Among them, adjacent fundus strip images may have overlapping areas in terms of the number of lines, and fundus strip images may have artifacts. In this embodiment, multiple fundus strip images can be collected by a fundus camera: assuming that the vertical direction is the Y direction and the horizontal direction is the X direction. The fundus camera collects N rectangular fundus strip images with a width of w and a height of h in the Y direction from top to bottom (or from bottom to top). As a preferred option, w=4608, h=256.

[0030] like Figure 1 and Figure 2 As shown, Figure 1 and Figure 2 Shown are two adjacent fundus strip images ( Figure 1 For strip1, Figure 2 for strip2).

[0031] Among them, the fundus strip image has the following characteristics: (1) Two adjacent fundus strip images overlap in the Y direction for a certain number of lines; (2) The fundus strip image is bright in the middle and dark on both sides in the Y direction; (3) Due to the optical system, artifacts usually appear in the center of the field of view, and the corresponding fundus strip image will contain artifacts, which can be seen in Figure 5 FIG. 1 is a schematic diagram of a fundus strip image with artifacts. However, not all fundus strip images necessarily have artifacts.

[0032] Step S404: taking adjacent fundus strip images as a group of images to be processed.

[0033] In this embodiment, two adjacent fundus strip images are selected in sequence from top to bottom or from bottom to top as a group of images to be processed. For example, the fundus strip images collected by the fundus camera in sequence are strip1, strip2, ... strip N , you can first obtain Figure 1 The strip1 and Figure 2 The strip2 shown joins the bottom of strip1 and the top of strip2.

[0034] In some embodiments, the fundus strip image may be cropped to determine a new fundus strip image, and the new fundus strip image is used to compose the image to be processed, wherein the range of cropping the fundus strip image is determined based on the brightness distribution of the fundus strip image.

[0035] Before splicing two adjacent fundus strip images, the present embodiment may also perform appropriate cropping on the two adjacent fundus strip images to obtain a new fundus strip image. This ensures that before splicing the two adjacent fundus strip images, the two adjacent fundus strip images still retain a certain number of rows of overlapping areas. The specific number of cropped rows may be determined based on the brightness distribution of each fundus strip image.

[0036] For most fundus strip images, the typical brightness distribution in the Y direction is "bright in the middle, dark above and below", showing a bright band in the middle (fundus area) and a black band above and below (almost no information), such as Figure 1 strip1 and Figure 2 For strip2, no cropping is required for the fundus strip image.

[0037] However, for the top and bottom fundus strip images (for example, if there are 100 fundus strip images in total, the top and bottom fundus strip images may be 1-10 and 91-100), due to optical design reasons, stray light will appear in these fundus strip images, and the effect presented is: on the basis of the typical brightness distribution, stray light will appear where the black band should be, and if these areas are not cropped, these stray lights will be introduced into the final image after the processing method of this embodiment is adopted. Therefore, the above fundus strip images need to be cropped appropriately, and the specific number of cropping rows can be determined according to the brightness distribution of each fundus strip image, and the rows where the stray light is located can be cropped as much as possible.

[0038] Step S406: determining the overlapping area and the non-overlapping area of ​​each group of images to be processed.

[0039] After determining the images to be processed, the overlapping area and non-overlapping area of ​​each group of images to be processed can be determined. For example: a group of images to be processed is Figure 1 The strip1 and Figure 2 The strip2 shown in FIG. 1 , wherein the bottom of strip1 overlaps with the top of strip2, and therefore, the bottom of strip1 and the top of strip2 can be considered as an overlapping area, and the top of strip1 and the bottom of strip2 can be considered as a non-overlapping area.

[0040] Step S408, determine the pixel value of the target position of the overlapping area of ​​each group of images to be processed in each fundus strip image in the group of images to be processed, determine the pixel value of the target position of the overlapping area according to the pixel value in each fundus strip image, and obtain the target overlapping area according to the determined pixel value.

[0041] This embodiment can determine the pixel value of the target position of the overlapping area of ​​each group of images to be processed in each fundus strip image in the group of images to be processed, and then determine the pixel value of the target position of the overlapping area according to the pixel value in each fundus strip image.

[0042] In some embodiments, the larger value of the pixel values ​​in each of the fundus strip images may be used as the pixel value of the target position of the overlapping area.

[0043] In this embodiment, a larger value of the pixel values ​​(which may be referred to as candidate pixel values) in each fundus strip image is used as the pixel value of the target position of the overlapping area.

[0044] by Figure 1 strip1 and Figure 2 For each pixel in the overlapping area of ​​strip1 and strip2, this embodiment can take the larger of the two candidate pixel values ​​at the target position of strip1 and strip2 as the output, which is referred to as "taking the max". The result can be seen in Figure 6 A schematic diagram of the splicing result of strip1 and strip2 is shown.

[0045] Step S410 , stitching each group of non-overlapping areas and the target overlapping area according to the scanning information to obtain a fundus image.

[0046] The scanning information may include: scanning position, scanning order, scanning time, etc. After completing the splicing of a group of images to be processed, the present embodiment may reselect two adjacent fundus strip images in sequence as a new group of images to be processed for splicing in the previous order, and after all fundus strip images are spliced, a complete fundus image may be obtained.

[0047] See also Figure 7 A schematic diagram of a fundus image obtained by stitching using the "hard stitching method" and Figure 8 The schematic diagram of a fundus image obtained by stitching using the "max" method is shown in FIG. Figure 8 Fundus images and Figure 7 In comparison, the splicing stripes in the horizontal direction can be obviously eliminated or greatly reduced.

[0048] The above-mentioned method of stitching by "taking the max" provided by an embodiment of the present invention can take the larger of the two candidate pixel values ​​as the output for any position in the overlapping area of ​​the two fundus strip images, thereby making full use of the effective brightness information and eliminating or greatly reducing the stitching stripes in the horizontal direction.

[0049] An embodiment of the present invention provides a method for processing fundus images, which includes collecting multiple fundus strip images through a fundus camera; taking adjacent fundus strip images as a group of images to be processed; determining overlapping areas and non-overlapping areas of each group of images to be processed; determining pixel values ​​of target positions of overlapping areas of each group of images to be processed in each fundus strip image in the group of images to be processed, determining pixel values ​​of target positions of overlapping areas according to pixel values ​​in each fundus strip image, and obtaining target overlapping areas according to the determined pixel values; and splicing each group of non-overlapping areas and target overlapping areas according to scanning information to obtain a fundus image.

[0050] Embodiment 2: This embodiment provides another method for processing fundus images, which is implemented on the basis of the above embodiment. This embodiment focuses on describing a method for removing artifacts from fundus images. All methods used in this embodiment, unless otherwise specified, are single-channel processing. After the RGB three channels are processed separately, they are fused into a color image, which will not be described in detail later.

[0051] There are still some problems with the aforementioned "hard stitching" method: for fundus strip images containing artifacts, if the artifacts are not cropped out, abnormal highlight areas will appear in the final large image.

[0052] Therefore, in this embodiment, the fundus strip image with artifacts is called a target fundus strip image. The target fundus strip image needs to be processed for artifact removal before the image stitching of the above embodiment is performed.

[0053] Based on the above description, see Fig. 9 A flowchart of another method for processing fundus images is shown, and the method for processing fundus images comprises the following steps: Step S902 : collecting a plurality of background strip images by a fundus camera; wherein the background strip images do not include the photographed object.

[0054] The artifact removal processing method provided in this embodiment relies on pre-collected background strip images. In this embodiment, multiple background strip images that do not include the photographed object can be collected by a fundus camera. Among them, matte paper can be covered on the lens of the fundus camera; multiple background strip images can be collected by the fundus camera covered with matte paper.

[0055] Step S904: determining a target background strip image from the plurality of background strip images.

[0056] In this embodiment, a background strip image with artifacts can be directly determined from a plurality of background strip images as a target background strip image.

[0057] In some embodiments, a fundus strip image with artifacts may be determined as a target fundus strip image, and a background strip image corresponding to the target fundus strip image may be determined from a plurality of background strip images as a target background strip image.

[0058] In addition to the above method of directly determining the background strip image with artifacts from the multiple background strip images as the target background strip image, in this embodiment, the fundus strip image with artifacts can be first determined from the fundus strip images as the target fundus strip image, and then the background strip image corresponding to the target fundus strip image can be determined from the multiple background strip images as the target background strip image. The target background strip image determined in this way is also a background strip image with artifacts.

[0059] In some embodiments, the background strip image having the same shooting area as the target fundus strip image may be determined from the plurality of background strip images as the target background strip image.

[0060] That is, whether directly determining a background strip image with artifacts from a plurality of the background strip images as a target background strip image, or first determining a fundus strip image with artifacts from a fundus strip image as a target fundus strip image, and then determining a background strip image corresponding to the target fundus strip image from a plurality of the background strip images as a target background strip image. In this embodiment, both the target fundus strip image and the target background strip image need to be photographed in the same area, and the target background strip image can be used to remove artifacts from the target fundus strip image.

[0061] In some embodiments, a background strip image having a pixel value greater than a preset pixel threshold can be determined from a plurality of the background strip images as a target background strip image having artifacts; and an artifact mask is determined based on an area in each of the target background strip images having a pixel value greater than the pixel threshold.

[0062] In this embodiment, a pixel threshold can be set, and the area in the background strip image that is greater than the pixel threshold can be considered as an artifact area, and the background strip image with the artifact area is the target background strip image. In addition, the area in the target background strip image with a pixel value greater than the pixel threshold can also be used to determine the artifact mask.

[0063] In some embodiments, the pixel threshold is determined based on the light intensity and exposure time of the fundus camera when collecting the background strip image.

[0064] The pixel threshold in this embodiment may be strongly related to factors such as light intensity and exposure time, and different thresholds need to be set for RGB channels. A reasonable threshold may be determined based on experiments to ensure that the processed image is free of abnormalities.

[0065] The pixel threshold can be a number between 0 and 255. Different thresholds need to be set for the RGB channels, which can be simply understood as follows: the illumination intensities of the three RGB channels are different, so the intensities of the artifacts are also different. If the red light in the hardware configuration is very strong and the blue light is very weak, then the artifacts of the red channel (R channel) must be stronger than those of the blue channel (B channel), so the pixel threshold of the red channel must be higher. Therefore, for the three RGB channels, three pixel thresholds can be set separately.

[0066] In some embodiments, the fundus camera can be used to capture multiple background strip images corresponding to preset diopters; each of the preset diopters corresponds to multiple background strip images; and the multiple background strip images corresponding to the preset diopters closest to the target diopters corresponding to the target fundus strip image are determined.

[0067] Fundus strip images are collected by doctors in actual use. When collecting images of human eyes, doctors will select a diopter based on the condition of the human eye. At this diopter, the image is clearest. For example, 1D, then for the entire fundus image or each fundus strip image, regardless of the position, the diopter is 1D. Among them, the diopter is the same for different fundus positions of the same eye.

[0068] Background strip images are usually not collected on-site by doctors, but can be considered to be pre-collected and stored in the device. Background strip images can be collected in many groups in advance, such as 10 groups of background strip images corresponding to diopters (1D, 2D...9D, 10D). In the background strip images collected under a set of diopters (such as 1D), the entire background image or each background strip image, regardless of the position, has the same diopters (all 1D).

[0069] The shapes and brightness of artifacts at different diopters are different, so it is necessary to collect and save background strip images at certain diopters (such as 1D) to establish a background strip image database.

[0070] See also Fig.10 Schematic diagram of a background strip image with artifacts. Figure 5 Artifacts of fundus strip images and Fig.10 The artifacts of the background strip image in the fundus strip image are consistent; when the refractive power difference is large, the artifacts in the fundus strip image and the artifacts in the background strip image are also very different. In actual use, it is best to use a background strip image with the same refractive power as the fundus strip image for artifact removal. If there is no background strip image with the same refractive power as the fundus strip image, use the background strip image with the closest refractive power to the fundus strip image.

[0071] In addition, the artifact removal method proposed in this embodiment has a certain refractive error tolerance. When the same refractive background cannot be used, artifact removal can also be achieved. Fig.11 A schematic diagram of another background stripe image with artifacts is shown. Fig.11 The background stripe image shown is acquired at -10D diopter. Figure 5 The fundus strip image shown is collected at -5D diopter. Fig.11 The background strip image shown is Figure 5 Fundus strip image artifact removal is shown.

[0072] For example: when the doctor collects the fundus strip image corresponding to the target eye with 1D refractive power, the background strip image collected under 1D refractive power is automatically used for "de-artifacting" processing, which is the most ideal situation. However, the actual situation may be that, for example, the target eye collected has a refractive power of 1.3D, and there is no pre-collected 1.3D background strip image in the background library. In this case, the background strip image with the closest refractive power, that is, the background strip image corresponding to 1D, is used for "de-artifacting" processing; for another example, if the fundus strip image corresponding to the target eye with 1.7D is collected, the background strip image corresponding to 2D refractive power is used for "de-artifacting" processing.

[0073] Step S906, determining an artifact mask from each of the target background strip images; and performing a negation operation on the artifact mask after dilating the artifact mask to obtain a non-artifact mask.

[0074] In this embodiment, the artifact mask may be first expanded, and then the artifact mask may be inverted to obtain a non-artifact mask. Fig.12 Schematic diagram of a non-artifact mask shown.

[0075] The purpose of dilating the artifact mask is to ensure that all highlight artifacts are removed. If the artifact mask is not dilated, it may not be possible to completely remove all highlight artifacts, and the subsequent "max" stitching will definitely expose the highlight artifacts, resulting in poor stitching results.

[0076] In addition, the embodiment can also soften the edge of the non-artifact mask. As a preferred option, the embodiment can also soften the edge of the non-artifact mask to achieve a smooth transition between the artifact area and the non-artifact area. For example, the image can be smoothed by filtering to soften the edge.

[0077] Among them, edge softening is an image processing method that can adjust the parameters of softened edges to make the transition of shape edges softer, thereby reducing image distortion.

[0078] Step S908 , determining the corresponding non-artifact area of ​​the fundus strip image based on each of the non-artifact masks, and obtaining the fundus strip image after artifact removal processing.

[0079] After obtaining the non-artifact mask, the present embodiment can extract the non-artifact area of ​​the fundus strip image based on the non-artifact mask to obtain the fundus strip image after artifact removal, thereby completing the artifact removal process. The actual operation process of determining the non-artifact area of ​​the target fundus strip image can be to multiply the non-artifact mask by the non-artifact area of ​​the target fundus strip image.

[0080] See also Fig.13 Schematic diagram of a fundus strip image after artifact removal processing, Fig.13 The fundus strip image after artifact removal is shown in FIG. Figure 5 The fundus strip image shown is Fig.12 The non-artifact mask shown is obtained by removing the non-artifact area.

[0081] After the artifact removal process is completed, this embodiment can also compensate the fundus stripe image after the artifact removal process through the "max" image stitching method provided in the above embodiments.

[0082] In this embodiment, the artifact area in the fundus strip image will be discarded during the artifact removal process. Therefore, when performing the "max" image stitching provided in the embodiment after the artifact removal, if there is a fundus strip image, and the fundus strip image has a target area at the same position as the artifact area in other fundus strip images, and the above target area of ​​the fundus strip image is not covered by the artifact, then the above target area can be taken from the fundus strip image, and the above target area image can be added to the fundus strip image after the artifact removal process, so as to obtain the compensated fundus strip image, and the compensation of the fundus strip image after the artifact removal process is completed in the process of image stitching.

[0083] For example, the same position of the fundus may appear in different fundus strip images, that is, the same position is photographed more than once. For example, strip1 is obtained in the first shot, and an artifact appears at position (x1, y1). Strip2 is obtained in the second shot, and (x2, y2) is in strip2. However, there may be no artifact at position (x2, y2) in this shot. (x1, y1) and (x2, y2) correspond to the same position of the fundus. If there is no artifact at position (x2, y2), when strip1 and strip2 are spliced, the pixel values ​​at the corresponding positions of (x1, y1) and (x2, y2) in the spliced ​​fundus image are the pixel values ​​corresponding to the real pixel values ​​(x2, y2), thus achieving the purpose of compensating the fundus strip image after artifact removal.

[0084] In addition, when the hardware configuration (including optics, system, etc.) is fixed, the location where the artifact appears is also fixed, so it is possible to know in advance which fundus strip images contain artifacts, and only these fundus strip images need to be de-artifacted. After that, the fundus image without artifacts can be obtained by performing the jigsaw puzzle. The specific algorithm of the de-artifacting process in this embodiment can be found in Fig.14 A schematic diagram of a specific algorithm for artifact removal is shown.

[0085] For example: the height of the fundus area is 50 lines of pixels, and the height of the fundus strip image is 20 lines of pixels. The shooting process can be simply understood as scanning from top to bottom. Strip1: shoots lines 1-20 of the fundus area; strip2: shoots lines 11-30 of the fundus area; strip3: shoots lines 21-40 of the fundus area; strip4: shoots lines 31-50 of the fundus area. This shooting process is precisely controlled by mechanical and optical design, so the relative position of each fundus strip image can be accurately known. For example, the fundus area shot by lines 11-20 of pixels in strip1 must overlap with the fundus area shot by lines 1-10 of pixels in strip2.

[0086] The above-mentioned artifact removal method provided by the embodiment of the present invention uses the background strip image and sets the pixel threshold to detect the artifact mask to obtain the non-artifact mask. The non-artifact mask takes out the non-artifact area of ​​the fundus strip image, thereby completing the artifact removal process.

[0087] The above method provided by the embodiment of the present invention optimizes the stitching and artifact removal method of the fundus strip image of the fundus camera through the above stitching method by "taking the max" and the above artifact removal method. The stitched fundus image is natural, has no stitching marks, and is free of artifacts.

[0088] Embodiment three: Corresponding to the above method embodiment, the present invention provides a fundus image processing device, see Fig.15 The schematic diagram of the structure of a fundus image processing device shown in FIG. 1 includes: A fundus strip image acquisition module 1501 is used to acquire multiple fundus strip images through a fundus camera; The to-be-processed image determination module 1502 is used to take adjacent fundus strip images as a group of to-be-processed images; The to-be-processed image division module 1503 is used to determine the overlapping area and the non-overlapping area of ​​each group of to-be-processed images; A pixel value determination module 1504 is used to determine the pixel value of the target position of the overlapping area of ​​each group of images to be processed in each fundus strip image in the group of images to be processed, determine the pixel value of the target position of the overlapping area according to the pixel value in each fundus strip image, and obtain the target overlapping area according to the determined pixel value; The image to be processed stitching module 1505 is used to stitch each group of non-overlapping areas and the target overlapping area according to the scanning information to obtain a fundus image.

[0089] The embodiment of the present invention provides a fundus image processing device, which collects multiple fundus strip images through a fundus camera; takes adjacent fundus strip images as a group of images to be processed; determines the overlapping area and non-overlapping area of ​​each group of images to be processed; determines the pixel value of the target position of the overlapping area of ​​each group of images to be processed in each fundus strip image in the group of images to be processed, determines the pixel value of the target position of the overlapping area according to the pixel value in each fundus strip image, and obtains the target overlapping area according to the determined pixel value; splices each group of non-overlapping areas and the target overlapping area according to scanning information to obtain the fundus image. In this method, the method of determining the pixel value of the target position of the overlapping area by the pixel value in the fundus strip image of each group of images to be processed can make full use of the effective brightness information, thereby eliminating or greatly weakening the splicing stripes in the horizontal direction.

[0090] The above-mentioned device also includes: a pixel value processing module, which is used to use the larger value of the pixel values ​​in each of the fundus strip images as the pixel value of the target position of the overlapping area.

[0091] The above-mentioned device also includes: a fundus strip image cropping module, which is used to crop the fundus strip image to determine a new fundus strip image, and the new fundus strip image is used to form the image to be processed, wherein the range of cropping the fundus strip image is determined based on the brightness distribution of the fundus strip image.

[0092] The above-mentioned device also includes: an artifact removal processing module, which is used to collect multiple background strip images through the fundus camera; wherein the background strip images do not include the photographed object; determine the target background strip image from the multiple background strip images; determine the artifact mask from each of the target background strip images; after expanding the artifact mask, perform an inversion operation on the artifact mask to obtain a non-artifact mask; based on each of the non-artifact masks, determine the corresponding non-artifact area of ​​the fundus strip image to obtain the fundus strip image after artifact removal processing.

[0093] The artifact removal processing module is used to determine a fundus strip image with artifacts as a target fundus strip image, and determine a background strip image corresponding to the target fundus strip image from a plurality of background strip images as a target background strip image.

[0094] The artifact removal processing module is used to determine, from the multiple background strip images, a background strip image having the same shooting area as the target fundus strip image as the target background strip image.

[0095] The above-mentioned artifact removal processing module is used to collect background strip images corresponding to multiple preset diopters through a fundus camera; each preset diopter corresponds to multiple background strip images; and determine multiple background strip images corresponding to the preset diopters that are closest to the target diopters corresponding to the target fundus strip image.

[0096] The above-mentioned artifact removal processing module is used to determine the background strip images with pixel values ​​greater than a preset pixel threshold from multiple background strip images as target background strip images with artifacts; and determine the artifact mask according to the area with pixel values ​​greater than the pixel threshold in each target background strip image.

[0097] The pixel threshold is determined based on the light intensity and exposure time of the background strip image collected by the fundus camera.

[0098] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the fundus image processing device described above can refer to the corresponding process in the aforementioned embodiment of the fundus image processing method, and will not be repeated here.

[0099] Embodiment 4: The embodiment of the present invention further provides an electronic device for executing the above-mentioned fundus image processing method; see Fig.16 A structural schematic diagram of an electronic device is shown, which includes a memory 100 and a processor 101, wherein the memory 100 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor 101 to implement the above-mentioned fundus image processing method.

[0100] Further, Fig.16 The electronic device shown further includes a bus 102 and a communication interface 103 , and the processor 101 , the communication interface 103 and the memory 100 are connected via the bus 102 .

[0101] The memory 100 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 102 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.16 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0102] The processor 101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 101. The above processor 101 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 100, and the processor 101 reads the information in the memory 100 and completes the steps of the method of the above embodiment in combination with its hardware.

[0103] An embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned fundus image processing method. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0104] The computer program product of the fundus image processing method and device provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method in the previous method embodiment. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0105] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and / or device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0106] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0107] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0108] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0109] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for processing fundus images, characterized in that: The method comprises: Acquiring multiple fundus strip images by a fundus camera; The adjacent fundus strip images are taken as a group of images to be processed; Determine the overlapping area and the non-overlapping area of ​​each group of the images to be processed; Determine the pixel value of the target position of the overlapping area of ​​each group of the images to be processed in each of the fundus strip images in the group of the images to be processed, determine the pixel value of the target position of the overlapping area according to the pixel value in each of the fundus strip images, and obtain the target overlapping area according to the determined pixel value; Each group of the non-overlapping areas and the target overlapping area are spliced ​​according to the scanning information to obtain a fundus image.

2. The method according to claim 1, characterized in that The step of determining the pixel value of the target position of the overlapping area according to the pixel value in each of the fundus strip images comprises: The larger value of the pixel values ​​in each of the fundus strip images is used as the pixel value of the target position of the overlapping area.

3. The method according to claim 1, characterized in that After the step of collecting a plurality of fundus strip images by a fundus camera, the method further includes: The fundus strip image is cropped to determine a new fundus strip image, where the new fundus strip image is used to form the image to be processed, wherein a range for cropping the fundus strip image is determined based on a brightness distribution of the fundus strip image.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Collecting a plurality of background strip images by the fundus camera; wherein the background strip images do not include the photographed object; Determine a target background strip image from the plurality of background strip images; Determine an artifact mask from each of the target background strip images; perform an inversion operation on the artifact mask after dilating the artifact mask to obtain a non-artifact mask; Based on each of the non-artifact masks, a non-artifact area of ​​the corresponding fundus strip image is determined to obtain the fundus strip image after artifact removal processing.

5. The method according to claim 4, characterized in that The step of determining a target background strip image from a plurality of the background strip images comprises: A fundus strip image with artifacts is determined as a target fundus strip image, and a background strip image corresponding to the target fundus strip image is determined from a plurality of background strip images as a target background strip image.

6. The method according to claim 5, characterized in that The step of determining a background strip image corresponding to the target fundus strip image from a plurality of the background strip images as a target background strip image comprises: The background strip image having the same photographing area as the target fundus strip image is determined from the plurality of background strip images as the target background strip image.

7. The method according to claim 5, characterized in that The step of collecting a plurality of background strip images by the fundus camera comprises: Collecting a plurality of background strip images corresponding to preset diopters through the fundus camera; each of the preset diopters corresponds to a plurality of the background strip images; A plurality of background strip images corresponding to the preset diopter closest to the target diopter corresponding to the target fundus strip image are determined.

8. The method according to claim 4, characterized in that Determine a target background strip image from the plurality of background strip images; The step of determining an artifact mask from each of the target background strip images comprises: Determine, from the plurality of background strip images, a background strip image having a pixel value greater than a preset pixel threshold as a target background strip image having artifacts; An artifact mask is determined according to a region in each of the target background strip images whose pixel value is greater than the pixel threshold.

9. The method according to claim 8, characterized in that The pixel threshold is determined based on the light intensity and exposure time of the fundus camera when collecting the background strip image.

10. A fundus image processing device, characterized in that: The device comprises: A fundus strip image acquisition module, used for acquiring multiple fundus strip images through a fundus camera; A module for determining images to be processed, used for taking adjacent fundus strip images as a group of images to be processed; A module for dividing images to be processed, used for determining overlapping areas and non-overlapping areas of each group of images to be processed; A pixel value determination module, used to determine the pixel value of the target position of the overlapping area of ​​each group of the images to be processed in each of the fundus strip images in the group of the images to be processed, determine the pixel value of the target position of the overlapping area according to the pixel value in each of the fundus strip images, and obtain the target overlapping area according to the determined pixel value; The image splicing module to be processed is used to splice each group of the non-overlapping areas and the target overlapping area according to the scanning information to obtain a fundus image.

Images