Image white-eye elimination method, device and equipment based on low-brightness infrared light supplement

Abstract

The present application belongs to the technical field of video image data processing, and solves the technical problem that the eye region of an infrared image taken under light compensation in a low-luminance environment has white eyes, resulting in discomfort when viewing the image, and provides an image white eye elimination method, device and equipment based on low-luminance infrared light compensation. The method comprises: obtaining an eye original image of a camera under infrared light compensation in a low-luminance environment, at which time the eye original image has a white eyeball portion, sending the eye original image to image segmentation of the eyeball portion and the non-eyeball portion, and then performing gray-scale fusion on a first eye image corresponding to the segmented eyeball portion and a second eye image corresponding to the non-eyeball portion to obtain a target image corresponding to a normal eye image; the present application segments the eyeball portion and the non-eyeball portion, performs gray-scale fusion to adapt the luminance of the eyeball portion and the non-eyeball portion, eliminates the white eye phenomenon, and avoids discomfort for users when viewing.

Description

Image white-eye elimination method, apparatus and equipment based on low-brightness infrared supplementary lighting Technical Field

[0001] This invention relates to the field of video data processing technology, and in particular to a method, apparatus, and device for eliminating white-eye in images based on low-brightness infrared illumination. Background Technology

[0002] With the continuous development of computer technology, electronic devices used for taking photos or recording videos have become an indispensable part of people's lives, such as mobile phones, tablets, cameras, and smart care devices.

[0003] When the ambient light of the target object is low, it is often necessary to use supplementary lighting, such as red light supplementary lighting. However, the infrared images or videos taken under supplementary lighting often appear abnormally white in the eye area due to specular reflection of the eyeball, causing great discomfort to the viewer. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method, apparatus and device for eliminating white eye in images based on low-brightness infrared supplementary lighting, in order to solve the technical problem that white eye exists in the eye area of ​​infrared images taken under supplementary lighting in existing low-brightness environments, causing discomfort when viewing the images.

[0005] The technical solution adopted in this invention is:

[0006] This invention provides a method for eliminating white-eye in images based on low-brightness infrared illumination, the method comprising:

[0007] S1: Acquire the original image of the eye when using infrared light to supplement illumination in a low-brightness environment;

[0008] S2: Based on the eye region and non-eye region of the original eye image, the image is partitioned to obtain a first eye image corresponding to the white eye region and a second eye image corresponding to the light-colored non-eye region;

[0009] S3: Perform grayscale fusion on the first eye image and the second eye image to obtain a target eye image corresponding to the normal eye image.

[0010] Preferably, S1 includes:

[0011] S11: Acquire an infrared image when supplementing low-brightness environments with infrared light, wherein the infrared image includes an eye image;

[0012] Specifically, infrared images refer to the frames of images in a video stream captured by a camera under a 940nm infrared fill light;

[0013] S12: Perform grayscale processing on the infrared image to obtain a grayscale image corresponding to the infrared image;

[0014] S13: Perform target detection on the grayscale image and segment the original eye image containing the eye region.

[0015] Preferably, S2 includes:

[0016] S21: Convert the original eye image into a histogram;

[0017] S22: Based on the pixels at the valley positions of the histogram, divide the histogram into a left histogram and a right histogram;

[0018] S23: Calculate the average and variance values ​​of the pixel values ​​of the pixels in the left histogram and / or the right histogram, respectively;

[0019] S24: Compare the pixel values ​​of each pixel in the original eye image with the variance and the average value to obtain each first pixel corresponding to the white eyeball area and each second pixel corresponding to the light-colored non-eyeball area;

[0020] The first eye image is composed of all first pixels, and the second eye image is composed of all second pixels.

[0021] Preferably, S22 includes:

[0022] S221: Obtain the pixel value P corresponding to the pixel at the valley position of the histogram;

[0023] S222: Based on the pixel value P, traverse the pixel values ​​of each pixel in the histogram to obtain the left histogram corresponding to each first pixel value with a pixel value less than P and the right histogram corresponding to each second pixel value with a pixel value greater than P.

[0024] Preferably, S24 includes:

[0025] S241: Obtain the first average value and the first variance value corresponding to the pixel values ​​of the left histogram pixels, and the second average value and the second variance value corresponding to the pixel values ​​of the right histogram pixels;

[0026] S242: Based on the pixel values ​​of each pixel in the original eye image, use the formula... Obtain each second pixel;

[0027] S243: Based on the pixel value X of each pixel in the original eye image, use the formula... Obtain the first pixel;

[0028] Where n is a constant, K1 is the first average value, Q1 is the first variance value, K2 is the second average value, Q2 is the second variance value, and X is the pixel value of a pixel in the original eye image.

[0029] Preferably, S24 includes:

[0030] S244: Randomly obtain at least one first target pixel with a pixel value of a first average value and at least one second target pixel with a pixel value of a second average value;

[0031] S245: Based on the first target pixel and the second target pixel, perform edge detection on the grayscale image in a preset tracking direction to obtain the contour region and the region within the contour corresponding to the tracking path of the first target pixel and the second target pixel;

[0032] S246: Repeat S244 to S245 until the first eye image corresponding to each contour region of all the first target pixels and all the second target pixels and the second eye image corresponding to each contour region are obtained.

[0033] Preferably, S3 includes:

[0034] S31: Obtain the first pixel average value corresponding to the pixel values ​​of all pixels in the first eye image and the second pixel average value corresponding to the pixel values ​​of all pixels in the second eye image;

[0035] S32: Based on the average value of the first pixel, use the formula Update the pixel values ​​of each pixel in the first eye image;

[0036] S33: Based on the average value of the second pixel, use the formula Update the pixel values ​​of each pixel in the second eye image;

[0037] S34: Obtain the target eye image based on the updated first eye image and the second eye image;

[0038] Where A(x, y) are the coordinates of a pixel in the first eye image, B(x, y) are the coordinates of a pixel in the second eye image, and a x Let a be the x-coordinate of the center point of the first eye image. y Let q1 be the ordinate of the center point of the first eye image, q2 be the average value of the first pixel, and q3 be the average value of the first pixel.

[0039] The present invention also provides an image white-eye elimination device based on low-brightness infrared supplementary light, comprising:

[0040] Image acquisition module: used to acquire raw images of the eye when using infrared light to supplement illumination in low-brightness environments;

[0041] Image segmentation module: used to partition the image based on the eye region and non-eye region of the original eye image to obtain a first eye image corresponding to the white eye region and a second eye image corresponding to the light-colored non-eye region;

[0042] Image fusion module: used to perform grayscale fusion on the first eye image and the second eye image to obtain a target eye image corresponding to the normal eye image.

[0043] The present invention also provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, wherein the computer program instructions, when executed by the processor, implement the method described in any of the preceding embodiments.

[0044] The present invention also provides a medium having computer program instructions stored thereon, which, when executed by a processor, implement the method described in any of the preceding claims.

[0045] In summary, the beneficial effects of the present invention are as follows:

[0046] This invention provides a method, apparatus, and device for eliminating white-eye in images based on low-brightness infrared supplemental lighting. The method involves acquiring a raw image of the eye in a low-brightness environment under infrared supplemental lighting. In this raw image, the eyeball appears white. The raw image is then segmented into the eyeball portion and the non-eyeball portion. The first eye image corresponding to the segmented eyeball portion and the second eye image corresponding to the non-eyeball portion are then fused in grayscale to obtain a target image corresponding to a normal eye image. This invention eliminates the white-eye phenomenon by segmenting the eyeball portion and the non-eyeball portion and then performing grayscale fusion, thus ensuring brightness matching between the two portions and preventing discomfort for the user. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, and these are all within the protection scope of the present invention.

[0048] Figure 1 is a schematic flowchart of the image white-eye elimination method based on low-brightness infrared supplementary light in Embodiment 1 of the present invention;

[0049] Figure 2 is a schematic diagram of the process of obtaining the original eye image in Embodiment 1 of the present invention;

[0050] Figure 3 is a schematic diagram of the process of obtaining the pixel values ​​of each pixel in the eye image in Embodiment 1 of the present invention;

[0051] Figure 4 is a schematic diagram of the image white-eye elimination device based on low-brightness infrared supplementary light in Embodiment 2 of the present invention;

[0052] Figure 5 is a schematic diagram of the electronic device in Embodiment 3 of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Where there is no conflict, the various features of this invention and its embodiments can be combined with each other, all of which are within the scope of protection of this invention.

[0054] Example 1

[0055] Please refer to Figure 1, which illustrates the image white-eye elimination method based on low-brightness infrared supplementary lighting in Embodiment 1 of the present invention; the method includes:

[0056] S1: Acquire the original image of the eye when using infrared light to supplement illumination in a low-brightness environment;

[0057] S2: Based on the eye region and non-eye region of the original eye image, the image is partitioned to obtain a first eye image corresponding to the white eye region and a second eye image corresponding to the light-colored non-eye region;

[0058] S3: Perform grayscale fusion on the first eye image and the second eye image to obtain a target eye image corresponding to the normal eye image.

[0059] Specifically, when ambient light is insufficient, such as at night, on cloudy days, at dusk, or when shooting in backlight, the camera needs to capture scene images under the supplementary lighting of an 850nm-950nm infrared fill light, preferably 940nm infrared light. If the scene image captured at this time contains the eye area of ​​the target object, the brightness of the eye image will be too high due to the reflection of the eyeball, resulting in an abnormal original eye image. This original eye image is then segmented into a first eye image corresponding to the eyeball area and a second eye image corresponding to the non-eyeball area. First, the captured infrared image is converted into a grayscale image. The image of the eyeball area and the non-eyeball area are extracted using histogram and variance domain. Then, the grayscale of the first eye image and the second eye image are fused to make the grayscale of the eyeball area similar to that of a normal eyeball. At the same time, the grayscale of the adjusted eye area is adapted to the grayscale of the non-eyeball area, thereby obtaining a normal target image of the eyeball area and the non-eyeball area.

[0060] In one embodiment, referring to FIG2, S1 includes:

[0061] S11: Acquire an infrared image when supplementing low-brightness environments with infrared light, wherein the infrared image includes an eye image;

[0062] Specifically, infrared images refer to the frames of images in a video stream captured by a camera under a 940nm infrared fill light;

[0063] S12: Perform grayscale processing on the infrared image to obtain a grayscale image corresponding to the infrared image;

[0064] S13: Perform target detection on the grayscale image and segment the original eye image containing the eye region.

[0065] Specifically, when capturing images of low-light environments, infrared light is used to supplement the low-light environment, resulting in an infrared image based on infrared light supplementation. This infrared image includes an image of the eyes of a person or animal. The infrared image is then converted into a grayscale image, and an eye detection network is used for target detection to obtain an original eye image containing the eyeball. This eye detection network is trained based on the YoloV5-S detection algorithm. It should be noted that the infrared image is any frame in the target video continuously recorded by the camera that has infrared light supplementation. The corresponding grayscale image is the Y channel image extracted directly from the underlying YUV channel of the target video as the grayscale image of the corresponding frame.

[0066] In one embodiment, referring to FIG3, S2 includes:

[0067] S21: Convert the original eye image into a histogram;

[0068] S22: Based on the pixels at the valley positions of the histogram, divide the histogram into a left histogram and a right histogram;

[0069] S23: Calculate the average and variance values ​​of the pixel values ​​of the pixels in the left histogram and / or the right histogram, respectively;

[0070] S24: Compare the pixel values ​​of each pixel in the original eye image with the variance and the average value to obtain each first pixel corresponding to the white eyeball area and each second pixel corresponding to the light-colored non-eyeball area;

[0071] The first eye image is composed of all first pixels, and the second eye image is composed of all second pixels.

[0072] Specifically, the original eye image is converted into a histogram. Utilizing the distinct dark and bright areas of the histogram, pixels at the troughs are divided into a left histogram corresponding to the dark area and a right histogram corresponding to the bright area. This quickly and initially determines the pixel segmentation line between the eye region and the non-eye region. Then, using the pixel values ​​of this segmentation line as a reference, combined with the average and variance values ​​of the pixel values ​​in the left and right histograms, the original eye image is further subdivided into first and second pixels, achieving rapid and accurate segmentation of the eye region and the non-eye region.

[0073] In one embodiment, S22 includes:

[0074] S221: Obtain the pixel value P corresponding to the pixel at the valley position of the histogram;

[0075] S222: Based on the pixel value P, traverse the pixel values ​​of each pixel in the histogram to obtain the left histogram corresponding to each first pixel value with a pixel value less than P and the right histogram corresponding to each second pixel value with a pixel value greater than P.

[0076] Specifically, pixels with pixel values ​​less than those of the trough pixels in the histogram are classified as dark area pixels, i.e., the histogram to the left of the trough; pixels with pixel values ​​greater than those of the trough pixels in the histogram are classified as bright area pixels, i.e., the histogram to the right of the trough. This yields the basic number and size of pixels corresponding to the first pixel value in the non-eye region, and the basic number and size of pixels corresponding to the second pixel value in the eye region.

[0077] In one embodiment, S24 includes:

[0078] S241: Obtain the first average value and the first variance value corresponding to the pixel values ​​of the left histogram pixels, and the second average value and the second variance value corresponding to the pixel values ​​of the right histogram pixels;

[0079] S242: Based on the pixel values ​​of each pixel in the original eye image, use the formula... Obtain each second pixel;

[0080] S243: Based on the pixel value X of each pixel in the original eye image, use the formula... Obtain the first pixel;

[0081] Where n is a constant, K1 is the first average value, Q1 is the first variance value, K2 is the second average value, Q2 is the second variance value, and X is the pixel value of a pixel in the original eye image.

[0082] Specifically, the original eye image is converted into a histogram, and the troughs in the histogram are calculated. Based on these troughs, the histogram is divided into a left histogram and a right histogram. The left and right histograms are defined by the pixel values ​​of corresponding pixels and the pixel values ​​P at the troughs. Specifically, the pixel values ​​P at the troughs are used to iterate through the pixel values ​​of all pixels. Pixels with values ​​greater than P form the right histogram, and pixels with values ​​less than or equal to P form the left histogram. That is, the pixel values ​​in the left histogram are [0, 1, 2, ..., P-2, P-1, P], and the pixel values ​​in the right histogram are [P+1, P+2, ..., 254, 255]. The average and variance of the pixel values ​​in the left and / or right histograms are calculated respectively. Since the average grayscale value of the white eye region differs from that of the non-eye region by approximately 200, Formula 1 is used... And Formula 2 The system categorizes each pixel to obtain the first pixels of the eye region and the second pixels of the non-eye region. Specifically, when the pixel value satisfies Formula 1, the pixel is assigned to the second eye image in the non-eye region. When the pixel value does not satisfy Formula 1 but satisfies Formula 2, the pixel is assigned to the first eye image in the eye region. When the pixel value satisfies neither Formula 1 nor Formula 2, its position determines whether it belongs to the eye region or the non-eye region. Further region division based on histograms is then performed to achieve region division for all pixels, preventing abnormal pixels in the eye region from being assigned to the non-eye region, and vice versa.

[0083] In one embodiment, S24 includes:

[0084] S244: Randomly obtain at least one first target pixel with a pixel value of a first average value and at least one second target pixel with a pixel value of a second average value;

[0085] S245: Based on the first target pixel and the second target pixel, perform edge detection on the grayscale image in a preset tracking direction to obtain the contour region and the region within the contour corresponding to the tracking path of the first target pixel and the second target pixel;

[0086] S246: Repeat S244 to S245 until the first eye image corresponding to each contour region of all the first target pixels and all the second target pixels and the second eye image corresponding to each contour region are obtained.

[0087] In one embodiment, S246 includes:

[0088] S2461: Obtain the area of ​​each region within the contour;

[0089] S2462: Compare the area of ​​each region within the contour, and output the region within the contour corresponding to the largest area as the first eye image.

[0090] Specifically, based on the first average value corresponding to the pixel values ​​of the pixels in the left histogram, at least one pixel matching the first average value is randomly selected from the original eye image as the first target pixel. Similarly, at least one pixel matching the second average value is randomly selected as the second target pixel. It should be noted that: preferably, the pixel value equal to the first average value is used as the first target pixel, and the pixel value equal to the second average value is used as the second target pixel. When there is no pixel value equal to the first average value and / or the second average value, the pixel value adjacent to the first average value is used as the first target pixel, and the pixel value adjacent to the second average value is used as the second target pixel. For example, if the first average value is 100.5, then the first target pixel is the pixel value equal to 100 or 101; similarly, if the second average value is 200.5, then the second target pixel is the pixel value equal to 200 or 201. After determining the first and second target pixels, tracking is performed in a predetermined direction, including directions that do not... Limited to: clockwise, counterclockwise, and straight-line back-and-forth movements; the specific tracking method is as follows: the first target pixel is tracked in a predetermined direction, continuously connecting pixels that meet the pixel value requirements to form contour regions. When a neighborhood contour exists, tracking stops, and the next first target pixel value and second target pixel value are tracked, ultimately obtaining each contour region and the region within the contour. The images of all regions within the contour are recorded as the first eye image, and the images of all contour regions are recorded as the second eye image. Furthermore, due to the special nature of eye images, when delineating the eyeball region and non-eyeball region, the contour area of ​​each region within the contour can be directly determined. The image corresponding to the largest area within the contour is determined as the eyeball region, and the region outside is recorded as the non-eyeball region, which can reduce the amount of data processing and improve detection efficiency. When all first target pixels and all second target pixels correspond to a contour image, the segmentation of the eyeball region and non-eyeball region of the original eye image is completed. This method can avoid omissions or misdrawing of the transition area between the eyeball region and non-eyeball region, improving the segmentation effect of the eyeball region and non-eyeball region.

[0091] In one embodiment, S3 includes:

[0092] S31: Obtain the first pixel average value corresponding to the pixel values ​​of all pixels in the first eye image and the second pixel average value corresponding to the pixel values ​​of all pixels in the second eye image;

[0093] S32: Based on the average value of the first pixel, use the formula Update the pixel values ​​of each pixel in the first eye image;

[0094] S33: Based on the average value of the second pixel, use the formula Update the pixel values ​​of each pixel in the second eye image;

[0095] S34: Obtain the target eye image based on the updated first eye image and the second eye image;

[0096] Where A(x, y) are the coordinates of a pixel in the first eye image, B(x, y) are the coordinates of a pixel in the second eye image, and a x Let a be the x-coordinate of the center point of the first eye image. y Let q1 be the ordinate of the center point of the first eye image, q2 be the average value of the first pixel, and q3 be the average value of the first pixel.

[0097] Specifically, after segmenting the eyeball into the non-eyeball portion, the average pixel value of all pixels in the eyeball portion is used to calculate the result using the formula... The pixel values ​​of each pixel in the eye area are updated to convert the pixel values ​​of the eye area to grayscale values ​​of the non-eye area. This is achieved by averaging the pixel values ​​of all pixels in the non-eye area using the formula... The pixel values ​​of each pixel in the non-eye area are updated, and the gray values ​​of the non-eye area are converted based on the gray values ​​of the previous eye area and the distance. The target eye image is composed of the pixels with updated pixel values, eliminating the white-eye phenomenon in the original eye image.

[0098] In one embodiment, the process further includes the following after S3:

[0099] S4: Obtain the grayscale image;

[0100] S5: Merge the target eye image with the eye region of the grayscale image to obtain the target grayscale image;

[0101] S6: Restore the target grayscale image to obtain a target infrared image containing a normal eye image.

[0102] Specifically, after the pixel values ​​of the eye region and the non-eye region are updated, the eye region and the non-eye region are merged to obtain the target grayscale image of the eye region. Then, the pixels in the original grayscale image are replaced according to the corresponding pixel positions to obtain the grayscale image corresponding to the normal eye region, and further, the infrared image of the normal eye region is obtained.

[0103] The image white-eye elimination device based on low-brightness infrared supplementary lighting in this embodiment acquires the original eye image of a low-brightness environment under infrared supplementary lighting. At this time, the original eye image shows that the eyeball is white. The original eye image is sent for image segmentation of the eyeball part and the non-eyeball part. Then, the first eye image corresponding to the segmented eyeball part and the second eye image corresponding to the non-eyeball part are fused in grayscale to obtain the target image corresponding to the normal eye image. The present invention eliminates the white-eye phenomenon by segmenting the eyeball part and the non-eyeball part and then performing grayscale fusion to make the brightness of the eyeball part and the non-eyeball part match, thereby avoiding discomfort to the user.

[0104] Example 2

[0105] The present invention also provides an image white-eye elimination device based on low-brightness infrared supplementary light, as shown in Figure 4, comprising:

[0106] Image acquisition module: used to acquire raw images of the eye when using infrared light to supplement illumination in low-brightness environments;

[0107] Image segmentation module: used to partition the image based on the eye region and non-eye region of the original eye image to obtain a first eye image corresponding to the white eye region and a second eye image corresponding to the light-colored non-eye region;

[0108] Image fusion module: used to perform grayscale fusion on the first eye image and the second eye image to obtain a target eye image corresponding to the normal eye image.

[0109] The image white-eye elimination device based on low-brightness infrared supplementary lighting in this embodiment acquires the original eye image of a low-brightness environment under infrared supplementary lighting. At this time, the original eye image shows that the eyeball is white. The original eye image is sent for image segmentation of the eyeball part and the non-eyeball part. Then, the first eye image corresponding to the segmented eyeball part and the second eye image corresponding to the non-eyeball part are fused in grayscale to obtain the target image corresponding to the normal eye image. The present invention eliminates the white-eye phenomenon by segmenting the eyeball part and the non-eyeball part and then performing grayscale fusion to make the brightness of the eyeball part and the non-eyeball part match, thereby avoiding discomfort to the user.

[0110] In one embodiment, the image acquisition module includes:

[0111] Infrared image unit: acquires infrared images when supplementing low-brightness environments with infrared light, wherein the infrared images include eye images;

[0112] Grayscale processing unit: performs grayscale processing on the infrared image to obtain a grayscale image corresponding to the infrared image;

[0113] Image segmentation unit: performs target detection on the grayscale image and segments out the original eye image containing the eye region.

[0114] In one embodiment, the image segmentation module includes:

[0115] Image conversion unit: converts the original eye image into a histogram;

[0116] Histogram segmentation unit: Based on the pixels at the valley positions of the histogram, the histogram is divided into a left histogram and a right histogram;

[0117] Pixel value calculation unit: calculates the average and variance values ​​of the pixel values ​​of the pixels in the left histogram and / or the right histogram, respectively;

[0118] Pixel classification unit: compares the pixel value of each pixel in the original eye image with the variance value and the average value to obtain each first pixel corresponding to the white eyeball area and each second pixel corresponding to the light-colored non-eyeball area;

[0119] The first eye image is composed of all first pixels, and the second eye image is composed of all second pixels.

[0120] In one embodiment, the histogram segmentation unit includes:

[0121] Pixel acquisition unit: acquires the pixel value P corresponding to the pixel at the valley position of the histogram;

[0122] Pixel processing unit: Based on the pixel value P, it traverses the pixel values ​​of each pixel in the histogram to obtain the left histogram corresponding to each first pixel value with a pixel value less than P and the right histogram corresponding to each second pixel value with a pixel value greater than P.

[0123] In one embodiment, the pixel classification unit includes:

[0124] Pixel value acquisition unit: acquires the first average value and the first variance value corresponding to the pixel value of the left histogram pixel, and the second average value and the second variance value corresponding to the pixel value of the right histogram pixel;

[0125] First pixel value calculation unit: Based on the pixel values ​​of each pixel in the original eye image, using the formula... Obtain each second pixel;

[0126] Second pixel value calculation unit: Based on the pixel value X of each pixel in the original eye image, using the formula... Obtain the first pixel;

[0127] Where n is a constant, K1 is the first average value, Q1 is the first variance value, K2 is the second average value, Q2 is the second variance value, and X is the pixel value of a pixel in the original eye image.

[0128] In one embodiment, the pixel classification unit includes:

[0129] Random acquisition unit: randomly acquires at least one first target pixel with a pixel value of a first average value and at least one second target pixel with a pixel value of a second average value;

[0130] Pixel tracking unit: Based on the first target pixel and the second target pixel, edge detection is performed on the grayscale image in a preset tracking direction to obtain the contour region and the region within the contour corresponding to the tracking path of the first target pixel and the second target pixel;

[0131] Looping tracking unit: Repeat the above steps until the first eye image corresponding to each contour region of all the first target pixels and all the second target pixels and the second eye image corresponding to each contour region are obtained.

[0132] In one embodiment, the image fusion module includes:

[0133] Parameter acquisition unit: acquires the first pixel average value corresponding to the pixel values ​​of all pixels in the first eye image and the second pixel average value corresponding to the pixel values ​​of all pixels in the second eye image;

[0134] First update unit: Based on the average value of the first pixel, using the formula... Update the pixel values ​​of each pixel in the first eye image;

[0135] Second update unit: Based on the second pixel average value, using the formula Update the pixel values ​​of each pixel in the second eye image;

[0136] Image merging unit: Obtains the target eye image based on the updated first eye image and the second eye image;

[0137] Where A(x, y) are the coordinates of a pixel in the first eye image, B(x, y) are the coordinates of a pixel in the second eye image, and a x Let a be the x-coordinate of the center point of the first eye image. y Let q1 be the ordinate of the center point of the first eye image, q2 be the average value of the first pixel, and q3 be the average value of the first pixel.

[0138] The image white-eye elimination device based on low-brightness infrared supplementary lighting in this embodiment acquires the original eye image of a low-brightness environment under infrared supplementary lighting. At this time, the original eye image shows that the eyeball is white. The original eye image is sent for image segmentation of the eyeball part and the non-eyeball part. Then, the first eye image corresponding to the segmented eyeball part and the second eye image corresponding to the non-eyeball part are fused in grayscale to obtain the target image corresponding to the normal eye image. The present invention eliminates the white-eye phenomenon by segmenting the eyeball part and the non-eyeball part and then performing grayscale fusion to make the brightness of the eyeball part and the non-eyeball part match, thereby avoiding discomfort to the user.

[0139] Example 3

[0140] The present invention provides an electronic device and a medium, as shown in FIG5, including at least one processor, at least one memory, and computer program instructions stored in the memory.

[0141] Specifically, the processor may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention. The electronic device includes at least one of the following: a camera, a mobile device with a camera, or a wearable device with a camera.

[0142] The memory may include a large-capacity storage device for data or instructions. For example, and not limitingly, the memory may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory may include removable or non-removable (or fixed) media. Where appropriate, the memory may be internal or external to a data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0143] The processor reads and executes computer program instructions stored in memory to implement any of the methods for automatically capturing highlights of baby videos in the above embodiment, Method 1.

[0144] In one example, the electronic device may also include a communication interface and a bus. The processor, memory, and communication interface are connected via the bus and communicate with each other.

[0145] The communication interface is mainly used to enable communication between various modules, devices, units and / or equipment in the embodiments of the present invention.

[0146] A bus, including hardware, software, or both, couples components of an electronic device together. For example, and not limitingly, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, a bus may include one or more buses. While specific buses are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus or interconnect.

[0147] In summary, this invention provides a method, apparatus, and device for eliminating white-eye in images based on low-brightness infrared supplemental lighting. The method involves acquiring a raw image of the eye in a low-brightness environment under infrared supplemental lighting. In this raw image, the eyeball appears white. The raw image is then segmented into the eyeball portion and the non-eyeball portion. The first eye image corresponding to the segmented eyeball portion and the second eye image corresponding to the non-eyeball portion are then fused in grayscale to obtain a target image corresponding to a normal eye image. This invention eliminates the white-eye phenomenon by segmenting the eyeball portion and the non-eyeball portion and then performing grayscale fusion, thus ensuring brightness matching between the two portions and preventing discomfort for the user.

[0148] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0149] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for eliminating white-eye in images based on low-brightness infrared supplemental lighting; characterized in that, The method includes: S1: acquiring an original eye image when supplementing low-brightness environments with infrared light; S2: partitioning the original eye image into eyeball and non-eyeball regions to obtain a first eye image corresponding to the white eyeball region and a second eye image corresponding to the light-colored non-eyeball region; S3: performing grayscale fusion on the first eye image and the second eye image to obtain a target eye image corresponding to a normal eye image, wherein S3 includes: S31: acquiring the first pixel average value corresponding to the pixel values ​​of all pixels in the first eye image and the second pixel average value corresponding to the pixel values ​​of all pixels in the second eye image; S32: based on the first pixel average value, using the formula... Update the pixel values ​​of each pixel in the first eye image; S33: Based on the average value of the second pixel, use the formula Update the pixel values ​​of each pixel in the second eye image; S34: Obtain the target eye image based on the updated first eye image and the second eye image; where A(x, y) is the coordinate of a pixel in the first eye image, B(x, y) is the coordinate of a pixel in the second eye image, and a x Let a be the x-coordinate of the center point of the first eye image. y Let q1 be the ordinate of the center point of the first eye image, q2 be the average value of the first pixel, and q3 be the average value of the first pixel.

2. The image white-eye elimination method based on low-brightness infrared supplementary lighting according to claim 1; characterized in that, S1 includes: S11: acquiring an infrared image when supplementing low-brightness environments with infrared light, wherein the infrared image includes an eye image; S12: performing grayscale processing on the infrared image to obtain a grayscale image corresponding to the infrared image; S13: performing target detection on the grayscale image and segmenting the original eye image containing the eye region.

3. The image white-eye elimination method based on low-brightness infrared supplementary lighting according to claim 1; characterized in that, S2 includes: S21: converting the original eye image into a histogram; S22: dividing the histogram into a left histogram and a right histogram based on the pixels at the trough positions of the histogram; S23: calculating the average and variance values ​​of the pixel values ​​of the pixels in the left histogram and / or the right histogram, respectively; S24: comparing the pixel values ​​of each pixel in the original eye image with the variance value and the average value to obtain each first pixel corresponding to the white eyeball area and each second pixel corresponding to the light-colored non-eyeball area; wherein, the first eye image is composed of all the first pixels, and the second eye image is composed of all the second pixels.

4. The image white-eye elimination method based on low-brightness infrared supplementary lighting according to claim 3; characterized in that, S22 includes: S221: obtaining the pixel value P corresponding to the pixel at the valley position of the histogram; S222: traversing the pixel values ​​of each pixel in the histogram according to the pixel value P, to obtain the left histogram corresponding to each first pixel value with a pixel value less than P and the right histogram corresponding to each second pixel value with a pixel value greater than P.

5. The image white-eye elimination method based on low-brightness infrared supplementary lighting according to claim 4; characterized in that, S24 includes: S241: obtaining the first average value and first variance value corresponding to the pixel values ​​of the left histogram pixels, and the second average value and second variance value corresponding to the pixel values ​​of the right histogram pixels; S242: based on the pixel values ​​of each pixel in the original eye image, using the formula... S243: Based on the pixel value X of each pixel in the original eye image, use the formula to obtain each second pixel point; Each first pixel is obtained; where n is a constant, K1 is the first average value, Q1 is the first variance value, K2 is the second average value, Q2 is the second variance value, and X is the pixel value of a pixel in the original eye image.

6. The image white-eye elimination method based on low-brightness infrared supplementary lighting according to claim 4; characterized in that, S24 includes: S244: randomly acquiring at least one first target pixel with a first average pixel value and at least one second target pixel with a second average pixel value; S245: performing edge detection on the grayscale image according to the first target pixel and the second target pixel in a preset tracking direction to obtain the contour region and the in-contour region corresponding to the tracking path of the first target pixel and the second target pixel; S246: repeating S244 to S245 until obtaining the first eye image corresponding to each in-contour region of all the first target pixels and all the second target pixels and the second eye image corresponding to each contour region.

7. An image white-eye elimination device based on low-brightness infrared supplementary light, characterized in that, include: Image acquisition module: used to acquire the original eye image when using infrared light to supplement illumination in a low-brightness environment; Image segmentation module: used to partition the original eye image into eyeball regions and non-eyeball regions, obtaining a first eye image corresponding to the white eyeball region and a second eye image corresponding to the light-colored non-eyeball region; Image fusion module: used to perform grayscale fusion on the first eye image and the second eye image to obtain a target eye image corresponding to a normal eye image, specifically used to: obtain the first pixel average value corresponding to the pixel values ​​of all pixels in the first eye image and the second pixel average value corresponding to the pixel values ​​of all pixels in the second eye image; based on the first pixel average value, using the formula... Update the pixel values ​​of each pixel in the first eye image; Based on the second pixel average value, using the formula Update the pixel values ​​of each pixel in the second eye image; The target eye image is obtained based on the updated first eye image and the second eye image; where A(x, y) is the coordinate of a pixel in the first eye image, B(x, y) is the coordinate of a pixel in the second eye image, and a x Let a be the x-coordinate of the center point of the first eye image. y Let q1 be the ordinate of the center point of the first eye image, q2 be the average value of the first pixel, and q3 be the average value of the first pixel.

8. An electronic device, characterized in that, include: At least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method as described in any one of claims 1-6.

9. A medium having computer program instructions stored thereon, characterized in that, The method as described in any one of claims 1-6 is implemented when the computer program instructions are executed by the processor.

Images