Methods, devices, electronic equipment, and storage media for processing eye images

Abstract

This application relates to a method, apparatus, computer device, and storage medium for processing eye images. The method includes: acquiring a first eye image; the first eye image including a chroma channel component and a luminance channel component; adjusting the luminance channel component to obtain an adjusted luminance channel component; adjusting the chroma channel component to obtain an adjusted chroma channel component; and obtaining a second eye image based on the adjusted luminance channel component and the adjusted chroma channel component. This method can improve the accuracy of eye image processing.

Description

Technical Field

[0001] This application relates to computer technology, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for processing eye images. Background Technology

[0002] With the development of computer technology, image enhancement techniques have emerged. These techniques can be used to enhance images, resulting in more accurate, realistic, and aesthetically pleasing representations.

[0003] Typically, electronic devices enhance the entire image. It's understandable that the eyes, being a crucial part of the face, play a vital role in conveying facial expressions. However, this method of global enhancement of the entire image suffers from lower accuracy when processing the eye area. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and computer-readable storage medium for processing eye images, which can improve the accuracy of eye image processing.

[0005] A method for processing eye images, comprising:

[0006] Acquire a first eye image; the first eye image includes chroma channel components and luminance channel components;

[0007] The brightness of the brightness channel component is adjusted to obtain the adjusted brightness channel component;

[0008] The chroma channel components are chroma-adjusted to obtain the adjusted chroma channel components.

[0009] The second eye image is obtained based on the adjusted luminance channel component and the adjusted chrominance channel component.

[0010] An eye image processing apparatus, comprising:

[0011] An acquisition module is used to acquire a first eye image; the first eye image includes a chroma channel component and a luminance channel component;

[0012] An adjustment module is used to adjust the brightness of the brightness channel component to obtain the adjusted brightness channel component.

[0013] The adjustment module is also used to perform chromaticity adjustment on the chromaticity channel components to obtain the adjusted chromaticity channel components.

[0014] A generation module is used to obtain a second eye image based on the adjusted luminance channel component and the adjusted chrominance channel component.

[0015] An electronic device includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of the eye image processing method described above.

[0016] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0017] The aforementioned eye image processing method, apparatus, electronic device, and computer-readable storage medium acquire a first eye image, which includes chroma channel components and luminance channel components. By adjusting the luminance channel components and the chroma channel components of the first eye image, more accurate luminance channel components and more accurate chroma channel components can be obtained, respectively. Based on the adjusted luminance channel components and the adjusted chroma channel components, a more moisturizing second eye image can be obtained, improving the accuracy of eye image processing and resulting in a more natural processed eye image. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of an image processing circuit in one embodiment;

[0020] Figure 2 This is a flowchart of a method for processing eye images in one embodiment;

[0021] Figure 3 This is a schematic diagram of a first eye image in one embodiment;

[0022] Figure 4 This is a schematic diagram of a first eye image in another embodiment;

[0023] Figure 5 This is a flowchart of the step of obtaining the gain value of the luminance channel component after tone mapping in one embodiment;

[0024] Figure 6 This is a schematic diagram of a method for processing eye images in another embodiment;

[0025] Figure 7This is a schematic diagram of the eye image processing flow in another embodiment;

[0026] Figure 8 This is a structural block diagram of an eye image processing apparatus according to one embodiment;

[0027] Figure 9 This is a schematic diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] It is understood that the terms "first," "second," etc., used herein may be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, a first eye image may be referred to as a second eye image, and similarly, a second eye image may be referred to as a first eye image. Both the first eye image and the second eye image are eye images, but they are not the same eye image.

[0030] This application provides an electronic device that includes an image processing circuit. The image processing circuit can be implemented using hardware and / or software components and may include various processing units that define an ISP (Image Signal Processing) pipeline. Figure 1 This is a schematic diagram of an image processing circuit in one embodiment. For example... Figure 1 As shown, for ease of explanation, only aspects of the image processing technology related to the embodiments of this application are shown.

[0031] like Figure 1 As shown, the image processing circuitry includes an ISP processor 120. Image data captured by the imaging device 110 is first processed by the ISP processor 120, which analyzes the image data to capture image statistics that can be used to determine and / or control parameters of the imaging device 110. The imaging device 110 may include a camera having one or more lenses 112 and an image sensor 114.

[0032] The ISP processor 120 processes raw image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits. The ISP processor 120 may perform one or more image processing operations on the raw image data and collect statistical information about the image data. The image processing operations may be performed with the same or different bit depth precision.

[0033] The ISP processor 120 may also receive image data from the image memory 130. For example, raw image data in the image memory 130 may be provided to the ISP processor 120 for processing. The image memory 130 may be part of a memory device, a storage device, or a separate dedicated memory within an electronic device, and may include DMA (Direct Memory Access) features.

[0034] Upon receiving raw image data from the image sensor 114 interface or from the image memory 130, the ISP processor 120 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to the image memory 130 for further processing before display. The ISP processor 120 may also receive processed data from the image memory 130 and perform image data processing in the raw domain and in the RGB and YCbCr color spaces. The processed image data may be output to the display 140 for user viewing and / or further processed by a graphics engine or GPU (Graphics Processing Unit). Furthermore, the output of the ISP processor 120 may also be sent to the image memory 130, and the display 140 may read image data from the image memory 130. In one embodiment, the image memory 130 may be configured to implement one or more frame buffers.

[0035] The steps of image data processing by the ISP processor 120 include: VFE (Video Front End) processing and CPP (Camera Post Processing) processing. VFE processing may include correcting the contrast or brightness of the image data, modifying digitally recorded lighting status data, performing compensation processing on the image data (such as white balance, automatic gain control, gamma correction, etc.), and filtering the image data. CPP processing may include scaling the image and providing preview frames and recorded frames to each path. Different codecs can be used to process the preview frames and recorded frames in CPP.

[0036] In one embodiment, the electronic device captures a raw image using an imaging device 110 and sends the raw image to an ISP processor 120. The ISP processor 120 performs eye detection on the raw image. If an eye is detected in the raw image, it separates a first eye image and a non-eye image from the raw image. The first eye image includes a chroma channel component and a luminance channel component. The luminance channel component is then adjusted to obtain an adjusted luminance channel component. The chroma channel component is then adjusted to obtain an adjusted chroma channel component. Based on the adjusted luminance channel component and the adjusted chroma channel component, a more moisturizing and accurate second eye image can be obtained. The second eye image and the non-eye image are then fused to obtain the target image. The ISP processor 120 can send the target image to an image memory 130 for storage or to a display 140 for display on the electronic device's display interface.

[0037] Figure 2 This is a flowchart of an eye image processing method in one embodiment. The eye image processing method in this embodiment is designed to run on... Figure 1 The description will be based on an example of an electronic device. Figure 2 As shown, the method for processing eye images includes steps 202 to 208.

[0038] Step 202: Obtain the first eye image; the first eye image includes chroma channel components and luminance channel components.

[0039] A first-eye image refers to an image that includes the eyes and is not processed. A first-eye image may include only the eye area or the surrounding area. The surrounding area may specifically include the eyebrow area, the skin area around the eyes, the forehead area, etc.

[0040] Figure 3 This is a schematic diagram of a first eye image in one embodiment. Figure 4 This is a schematic diagram of a first eye image in another embodiment. (See diagram below.) Figure 3 As shown, the first eye image includes not only the area containing the eye but also the area surrounding the eye. Figure 4 As shown, the first eye image only includes the area containing the eye.

[0041] The first eye image may include one eye region or at least two eye regions. The first eye image may include at least one of a human eye region or an animal eye.

[0042] The first eye image includes at least two channel components: a chroma channel component and a luma channel component. The chroma channel component specifies the color of a pixel and describes the image's color and saturation. The luma channel component specifies the brightness of a pixel, which is its grayscale value.

[0043] The first eye image may include one or more chroma channel components; the first eye image may also include one or more luminance channel components. For example, a first eye image of type LUV includes U and V channel components for chroma and L channel components for luminance.

[0044] In one implementation, the electronic device captures an original image using a camera and obtains a first eye image from the original image. In another implementation, the electronic device retrieves the original image from a memory and obtains the first eye image from the original image. In yet another implementation, the electronic device directly receives a first eye image sent by another device.

[0045] Step 204: Adjust the brightness of the luminance channel component to obtain the adjusted luminance channel component.

[0046] Brightness adjustment refers to the technique of adjusting the brightness channel components.

[0047] In one implementation, the electronic device adjusts the brightness of the brightness channel components using a brightness adjustment model to obtain the adjusted brightness channel components. The brightness adjustment model can be pre-trained.

[0048] In another implementation, the electronic device can obtain the brightness adjustment parameters input by the user, and adjust the brightness of the brightness channel component according to the brightness adjustment parameters to obtain the adjusted brightness channel component.

[0049] Step 206: Perform chroma adjustment on the chroma channel components to obtain the adjusted chroma channel components.

[0050] Color adjustment refers to the technique of adjusting the color channel components.

[0051] In one implementation, the electronic device performs chroma adjustment on the chroma channel components using a chroma adjustment model to obtain the adjusted chroma channel components. The chroma adjustment model can be pre-trained.

[0052] In another implementation, the electronic device can obtain the chroma adjustment parameters input by the user, and adjust the chroma channel components according to the chroma adjustment parameters to obtain the adjusted chroma channel components.

[0053] Step 208: Based on the adjusted luminance channel component and the adjusted chrominance channel component, a second eye image is obtained.

[0054] The second eye image refers to an image that includes the eyes and has been processed.

[0055] In one embodiment, the electronic device directly combines the adjusted luminance channel component and the adjusted chroma channel component to obtain a second eye image. In another embodiment, the electronic device may also replace the unadjusted luminance channel component in the first eye image with the adjusted luminance channel component, and replace the unadjusted chroma channel component in the first eye image with the adjusted chroma channel component to obtain a second eye image.

[0056] In this embodiment, a first eye image is acquired, which includes a chroma channel component and a luminance channel component. Luminance adjustment is performed on the luminance channel component of the first eye image, and chroma adjustment is performed on the chroma channel component. This yields more accurate luminance channel components in the luminance channel and more accurate chroma channel components in the chroma channel. Based on the adjusted luminance channel components and the adjusted chroma channel components, a second eye image with a more moisturizing appearance can be obtained, improving the accuracy of eye image processing.

[0057] With the first eye image captured in real-time by the camera, users don't need to perform time-consuming image processing. The image of the eyes appears more moisturized immediately after taking the picture, thus improving the efficiency of image processing.

[0058] In one embodiment, adjusting the brightness of the luminance channel component to obtain the adjusted luminance channel component includes: performing tone mapping on the luminance channel component to obtain the tone-mapped luminance channel component; and adjusting the brightness of the tone-mapped luminance channel component to obtain the adjusted luminance channel component.

[0059] Tone mapping is a computer graphics technique used to approximate the display of high dynamic range images on media with a limited dynamic range. Essentially, tone mapping aims to significantly reduce contrast to bring scene brightness to a displayable range while preserving image details and colors—information crucial for representing the original scene.

[0060] Electronic devices use a tone mapping model to perform tone mapping on the luminance channel components, resulting in tone-mapped luminance channel components. The tone mapping model can be pre-trained.

[0061] Tone mapping is performed on the luminance channel components to obtain tone-mapped luminance channel components. Specifically, this includes: obtaining tone mapping parameter values; based on the pixel values ​​of each pixel in the luminance channel components and the tone mapping parameter values, obtaining the pixel values ​​of each pixel after tone mapping; and generating the tone-mapped luminance channel components. The tone mapping parameter values ​​are the numerical values ​​of the parameters used in the tone mapping process. These values ​​can be set as needed. For example, the tone mapping parameter value can be set to 3.

[0062] In one embodiment, the electronic device may use the following formula to perform tone mapping on the pixel values ​​of each pixel in the luminance channel component, thereby obtaining the tone-mapped pixel values ​​of each pixel in the luminance channel component:

[0063] Y = X gamma

[0064] Where Y is the pixel value of the pixel after tone mapping, X is the pixel value of the pixel before tone mapping, and gamma is the tone mapping parameter value.

[0065] The electronic device performs brightness adjustment on the luminance channel component after tone mapping to obtain the adjusted luminance channel component, including: obtaining the gain value of the luminance channel component after tone mapping; multiplying the pixel value of each pixel in the luminance channel component after tone mapping by the gain value to obtain the pixel value of each pixel after brightness adjustment, and generating the adjusted luminance channel component based on the pixel value of each pixel after brightness adjustment.

[0066] Gain refers to the amplification factor. In one implementation, the electronic device acquires the gain value of the tone-mapped luminance channel component input by the user. In another implementation, the electronic device determines the gain value of the tone-mapped luminance channel component based on the luminance channel component before and after tone mapping.

[0067] The electronic device acquires the pixel value of each pixel in the luminance channel component after tone mapping, multiplies the pixel value of each pixel in the luminance channel component after tone mapping by the gain value, and obtains the pixel value of each pixel after brightness adjustment. Based on the pixel value of each pixel after brightness adjustment, a more accurate adjusted luminance channel component can be generated, thereby obtaining a second eye image with a more moisturizing feel and improving the accuracy of eye image processing.

[0068] It is understandable that since tone mapping will darken the image, in this embodiment, the electronic device first performs tone mapping on the luminance channel component to obtain the tone-mapped luminance channel component, and then adjusts the brightness of the tone-mapped luminance channel component to obtain a more accurate luminance channel component, thereby obtaining a more moisturized second eye image and improving the accuracy of eye image processing.

[0069] In one embodiment, such as Figure 5 As shown, the gain values ​​of the luminance channel components after tone mapping are obtained, including:

[0070] Step 502: Obtain the first luminance value of the first specified region of the luminance channel component before adjustment, and obtain the second luminance value of the second specified region of the luminance channel component after tone mapping.

[0071] The first specified region is the region designated for obtaining the gain value in the luminance channel component before adjustment. The second specified region is the region designated for obtaining the gain value in the luminance channel component after tone mapping.

[0072] The first and second specified regions can completely overlap, partially overlap, or not overlap at all. For example, the first specified region may be the entire area of ​​the luminance channel component before adjustment, and the second specified region may be the entire area of ​​the luminance channel component after tone mapping; that is, the first and second specified regions can completely overlap. As another example, the first specified region may be the white area of ​​the eye in the luminance channel component, and the second specified region may also be the white area of ​​the eye in the tone-mapped luminance channel component; that is, the first and second specified regions can completely overlap. Yet another example is that the first specified region is a circular area with a radius of 5 pixels centered at the center position in the luminance channel component, and the second specified region is a circular area with a radius of 10 pixels centered at the center position in the tone-mapped luminance channel component; that is, the first and second specified regions partially overlap.

[0073] The first brightness value is a brightness value determined based on a first specified area. The second brightness value is a brightness value determined based on a second specified area.

[0074] An electronic device determines a first brightness value based on a first designated area. In one embodiment, the electronic device acquires the brightness values ​​of each pixel in the first designated area and calculates the average of the brightness values ​​of all pixels as the first brightness value. In another embodiment, the electronic device acquires the brightness values ​​of each pixel in the first designated area and uses the largest value among the brightness values ​​of all pixels as the first brightness value. In yet another embodiment, the electronic device acquires the brightness values ​​of each pixel in the first designated area and uses the smallest value among the brightness values ​​of all pixels as the first brightness value. The method by which the electronic device determines the first brightness value based on the first designated area can be set as needed and is not limited here. Each pixel value can represent the brightness value of that pixel.

[0075] An electronic device determines a second brightness value based on a second designated area. In one embodiment, the electronic device acquires the brightness values ​​of each pixel in the second designated area and calculates the average of the brightness values ​​of all pixels as the second brightness value. In another embodiment, the electronic device acquires the brightness values ​​of each pixel in the second designated area and uses the largest value among the brightness values ​​of all pixels as the second brightness value. In yet another embodiment, the electronic device acquires the brightness values ​​of each pixel in the second designated area and uses the smallest value among the brightness values ​​of all pixels as the second brightness value. The method by which the electronic device determines the second brightness value based on the second designated area can be set as needed and is not limited here. Each pixel value can represent the brightness value of that pixel.

[0076] Step 504: Based on the first luminance value and the second luminance value, determine the gain value of the luminance channel component after tone mapping.

[0077] Specifically, the electronic device divides the first brightness value by the second brightness value to obtain the gain value of the brightness channel component after tone mapping.

[0078] Electronic devices can use the following formula to determine the gain value of the luminance channel component after tone mapping:

[0079] gain = L1 ave / L2 ave

[0080] Among them, L1 ave This is the first brightness value, L2. ave This is the second brightness value, and gain is the gain value.

[0081] In this embodiment, the first luminance value of the first designated area of ​​the luminance channel component before adjustment is obtained, and the second luminance value of the second designated area of ​​the luminance channel component after tone mapping is obtained. Based on the first luminance value and the second luminance value, a more accurate gain value of the luminance channel component after tone mapping can be determined.

[0082] In one embodiment, chroma adjustment is performed on the chroma channel components to obtain the adjusted chroma channel components, including: determining a reference pixel from the chroma channel components; determining new pixel values ​​for each pixel in the chroma channel components based on the pixel values ​​of the reference pixel; and obtaining the adjusted chroma channel components based on the new pixel values ​​of each pixel in the chroma channel components.

[0083] A reference pixel is a pixel used as a reference during color adjustment. A reference pixel can be a white pixel. For example, the D65 light source is a white pixel.

[0084] When the first eye image is of type LUV, both the U and V channels are chroma channels. Therefore, the pixel value of the white point in the U channel can be 0.2032, and the pixel value in the V channel can be 0.4963. The reference pixel can also be other points, set as needed. The pixel value of the reference pixel in the chroma channel component is the chroma value of the reference pixel.

[0085] Furthermore, the electronic device acquires chroma parameter values ​​and, based on the new pixel values ​​and chroma parameter values ​​of each pixel in the chroma channel components, obtains the adjusted chroma channel components. The chroma parameter values ​​are the numerical values ​​of the parameters used in the chroma adjustment process. The chroma parameter values ​​can be set as needed. For example, the chroma parameter value can be set to 0.26.

[0086] When the first eye image is of type LUV, both the U and V channel components are chroma channel components, and the L channel image is a luma channel component. The electronic device uses the following formulas to determine the new pixel values ​​for each pixel in the chroma channel components:

[0087] S uv(i,j) =13{(u′) (i,j) -u′ n ) 2 +(v′ (i,j) -v′ n ) 2} 1 / 2 +offset

[0088] Among them, S uv(i,j) It is the adjusted chromaticity of the pixel at coordinate (i,j), u′ (i,j) It is the value of the pixel at coordinate (i,j) in the U channel, u′ n It is the value of the reference pixel in the U channel, v′ (i,j) It is the value of the pixel at coordinate (i,j) in the V channel, v′ n It is the value of the reference pixel in the V channel, and offset is the chroma parameter value.

[0089] In another embodiment, when the type of the first eye image is LUV, both the U channel component and the V channel component are chroma channel components, and the L channel image is the luminance channel component; the electronic device can also use the following formula to calculate the new hue value of each pixel in the chroma channel component:

[0090] H uv(i,j) =tan -1 {(v′ (i,j) -v′ n ) / (u′ (i,j) -u′ n )}

[0091] Among them, H uv(i,j) u′ represents the adjusted hue value of the pixel at coordinate (i,j). (i,j) It is the value of the pixel at coordinate (i,j) in the U channel, u′ n It is the value of the reference pixel in the U channel, v′ (i,j) It is the value of the pixel at coordinate (i,j) in the V channel, v′ n It refers to the value of the reference pixel in the V channel.

[0092] In this embodiment, a reference pixel is determined from the chroma channel components. Based on the pixel value of the reference pixel, new pixel values ​​for each pixel in the chroma channel components can be determined, thereby obtaining a more accurate adjusted chroma channel component. This results in a more moisturizing second eye image and improves the accuracy of eye image processing.

[0093] In one embodiment, acquiring a first eye image includes: if an RGB-type first eye image is acquired, spatially transforming the RGB-type first eye image to obtain a LUV-type first eye image; in the LUV-type first eye image, the U-channel image and V-channel image are both chroma channel components, and the L-channel image is the luminance channel component; the second eye image is of type LUV; the method further includes: spatially transforming the LUV-type second eye image to obtain an RGB-type second eye image.

[0094] RGB (Red, Green, Blue) and LUV are both color encoding methods for images. Here, L represents the luminance channel of the image, while U and V represent the chrominance channels.

[0095] When a first eye image of type RGB is acquired, the electronic device performs spatial transformation on the first eye image of type RGB, converting the first eye image of type RGB to the first eye image of type LUV.

[0096] After adjusting the first eye image to obtain the second eye image, the second eye image, which belongs to the LUV type, is then spatially transformed to obtain the second eye image, which belongs to the RGB type. This can restore the image type of the processed image to the RGB type before processing, making it easier to continue processing images of the RGB type and improving the efficiency of subsequent image processing.

[0097] In one embodiment, obtaining a first eye image includes: performing eye detection on the obtained original image; if an eye is detected in the original image, separating the first eye image and a non-eye image from the original image; after obtaining a second eye image, further including: fusing the second eye image and the non-eye image to obtain a target image.

[0098] Non-eye images refer to images that do not include eyes.

[0099] The electronic device performs eye contour detection on the acquired raw image. If an eye contour is detected in the raw image, it is determined that the raw image contains eyes; if no eye contour is detected, it is determined that the raw image does not contain eyes. The electronic device can employ AI (Artificial Intelligence) detection technology to perform eye contour detection on the acquired raw image. When an eye contour is detected, the feature points of the eye contour and the closed trajectory formed by these feature points are determined. The first eye image, located at the closed trajectory, is extracted and removed from the raw image to obtain a non-eye image.

[0100] The electronic device fuses the second eye image and the non-eye image, and then uses a bilateral filter to smooth the fused image to obtain the target image. A bilateral filter is a nonlinear filter that smooths the image. Smoothing filtering is a low-frequency enhanced spatial domain filtering technique. By applying smoothing filtering to the fused image, the electronic device can smooth the image while preserving details, avoiding unnatural eye edges in the fused image, and thus obtaining a more accurate target image.

[0101] If no eyes are detected in the original image, then the original image is not processed.

[0102] When an eye is detected in the original image, a first eye image and a non-eye image are separated from the original image. The first eye image is then processed to obtain a second eye image with a more moisturized appearance. The second eye image and the non-eye image are then fused together to obtain an eye image in the target image with a more moisturized appearance, thus improving the accuracy of target image processing.

[0103] Figure 6 This is a schematic diagram of an eye image processing method in another embodiment. The electronic device performs eye detection on the acquired original image 602. If an eye is detected in the original image 602, a first eye image 606 and a non-eye image 604 are separated from the original image 602. The first eye image is processed to obtain a second eye image 608. The second eye image 608 and the non-eye image 604 are then fused to obtain a target image 610.

[0104] Furthermore, given the original image of type RGB, the separated first eye image and non-eye image are also of type RGB. The method further includes: spatially transforming the first eye image of type RGB to obtain a first eye image of type LUV; in the first eye image of type LUV, the U channel image and V channel image are both chroma channel components, and the L channel image is the luminance channel component; the second eye image is of type LUV; the method further includes: spatially transforming the second eye image of type LUV to obtain a second eye image of type RGB; fusing the second eye image and non-eye image to obtain a target image, including: fusing the second eye image of type RGB and the non-eye image of type RGB to obtain a target image of type RGB.

[0105] Figure 7 This is a schematic diagram of the eye image processing flow in another embodiment. The electronic device acquires an original image of type RGB, and executes step 702, performing eye de-separation on the original RGB image to obtain a first eye image of type RGB and a non-eye image of type RGB. The electronic device then executes step 704, performing spatial transformation on the first eye image of type RGB to obtain a first eye image of type LUV. The first eye image includes chroma channel components and luminance channel components.

[0106] The electronic device performs step 706, adjusting the chroma of the chroma channel components to obtain the adjusted chroma channel components. Then, it performs step 708, toning the luminance channel components to obtain the toned luminance channel components; and finally, it performs step 710, adjusting the brightness of the toned luminance channel components to obtain the adjusted luminance channel components. Based on the adjusted luminance channel components and the adjusted chroma channel components, a second eye image of type LUV is obtained.

[0107] The electronic device performs step 712, spatially transforming the second eye image of type LUV to obtain a second eye image of type RGB. The electronic device then fuses the second eye image of type RGB and the non-eye image of type RGB, and performs step 714, smoothing and filtering the fused image to obtain the target image of type RGB.

[0108] In another embodiment, a method for processing an eye image includes steps a1 to a9, and is applied in an electronic device:

[0109] Step a1: Given an original image of type RGB, perform eye detection on the original image. If an eye is detected in the original image, separate the first eye image and the non-eye image from the original image. The first eye image includes chroma channel components and luminance channel components.

[0110] Step a2: After obtaining the first eye image of type RGB, perform spatial transformation on the first eye image of type RGB to obtain the first eye image of type LUV. In the first eye image of type LUV, the U channel component and V channel component are both chroma channel components, and the L channel component is the luminance channel component. The electronic device performs the luminance adjustment step and the chroma adjustment step respectively. The luminance adjustment step includes steps a3-a5, and the chroma adjustment step includes step a6.

[0111] Step a3: Perform tone mapping on the luminance channel components to obtain the tone-mapped luminance channel components.

[0112] Step a4: Obtain the first luminance value of the first specified region of the luminance channel component before adjustment, and obtain the second luminance value of the second specified region of the luminance channel component after tone mapping; based on the first luminance value and the second luminance value, determine the gain value of the luminance channel component after tone mapping.

[0113] Step a5: Multiply the pixel value of each pixel in the luminance channel component after tone mapping by the gain value to obtain the pixel value of each pixel after brightness adjustment, and generate the adjusted luminance channel component based on the pixel value of each pixel after brightness adjustment.

[0114] Step a6: Determine the reference pixel from the chroma channel components; based on the pixel value of the reference pixel, determine the new pixel value of each pixel in the chroma channel components, and based on the new pixel value of each pixel in the chroma channel components, obtain the adjusted chroma channel components.

[0115] Step a7: Based on the adjusted luminance channel components and the adjusted chrominance channel components, obtain the second eye image belonging to the LUV type.

[0116] Step a8: Perform spatial transformation on the second eye image belonging to the LUV type to obtain the second eye image belonging to the RGB type.

[0117] Step a9: Fuse the second eye image (which is of type RGB) and the non-eye image (which is of type RGB) to obtain the target image.

[0118] In this embodiment, a first eye image is acquired, which includes a chroma channel component and a luminance channel component. Luminance adjustment is performed on the luminance channel component of the first eye image, and chroma adjustment is performed on the chroma channel component. This yields more accurate luminance channel components in the luminance channel and more accurate chroma channel components in the chroma channel. Based on the adjusted luminance channel components and the adjusted chroma channel components, a second eye image with a more moisturizing appearance can be obtained, improving the accuracy of eye image processing.

[0119] It should be understood that, although Figure 2 , Figure 5 and Figure 7 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 , Figure 5 and Figure 7 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0120] Figure 8 This is a structural block diagram of an eye image processing apparatus according to one embodiment. Figure 8As shown, an eye image processing apparatus is provided, comprising: an acquisition module 802, an adjustment module 804, and a generation module 806, wherein:

[0121] The acquisition module 802 is used to acquire a first eye image; the first eye image includes a chroma channel component and a luminance channel component.

[0122] The adjustment module 804 is used to adjust the brightness of the brightness channel component to obtain the adjusted brightness channel component.

[0123] The adjustment module 804 is also used to perform chroma adjustment on the chroma channel components to obtain the adjusted chroma channel components.

[0124] The generation module 806 is used to obtain a second eye image based on the adjusted luminance channel component and the adjusted chrominance channel component.

[0125] The aforementioned eye image processing apparatus acquires a first eye image, which includes a chroma channel component and a luminance channel component. By performing luminance adjustment on the luminance channel component and chroma adjustment on the chroma channel component of the first eye image, more accurate luminance channel components and more accurate chroma channel components can be obtained, respectively. Based on the adjusted luminance channel components and the adjusted chroma channel components, a second eye image with a more moisturizing feel can be obtained, thereby improving the accuracy of eye image processing.

[0126] In one embodiment, the adjustment module 804 is further configured to perform tone mapping on the luminance channel component to obtain the tone-mapped luminance channel component; and to adjust the brightness of the tone-mapped luminance channel component to obtain the adjusted luminance channel component.

[0127] In one embodiment, the adjustment module 804 is further configured to obtain the gain value of the luminance channel component after tone mapping; multiply the pixel value of each pixel in the luminance channel component after tone mapping by the gain value to obtain the pixel value of each pixel after brightness adjustment, and generate the adjusted luminance channel component based on the pixel value of each pixel after brightness adjustment.

[0128] In one embodiment, the adjustment module 804 is further configured to obtain a first luminance value of a first designated region of the luminance channel component before adjustment, obtain a second luminance value of a second designated region of the luminance channel component after tone mapping, and determine a gain value of the luminance channel component after tone mapping based on the first luminance value and the second luminance value.

[0129] In one embodiment, the adjustment module 804 is further configured to determine a reference pixel from the chroma channel components; based on the pixel value of the reference pixel, determine the new pixel value of each pixel in the chroma channel components respectively, and obtain the adjusted chroma channel components based on the new pixel value of each pixel in the chroma channel components.

[0130] In one embodiment, the acquisition module 802 is further configured to, upon acquiring a first eye image of type RGB, perform spatial transformation on the first eye image of type RGB to obtain a first eye image of type LUV; in the first eye image of type LUV, the U channel component and the V channel component are both chroma channel components, and the L channel component is the luminance channel component; the device further includes a spatial transformation module configured to perform spatial transformation on a second eye image of type LUV to obtain a second eye image of type RGB.

[0131] In one embodiment, the acquisition module 802 is further configured to perform eye detection on the acquired original image, and if an eye is detected in the original image, to separate a first eye image and a non-eye image from the original image; the generation module 806 is further configured to fuse the second eye image and the non-eye image to obtain a target image.

[0132] The division of the various modules in the above-described eye image processing device is merely illustrative. In other embodiments, the eye image processing device may be divided into different modules as needed to complete all or part of the functions of the above-described eye image processing device.

[0133] Specific limitations regarding the eye image processing device can be found in the above description of the eye image processing method, and will not be repeated here. Each module in the aforementioned eye image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0134] Figure 9 This is a schematic diagram of the internal structure of an electronic device in one embodiment. For example... Figure 9As shown, the electronic device includes a processor and a memory connected via a system bus. The processor provides computing and control capabilities to support the operation of the entire electronic device. The memory may include non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The computer programs can be executed by the processor to implement an eye image processing method provided in the following embodiments. The internal memory provides a cached runtime environment for the operating system computer programs in the non-volatile storage media. The electronic device can be any terminal device such as a mobile phone, tablet computer, PDA (Personal Digital Assistant), POS (Point of Sales), in-vehicle computer, wearable device, etc.

[0135] The various modules in the eye image processing apparatus provided in this application embodiment can be implemented in the form of a computer program. This computer program can run on a terminal or server. The program modules constituted by this computer program can be stored in the memory of an electronic device. When the computer program is executed by a processor, it implements the steps of the method described in the embodiments of this application.

[0136] This application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, cause the processors to perform the steps of a method for processing eye images.

[0137] A computer program product containing instructions that, when run on a computer, causes the computer to perform a method for processing eye images.

[0138] Any references to memory, storage, databases, or other media used in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0139] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for processing eye images, characterized in that, include: Obtain the first eye image; The first eye image includes a chroma channel component and a luminance channel component; The brightness of the brightness channel component is adjusted to obtain the adjusted brightness channel component; Determine the reference pixel from the chroma channel components; The reference pixel is a white pixel; Based on the pixel value of the reference pixel, the new pixel value of each pixel in the chroma channel component is determined, and the adjusted chroma channel component is obtained based on the new pixel value of each pixel in the chroma channel component. Based on the adjusted luminance channel component and the adjusted chroma channel component, a second eye image with a moisturized appearance is obtained.

2. The method according to claim 1, characterized in that, The step of adjusting the brightness of the brightness channel component to obtain the adjusted brightness channel component includes: Perform hue mapping on the luminance channel components to obtain the hue channel components after hue mapping; The brightness of the luminance channel component after tone mapping is adjusted to obtain the adjusted luminance channel component.

3. The method according to claim 2, characterized in that, The step of adjusting the brightness of the luminance channel components after tone mapping to obtain the adjusted luminance channel components includes: Obtain the gain value of the luminance channel component after tone mapping; The pixel value of each pixel in the luminance channel component after tone mapping is multiplied by the gain value to obtain the pixel value of each pixel after brightness adjustment, and the adjusted luminance channel component is generated based on the pixel value of each pixel after brightness adjustment.

4. The method according to claim 3, characterized in that The step of obtaining the gain value of the luminance channel component after tone mapping includes: Obtain the first brightness value of the first specified region of the brightness channel component before adjustment, and obtain the second brightness value of the second specified region of the brightness channel component after tone mapping; Based on the first luminance value and the second luminance value, the gain value of the luminance channel component after tone mapping is determined.

5. The method according to any one of claims 1 to 4, characterized in that, The acquisition of the first eye image includes: In the case of obtaining a first eye image of type RGB, the first eye image of type RGB is spatially transformed to obtain a first eye image of type LUV; in the first eye image of type LUV, the U channel component and the V channel component are both chroma channel components, and the L channel component is the luminance channel component. The second eye image is of type LUV; the method further includes: The second eye image, which belongs to the LUV type, is spatially transformed to obtain a second eye image that belongs to the RGB type.

6. The method according to any one of claims 1 to 4, characterized in that, The acquisition of the first eye image includes: Eye detection is performed on the acquired original image. If an eye is detected in the original image, a first eye image and a non-eye image are separated from the original image. After obtaining the second eye image, the process also includes: The second eye image and the non-eye image are fused together to obtain the target image.

7. An apparatus for processing eye images, characterized in that, include: The acquisition module is used to acquire the first eye image; The first eye image includes a chroma channel component and a luminance channel component; An adjustment module is used to adjust the brightness of the brightness channel component to obtain the adjusted brightness channel component. The adjustment module is further configured to determine a reference pixel from the chroma channel components; the reference pixel is a white point. Based on the pixel value of the reference pixel, the new pixel value of each pixel in the chroma channel component is determined, and the adjusted chroma channel component is obtained based on the new pixel value of each pixel in the chroma channel component. The generation module is used to obtain a second eye image with a moisturized appearance based on the adjusted luminance channel component and the adjusted chroma channel component.

8. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, the processor performs the steps of the method for processing eye images as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images