Face image acquisition method and device, storage medium and camera

Abstract

The embodiment of the invention discloses a face image acquisition method and device, a storage medium and a camera. The method comprises the steps of collecting an initial image, performing face recognition on the initial image, and judging whether a face is recognized or not; when the face is recognized, counting brightness information and color information of the face based on the recognized face contour, calculating an exposure parameter combination according to the brightness information, and calculating a white balance gain according to the color information; and acquiring a face image based on the exposure parameter combination and the white balance gain. By applying the scheme provided by the embodiment of the invention, white balance adjustment and automatic exposure can be carried out only for the face area, and possible distortion caused by global adjustment is avoided, so that a face image meeting requirements can be acquired. And in addition, the naturalness of the face color can be kept under various different illumination and background colors, so that the method has strong adaptability.

Description

Technical Field

[0001] The present application relates to the field of image acquisition technology, and in particular to a facial image acquisition method, device, storage medium and camera. Background Art

[0002] In the field of image acquisition, in order to acquire images with quality that meets the requirements, it is usually necessary to adjust the camera parameters according to the actual scene. For example, the camera parameters can be adjusted according to the brightness and lighting conditions of the shooting scene. In the known technology, the camera parameters are usually adjusted according to the situation of the entire shooting scene.

[0003] However, there are some face collection scenarios, such as ID card collection, driver's license collection, and overseas photo collection scenarios, where only the image quality of the face area needs to be paid attention to. In this case, when the above method is used for image collection, the image quality of the face area may not meet the requirements when considering the overall image quality due to different backgrounds, different lighting, or different clothes. Figure 1 As shown in the figure, when the camera adjusts parameters according to the entire scene, there is a big difference in the face area in the collected images under different backgrounds; when the background brightness is high, the brightness of the face area in the image is darker, and when the background brightness is low, the brightness of the face area in the image is higher. In addition, it can be seen that the faces in both images are far from the real situation and are distorted. Therefore, how to collect face images that meet the image quality requirements has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a facial image acquisition method, device, storage medium and camera to acquire facial images with satisfactory image quality. The specific technical solution is as follows.

[0005] In a first aspect, an embodiment of the present application provides a method for collecting facial images, the method comprising:

[0006] Collecting an initial image, and performing face recognition on the initial image to determine whether a face is recognized;

[0007] When a face is recognized, brightness information and color information of the face are counted based on the recognized facial contour, and an exposure parameter combination is calculated according to the brightness information, and a white balance gain is calculated according to the color information;

[0008] A facial image is acquired based on the exposure parameter combination and the white balance gain.

[0009] Optionally, the step of performing face recognition on the initial image includes:

[0010] The initial image is input into a pre-trained neural network model to obtain the facial contour included in the initial image; wherein the neural network model is trained based on each sample image and the annotation results of each sample image.

[0011] Optionally, the step of calculating the white balance gain according to the color information includes:

[0012] Get the YUV range corresponding to the preset standard skin color;

[0013] Convert the color information into a YUV domain to obtain corresponding YUV information;

[0014] The red and blue gains in the white balance are adjusted so that the YUV information is within the YUV range.

[0015] Optionally, the exposure parameter combination includes at least: exposure time, sensitivity and aperture.

[0016] In a second aspect, an embodiment of the present application provides a facial image acquisition device, the device comprising:

[0017] A face recognition module is used to collect an initial image and perform face recognition on the initial image to determine whether a face is recognized;

[0018] A parameter adjustment module, for calculating brightness information and color information of the face based on the recognized facial contour when a face is recognized, and calculating an exposure parameter combination according to the brightness information and a white balance gain according to the color information;

[0019] An image acquisition module is used to acquire a face image based on the exposure parameter combination and the white balance gain.

[0020] Optionally, the face recognition module is specifically used to:

[0021] The initial image is input into a pre-trained neural network model to obtain the facial contour included in the initial image; wherein the neural network model is trained based on each sample image and the annotation results of each sample image.

[0022] Optionally, the parameter adjustment module is specifically used to:

[0023] Get the YUV range corresponding to the preset standard skin color;

[0024] Convert the color information into a YUV domain to obtain corresponding YUV information;

[0025] The red and blue gains in the white balance are adjusted so that the YUV information is within the YUV range.

[0026] Optionally, the exposure parameter combination includes at least: exposure time, sensitivity and aperture.

[0027] In a third aspect, an embodiment of the present application provides a camera, comprising: a memory and a processor; the memory and the processor are coupled;

[0028] The memory is used to store one or more computer instructions;

[0029] The processor is used to execute the one or more computer instructions to implement the facial image acquisition method as described in the first aspect.

[0030] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having one or more computer instructions stored thereon, which are executed by a processor to implement the facial image acquisition method as described in the first aspect.

[0031] In a fifth aspect, the present application provides a computer program product, comprising a computer program, and when the computer program is executed by a processor, the data quality assessment method as described in the first aspect is implemented.

[0032] The facial image acquisition method, device, storage medium and camera provided in the embodiments of the present application can perform high-precision facial detection to ensure the accuracy of facial area data. Furthermore, white balance adjustment and automatic exposure are performed only on the facial area to avoid distortion that may occur in global adjustment, thereby ensuring that facial images that meet the requirements are collected. In addition, the method can maintain the naturalness of facial colors under various lighting and background colors, so it has strong adaptability. Moreover, it can also be based on an automatic comparison verification system before and after image color correction to ensure that the correction effect conforms to the authenticity of human eye observation. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work.

[0034] Figure 1 Schematic diagram of face images corresponding to different backgrounds;

[0035] Figure 2 A schematic diagram of a facial image acquisition method provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of the structure of an image acquisition system provided in an embodiment of the present application;

[0037] Figure 4 A schematic diagram of the structure of a facial image acquisition device provided in an embodiment of the present application;

[0038] Figure 5 A schematic diagram of the structure of a camera provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0041] The embodiment of the present application discloses a method, device, storage medium and camera for collecting face images, which can collect face images with satisfactory image quality. The embodiment of the present application is described in detail below.

[0042] Known automatic white balance technologies all make statistics for the entire picture in the current shooting scene. Therefore, they have certain limitations and are restricted by the environment. For example, global white balance adjustment ignores the subject in the image, such as the face, which may cause the subject's color to be distorted; in addition, under complex lighting and colorful backgrounds, existing methods are difficult to accurately restore the true color of the face. Based on this, the purpose of this application is to provide a more accurate white balance adjustment method that can specifically target the face area and ensure the authenticity and naturalness of the face color even under different lighting and colorful backgrounds.

[0043] Figure 2 A schematic diagram of a facial image acquisition method provided in an embodiment of the present application is shown. The method can be applied to a camera, such as Figure 2 As shown, the process includes the following steps:

[0044] S210: Collect an initial image, and perform face recognition on the initial image to determine whether a face is recognized.

[0045] In an embodiment of the present application, the camera can first capture an image based on the current configuration, which can be called an initial image. The initial image can reflect the environmental conditions of the current scene. Therefore, the camera parameters can be adjusted based on the initial image, and finally an image that meets the requirements can be captured.

[0046] The embodiments of the present application are aimed at face acquisition scenarios. Therefore, after acquiring the initial image, the camera can perform face recognition on the initial image to determine whether a face is recognized. It can be understood that when a face is recognized, the area where the facial contour is located in the initial image can be obtained at the same time.

[0047] It is understandable that the face region is usually significantly different from other regions in the initial image, so in one implementation, any known image recognition method can be used to perform face recognition to determine whether there is a face in the initial image. For example, for a specific scene, edge detection or texture analysis can be used to assist in identifying the face region.

[0048] In order to improve the accuracy of face recognition, in another implementation method, a feature recognition method based on deep learning can be used, such as using the YOLO algorithm for model training, and the trained model can be deployed on the terminal product (camera) for real-time detection. Specifically, a large number of sample images can be collected first, and the sample images can be annotated, and the area where the face is located in each sample image can be annotated. After that, the sample images are used to train the neural network model. When the loss function meets the requirements, it indicates that the neural network model training is completed. When the camera captures the image, after the initial image is obtained, the initial image is input into the pre-trained neural network model to obtain the facial contour included in the initial image.

[0049] Use convolutional neural network models for face recognition technology, which can accurately identify faces under various backgrounds and lighting conditions.

[0050] S220: When a human face is recognized, brightness information and color information of the face are counted based on the recognized facial contour, and an exposure parameter combination is calculated according to the brightness information, and a white balance gain is calculated according to the color information.

[0051] When a face is recognized, the camera parameters can be adjusted only according to the image information of the face area, so as to ensure that the image quality of the face area meets the requirements when the image is captured again based on the adjusted parameters.

[0052] The data detected by the model is not necessarily all skin color information. Therefore, in an embodiment of the present application, after obtaining the facial contour, the facial contour can be first cropped and filtered to remove high-frequency information to obtain relatively stable data, which can be used as approximate data of the real data.

[0053] When counting the brightness information and color information of the face based on the recognized facial contour, for example, the brightness value and color value can be counted pixel by pixel within the facial contour range, and the final statistical result can be used as the brightness information and color information of the face. After obtaining the brightness information, the exposure parameter combination can be calculated based on the brightness information, and the exposure parameter combination can include, for example, exposure time, sensitivity, and aperture. For example, a brightness standard value can be set in advance. When the calculated brightness information is greater than or equal to the brightness standard value, it indicates that the brightness of the facial area in the initial image is relatively high. In this case, the exposure parameters can be adjusted to reduce the image brightness; when the calculated brightness information is less than the brightness standard value, it indicates that the brightness of the facial area in the initial image is relatively low. In this case, the exposure parameters can be adjusted to increase the image brightness.

[0054] In one implementation, the step of calculating the white balance gain according to the color information may include: obtaining a YUV range corresponding to a preset standard skin color; converting the color information to a YUV domain to obtain corresponding YUV information; and adjusting the red and blue gains in the white balance so that the YUV information is within the YUV range. For example, the standard skin color range may be set to a Y value greater than 80, a U value between 77-127, and a V value between 133-173.

[0055] YUV is a color encoding method that is often used in various video processing components. When encoding photos or videos, YUV allows for reduced chromaticity bandwidth, taking into account human perception. YUV is a type of true-color color space (color space). "Y" represents brightness (Luminance or Luma), which is the grayscale value, and "U" and "V" represent chromaticity (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel.

[0056] In another implementation, a white balance adjustment method based on skin color detection can be used, which can simplify the algorithm and improve the efficiency of the algorithm. Alternatively, a machine learning method can be used to learn the color characteristics of the face through a large number of samples to perform white balance adjustment, which can improve the accuracy of white balance adjustment.

[0057] S230: Acquire a face image based on the exposure parameter combination and the white balance gain.

[0058] The calculated exposure parameter combination and white balance gain are the optimal camera parameters adapted to the current scene. Therefore, facial images captured based on the calculated exposure parameter combination and white balance gain can capture facial images with image quality that meets the requirements.

[0059] In a specific embodiment, the method provided by the present application can be applied to Figure 3 The image acquisition system, such as Figure 3 As shown, the system consists of a Sensor (CCD / CMOS image sensor), a DSP (Digital Signal Processing), a NPU (Neural Processing Unit), an aperture drive controller, and an exposure and gain controller.

[0060] The sensor is responsible for collecting data, converting optical signals into digital signals and sending them to the DSP. The DSP converts the format of the received data and sends it to the NPU for analysis and identification, calculates the brightness and color information, and reversely infers the required exposure parameters and white balance parameters based on the expected brightness and color values. The aperture drive controller controls the aperture switching, and the exposure and gain controller controls the exposure time and gain of the sensor. The white balance algorithm calculates the white balance compensation parameters (Rgain, Bgain) and sets them to the ISP (Image Signal Processor). The ISP corrects the R and B channels based on the white balance parameters.

[0061] The specific steps are as follows:

[0062] 1. Initialize and collect data: start the camera and obtain image data;

[0063] 2. Face recognition: perform facial contour feature recognition on each collected image data;

[0064] 3. Calculate exposure parameters: Count the brightness and color information of the face through facial contour feature recognition in step 2;

[0065] 4. Calculate the exposure parameter combination based on the brightness information, and calculate the white balance gain based on the color information;

[0066] 5. Set parameters: set the exposure parameters and gain parameters to the corresponding hardware modules;

[0067] 6. Take a photo: If a photo trigger signal is received, take a photo according to the current parameter combination.

[0068] The adjustment of face white balance is similar to that of exposure parameters, the only difference is the data adjusted. Exposure focuses more on brightness information, while white balance focuses more on color, such as the ratio of the R channel to the G channel in RGB.

[0069] The method provided in the embodiment of the present application can perform high-precision face detection to ensure the accuracy of face area data. Furthermore, white balance adjustment and automatic exposure are performed only on the face area, avoiding distortion that may occur in global adjustment, thereby ensuring that a face image that meets the requirements is collected. In addition, the method can maintain the naturalness of face color under various lighting and background colors, so it has strong adaptability. Moreover, it is also possible to ensure that the correction effect is consistent with the authenticity of human eye observation based on the automatic comparison verification system before and after image color correction.

[0070] Figure 4 A structural diagram of a facial image acquisition device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the device comprises:

[0071] The face recognition module 410 is used to collect an initial image and perform face recognition on the initial image to determine whether a face is recognized;

[0072] The parameter adjustment module 420 is used to calculate the brightness information and color information of the face based on the recognized facial contour when a face is recognized, and calculate the exposure parameter combination according to the brightness information and calculate the white balance gain according to the color information;

[0073] The image acquisition module 430 is used to acquire a face image based on the exposure parameter combination and the white balance gain.

[0074] Optionally, the face recognition module 410 is specifically used for:

[0075] The initial image is input into a pre-trained neural network model to obtain the facial contour included in the initial image; wherein the neural network model is trained based on each sample image and the annotation results of each sample image.

[0076] Optionally, the parameter adjustment module 420 is specifically used to:

[0077] Get the YUV range corresponding to the preset standard skin color;

[0078] Convert the color information into a YUV domain to obtain corresponding YUV information;

[0079] The red and blue gains in the white balance are adjusted so that the YUV information is within the YUV range.

[0080] Optionally, the exposure parameter combination includes at least: exposure time, sensitivity and aperture.

[0081] The device provided in the embodiment of the present application can perform high-precision face detection to ensure the accuracy of face area data. Furthermore, white balance adjustment and automatic exposure are performed only on the face area, avoiding distortion that may occur in global adjustment, thereby ensuring that a face image that meets the requirements is collected. In addition, the method can maintain the naturalness of face color under various lighting and background colors, so it has strong adaptability. In addition, it can also be based on an automatic comparison verification system before and after image color correction to ensure that the correction effect is consistent with the authenticity of human eye observation.

[0082] Next, a camera provided by an embodiment of the present application is introduced. Figure 5 , Figure 5 A schematic diagram of the structure of a camera provided in an embodiment of the present application, the camera comprises:

[0083] one or more processors 40;

[0084] The processor 40 is coupled to a storage device 41, and the storage device 41 is used to store one or more programs.

[0085] When the one or more programs are executed by the one or more processors 40, the camera implements the following Figure 2 A technical solution of the facial image acquisition method.

[0086] The present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the following is achieved: Figure 2 A technical solution of the facial image acquisition method.

[0087] The present application provides a computer program product, the computer program product comprising a computer program, the computer program when executed by a processor to achieve the following Figure 2 A technical solution of the facial image acquisition method.

[0088] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present application.

[0089] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A facial image acquisition method, characterized in that: The method comprises: Collecting an initial image, and performing face recognition on the initial image to determine whether a face is recognized; When a face is recognized, brightness information and color information of the face are counted based on the recognized facial contour, and an exposure parameter combination is calculated according to the brightness information, and a white balance gain is calculated according to the color information; A facial image is acquired based on the exposure parameter combination and the white balance gain.

2. The method according to claim 1, characterized in that: The step of performing face recognition on the initial image comprises: The initial image is input into a pre-trained neural network model to obtain the facial contour included in the initial image; wherein the neural network model is trained based on each sample image and the annotation results of each sample image.

3. The method according to claim 1, characterized in that The step of calculating the white balance gain according to the color information comprises: Get the YUV range corresponding to the preset standard skin color; Convert the color information into a YUV domain to obtain corresponding YUV information; The red and blue gains in the white balance are adjusted so that the YUV information is within the YUV range.

4. The method according to claim 1, characterized in that: The exposure parameter combination includes at least: exposure time, sensitivity and aperture.

5. A facial image acquisition device, characterized in that: The device comprises: A face recognition module is used to collect an initial image and perform face recognition on the initial image to determine whether a face is recognized; A parameter adjustment module, for calculating brightness information and color information of the face based on the recognized facial contour when a face is recognized, and calculating an exposure parameter combination according to the brightness information and a white balance gain according to the color information; An image acquisition module is used to acquire a face image based on the exposure parameter combination and the white balance gain.

6. The device according to claim 5, characterized in that The face recognition module is specifically used for: The initial image is input into a pre-trained neural network model to obtain the facial contour included in the initial image; wherein the neural network model is trained based on each sample image and the annotation results of each sample image.

7. The device according to claim 5, characterized in that The parameter adjustment module is specifically used for: Get the YUV range corresponding to the preset standard skin color; Convert the color information into a YUV domain to obtain corresponding YUV information; The red and blue gains in the white balance are adjusted so that the YUV information is within the YUV range.

8. The device according to claim 5, characterized in that The exposure parameter combination includes at least: exposure time, sensitivity and aperture.

9. A camera, characterized in that: include: A memory and a processor; the memory and the processor are coupled; The memory is used to store one or more computer instructions; The processor is used to execute the one or more computer instructions to implement the facial image acquisition method as described in any one of claims 1-4.

10. A readable storage medium having one or more computer instructions stored thereon, characterized in that: The instruction is executed by the processor to implement the facial image acquisition method as described in any one of claims 1-4.

Images