A Facial Automatic Exposure Method Based on a Depth Camera

Abstract

The present invention discloses a method for automatic face exposure based on a depth camera. In order to overcome the problems of the prior art that the use of mature 2D image recognition algorithms has a relatively high cost and does not give full play to the advantages of cross-detection of depth information and gray information of a 3D depth camera, the present invention obtains a grayscale image with ideal brightness through automatic exposure by grayscale histogram statistics, then calculates the reflectivity, determines the approximate range of the facial contour according to the range of human skin reflectivity, and then confirms the facial position through a feature point scanning algorithm combining grayscale and depth, and calculates the exposure parameters for exposure according to the grayscale value of the face, so that an ideal exposure result can be obtained without large-scale image recognition and data operation, and the advantages of cross-detection of depth information and gray information of a 3D depth camera are fully utilized.

Description

Technical Field

[0001] The present invention relates to the field of 3D imaging technology, and particularly to a method for automatic face exposure based on a depth camera. Background Art

[0002] Compared with traditional 2D cameras, 3D depth cameras have added depth information, naturally have the advantage of live detection, high recognition accuracy, and are more favored by the market. Therefore, 3D depth cameras have gradually replaced traditional 2D cameras and are applied to various aspects of life, such as face payment, intelligent access control, identity recognition at high-speed rail stations and airports. With the popularization of applications, the demand for low-cost 3D depth camera solutions is also increasing, especially in the case of one main controller connecting multiple cameras.

[0003] The existing automatic exposure scheme for 3D depth cameras is to detect the face contour of a grayscale image. After detecting the face contour, the grayscale value of the optimal contour is selected and given to the exposure control unit for exposure. This scheme uses mature 2D image recognition algorithms, which are mature and stable. However, the face contour recognition algorithm and the multi-contour selection algorithm have high requirements for system resources, that is, high hardware costs. It also does not utilize the advantage of cross-detection of depth information and grayscale information of 3D depth cameras.

[0004] For example, a "method for image exposure adjustment based on face recognition" disclosed in a Chinese patent document, with the publication number CN111242086A, the camera captures and obtains a frame of RAW image in real time; adjusts the entire image to the required exposure value and transmits it to the main controller; the main controller executes the face recognition algorithm; when the camera detects a face, it performs brightness statistics on the face area and non-face area of the image; obtains the brightness of the statistical area, adjusts the exposure according to the face brightness to obtain a frame of image frame1; at the same time, adjusts the exposure according to the brightness of the non-face area to obtain a frame of image frame2; synthesizes the two frames of images to obtain the required image and executes the face recognition algorithm again.

[0005] This scheme uses mature 2D image recognition algorithms, which are mature and stable, but have high requirements for system resources and long detection time, and are not suitable for high-frame-rate detection scenarios. Summary of the Invention

[0006] The present invention mainly solves the problems that the existing technology uses mature 2D image recognition algorithms, has high requirements for system resources and long detection time, and does not utilize the advantage of cross-detection of depth information and grayscale information of 3D depth cameras; provides a method for automatic face exposure based on a depth camera, which determines the exposure target according to the grayscale image, depth information and image scanning algorithm of the 3D depth camera, and performs an automatic exposure scheme, and can obtain an ideal exposure result without huge image recognition and data operation.

[0007] The above technical problems of the present invention are mainly solved by the following technical solutions:

[0008] A face automatic exposure method based on a depth camera, comprising the following steps:

[0009] S1: Perform exposure according to initial value parameters and obtain an image frame;

[0010] S2: Conduct a histogram statistics on the gray values of the image, adjust the exposure parameters of the grayscale image according to the histogram statistics results, and obtain a grayscale image and a depth image with ideal brightness after re-exposure;

[0011] S3: Calculate the reflectivity of each pixel point in turn according to the depth information of the depth image and the gray value of the grayscale image;

[0012] S4: Determine the range area of the face contour according to the empirical value of the human skin reflectivity and the calculated reflectivity of each pixel point;

[0013] S5: Use an image scanning algorithm to scan the range area of the face contour in turn to determine the face contour coordinates and gray values; According to the nearest priority principle, select the gray value of the nearest contour to calculate the exposure parameters and perform re-exposure;

[0014] S6: Output the image frame after automatic exposure based on the face.

[0015] This solution automatically obtains a grayscale image with ideal brightness through histogram statistics for exposure, then calculates the reflectivity, determines the approximate range of the face contour according to the human skin reflectivity range, and then confirms the face position through a feature point scanning algorithm combining grayscale and depth, and calculates the exposure parameters for exposure according to the gray value of the face. Without large-scale image recognition and data operations, an ideal exposure result can be obtained, making full use of the cross-detection advantages of the depth information and gray information of the 3D depth camera.

[0016] Preferably, the process of adjusting the exposure parameters of the grayscale image according to the histogram statistics results includes:

[0017] S201: Divide the obtained grayscale image into pixels according to the gray intervals delimited by the histogram, and count the number of pixels in each interval;

[0018] S202: Count the number of overexposed points. If it is greater than the set overexposed point threshold, reduce the exposure time by a factor of 0.3; Repeat the process of S202 until the number of overexposed points is less than or equal to the set overexposed point threshold;

[0019] S203: Judge the maximum interval in which 80% of the pixel gray values are distributed in ascending order of gray values;

[0020] S204: Calculate the exposure time proportionality coefficient from the maximum grayscale interval to the target grayscale interval, and predict whether the number of overexposed points after adjustment exceeds the set overexposed point threshold according to the exposure time proportionality coefficient. If so, reduce the exposure time proportionality coefficient; if not, keep the exposure time proportionality coefficient;

[0021] S205: After adjusting the exposure time according to the exposure time proportionality coefficient, take an exposure image to obtain a grayscale image with ideal brightness.

[0022] Performing a histogram statistics on the grayscale values of the image can clearly show the grayscale distribution interval and proportion of the image. According to the grayscale distribution interval and proportion, the exposure proportionality coefficient to reach the target grayscale interval can be calculated.

[0023] Preferably, calculate the reflectivity of each pixel point according to the reflectivity formula:

[0024]

[0025] where G is the grayscale value of this point;

[0026] K is the attenuation coefficient, K = k1 + k2 + k3;

[0027] k1 is the transmittance of the laser cover plate;

[0028] k2 is the transmittance of the lens;

[0029] k3 is the conversion rate of the sensor;

[0030] P is the laser emission power,

[0031] f is the object reflectivity;

[0032] d is the object distance from the target to the lens.

[0033] Calculate the reflectivity of each point for comparison with the empirical value to judge the face contour.

[0034] Preferably, the process for determining the range area of the face contour is as follows:

[0035] a. When the reflectivities of a continuous number of pixel points scanned and detected are all within the range of the empirical value of the human skin reflectivity, and the depth difference is within the set range, then mark this pixel point and proceed to the next step; otherwise, continue to scan and detect;

[0036] b. Calculate the actual interval of the pixel mapped onto the image according to the depth value of the marked pixel point;

[0037] c. Based on the calculated actual interval, calculate the face contour according to 0.5 times the empirical value of the face size, and confirm the number of pixels in the range area of the face contour; when the number of pixels here is greater than or equal to 0.5 times the empirical value of the number of pixels of the face contour, it is determined that there may be a face area;

[0038] d. According to the calculated number of pixels, perform horizontal and vertical offsets, detect several pixels within this range area, and count the proportion of the reflectivity falling within the empirical value range of the human skin reflectivity;

[0039] e. Determine whether this range area is the range area of the face contour by comparing the statistically obtained proportion with the set proportion. If the statistically obtained proportion is greater than or equal to the set proportion, it is determined that this range area is the range area of the face contour, mark it, and proceed to step S5. Otherwise, return to step a to start scanning again.

[0040] Test the human skin reflectivity measurements for different skin colors and different environments, and considering the influence of measurement errors, determine the approximate range of the face contour according to the empirical value of the human skin reflectivity.

[0041] Preferably, the calculation process of the actual interval is as follows:

[0042]

[0043] where x is the horizontal actual interval of the pixel mapped onto the image;

[0044] y is the vertical actual interval of the pixel mapped onto the image;

[0045] S x is 1 / 2 of the horizontal value of the camera field of view angle;

[0046] S y is 1 / 2 of the vertical value of the camera field of view angle;

[0047] D is the depth value of this point;

[0048] P x is 1 / 2 of the horizontal value of the image resolution;

[0049] P y is 1 / 2 of the vertical value of the image resolution.

[0050] Calculate the actual interval of the pixel mapped onto the image according to the depth value of the marked point.

[0051] Preferably, the calculation process of the face contour is as follows:

[0052]

[0053] where x is the horizontal actual interval of the pixel mapped onto the image;

[0054] y is the vertical actual interval where the pixel is mapped onto the image;

[0055] X n is the number of horizontal pixels of the face contour;

[0056] Y n is the number of vertical pixels of the face contour;

[0057] X j is 1 / 2 of the horizontal empirical value of the face size;

[0058] Y j is 1 / 2 of the vertical empirical value of the face size.

[0059] According to the calculated pixel interval coefficient and 0.5 times of the face size empirical value, calculate the number of pixels in the face contour confirmation range.

[0060] Preferably, step S5 includes the following steps:

[0061] S501: Use the scanning algorithm to scan the grayscale image horizontally in sequence to find the suspected eye positions with symmetric grayscale;

[0062] S502: Calculate the horizontal distance of the suspected eye position with symmetric grayscale according to the depth information and the field of view angle, and judge whether it is within the empirical value range of the eye distance. If so, enter the next detection. If not, return to step S501 to continue scanning;

[0063] S503: According to the eye position determined in step S502, based on the depth value, the field of view angle, and the empirical value of the distance between the eyes and the nose on the face, find the height of the nose on the depth map compared to the two cheeks on both sides; If the empirical value is satisfied, it indicates that this area is the face area, record the grayscale value and the depth value at the nose, and make a mark.

[0064] S504: Repeat steps S501 to S503 until the scanning is completed; According to the recorded depth information of the face nose, select the grayscale value of the nose of the face with the smallest depth value and output it to the exposure time control unit.

[0065] The system adjusts the exposure parameters by expanding or shrinking in proportion according to this grayscale value and the target grayscale interval (800 LSB to 1500 LSB). At this time, the influence of overexposed points is no longer considered.

[0066] The beneficial effects of the present invention are:

[0067] This solution obtains a grayscale image with ideal brightness through automatic exposure by statistical analysis of the grayscale histogram, then calculates the reflectance, determines the approximate range of the face contour based on the range of human skin reflectance, and then confirms the face position through a feature point scanning algorithm that combines grayscale and depth. The exposure parameters are calculated based on the grayscale value of the face for exposure. Without the need for large-scale image recognition and data operations, an ideal exposure result can be obtained, making full use of the cross-detection advantages of the depth information and grayscale information of the 3D depth camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 is a flowchart of a method for automatic face exposure based on a depth camera according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0069] The technical solution of the present invention will be further specifically described below through embodiments in conjunction with the accompanying drawings.

[0070] Embodiment:

[0071] A method for automatic face exposure based on a depth camera in this embodiment, as Figure 1 shown, includes the following steps:

[0072] S1: Perform exposure according to the initial value parameters and obtain an image frame.

[0073] The initial exposure parameters are determined. Taking a certain project as an example, the depth camera module is installed at the B-pillar position of the car for vehicle owner face recognition to unlock the door. The main ranging range is 0.2 meters to 0.8 meters, and the extreme ranging range is 0.1 meter to 1.2 meters. The initial exposure parameters are set based on a test subject with a normal skin color at 0.5 meters, and the grayscale value at the cheek is set to about 1150 LSB. The repetition frequency of the exposure parameters obtained by testing at this time is 300.

[0074] A depth map and a grayscale map are calculated by integrating two sets of shutter and laser signals. In this example, a set of laser signals of the depth camera contains 12 groups, and each group contains several laser single pulse signals, that is, the repetition frequency. The repetition frequency obtained by the initial value test is 300. After the depth camera exposes according to the exposure parameters, an initial depth map and grayscale map are obtained.

[0075] S2: Perform histogram statistics on the grayscale values of the image, adjust the exposure parameters of the grayscale map according to the statistical results, and obtain a grayscale map and a depth map with ideal brightness after re-exposure.

[0076] Performing histogram statistics on the grayscale values of the image can clearly know the grayscale distribution interval and proportion of the image. According to the grayscale distribution interval and proportion, the exposure proportional coefficient to reach the target grayscale interval can be calculated. According to the statistical analysis of the face recognition algorithm, the optimal face grayscale interval in this example is 800 LSB to 1500 LSB.

[0077] The proportion refers to the proportion of the grayscale distribution. In this example, the grayscale value distribution range of 80% of the entire grayscale image is taken.

[0078] The interval division of the histogram statistics is generally divided by the grayscale ratio or the grayscale range size. In this example, combined with the background noise, the grayscale adjustment ratio, the linear region range, and the overexposure range are divided into a total of 12 intervals, which are: 0 to 50 LSB, 51 LSB to 200 LSB, 201 LSB to 400 LSB, 401 LSB to 600 LSB, 601 LSB to 800 LSB, 801 LSB to 1000 LSB, 1001 LSB to 1500 LSB, 1501 LSB to 2000 LSB, 2001 LSB to 2500 LSB, 2501 LSB to 3200 LSB, 3201 LSB to 3600 LSB, 3601 LSB to 4096 LSB.

[0079] Among them, 0 to 50 LSB mainly statistics the background noise, and 3601 LSB to 4096 LSB statistics the overexposed points. According to the empirical value, the overexposed point threshold set in this example is 1200.

[0080] Determination of the overexposed point threshold: In the measurement scene, there are high-reflectivity objects, such as glasses, earrings, necklaces, etc. These high-reflectivity objects are likely to confuse the overexposed points generated by them with the overexposed points caused by the proximity. If the overexposure of the test target is caused by the proximity, the exposure time needs to be reduced, while for the overexposure of the high-reflectivity objects, since the measurement target is not these high-reflectivity objects, no treatment is required. The algorithm eliminates the influence of high-reflectivity by setting the overexposed point threshold.

[0081] After testing, the maximum number of overexposed points of ornaments and glasses in the normal measurement scene will not exceed 1000. This normal scene means that the measurement target is greater than 0.35 meters. If it is less than this distance, it is also processed as overexposure at close range. A certain threshold is reserved. We set the overexposed point threshold to 1200. The judgment basis is whether the grayscale value is greater than 3600 LSB. If it is greater, it is an overexposed point.

[0082] The steps of automatic exposure according to the histogram grayscale statistics results are as follows:

[0083] a. The obtained grayscale image is pixel-divided according to the grayscale intervals delimited by the histogram, and the number of pixels in each interval is counted.

[0084] b. Count the number of overexposed points, calculate whether the number of pixels in the interval of 3601 LSB to 4096 LSB is greater than the set overexposed point threshold of 1200. If it is greater than this threshold, the exposure time is reduced by 0.3 times the empirical value, and then exposed again and the overexposed points are counted. If it continues to be greater than the set threshold, continue to adjust until the overexposed points are lower than the set threshold. On the contrary, if the number of overexposed points is less than the threshold, directly enter the next step.

[0085] c. Judging the maximum interval of the gray value distribution of 80% of the pixels in ascending order of gray value.

[0086] The resolution of the grayscale image is 640*480, with a total of 307,200 pixels. Judging the maximum interval of the distribution of 80% of the pixels in ascending order, that is, first judging whether the number of pixels within 50 LSB is greater than 80%. If it is greater, the maximum interval is 0 to 50 LSB. If not, judge whether the number of pixels within the interval of 0 to 200 LSB is approximately 80% of the total number of pixels. If it is greater, the maximum interval is 51 LSB to 200 LSB. If not, continue to judge from low to high until the maximum interval of the distribution of 80% of the pixels is determined.

[0087] d. Calculate the proportionality coefficient from the maximum gray interval to the target gray interval, and predict whether the number of overexposed points exceeds the set threshold after adjustment according to this proportionality coefficient. If it exceeds, reduce this coefficient; otherwise, do not make adjustments.

[0088] For example, if 80% of the gray value range of the current grayscale image is within 400 LSB, and the target gray value range is 800 to 1500 LSB, the laser exposure time adjustment ratio is 1500 LSB / 400 LSB = 3.75. After the exposure time is increased by 3.75 times, the gray values of all non-overexposed points are enlarged by 3.75 times proportionally. Therefore, it is necessary to predict the number of overexposed points after adjustment. The calculation is as follows: 3600 LSB / 3.75 = 960 LSB, that is, it is necessary to check the number of points with gray values greater than 960 LSB. The overexposed point threshold is 1200, leaving a certain margin. Here, 1000 is selected. If the number of points with gray values greater than 960 LSB is less than 1000, the exposure time can be adjusted in the way of expanding by 3.75. If it is greater, the adjustment multiple of the exposure time needs to be reduced until it is predicted that there will be no overexposure after the exposure modification.

[0089] e. After adjusting the exposure time according to this coefficient, take a picture during exposure to obtain a grayscale image with the ideal brightness.

[0090] S3: According to the depth information of the depth map and the gray value of the grayscale map, calculate the reflectivity of each pixel point in turn.

[0091] According to the reflectivity calculation formula:

[0092]

[0093] Among them, G is the gray value of this point; unit: nW / mm 2 / LSB;

[0094] K is the attenuation coefficient, K = k1 + k2 + k3;

[0095] Transmittance of the k1 laser cover plate;

[0096] Transmittance of the k2 lens;

[0097] Conversion rate of the k3 sensor;

[0098] P is the laser emission power, unit: W;

[0099] f is the reflectivity of the object;

[0100] d is the object distance from the target to the lens, that is, the depth value at this point, unit mm.

[0101] Among them, k1 = 0.95, k2 = 0.91, k3 = 0.13. By using a optical power meter and the laser homogenizing angle, the power P of the laser at the distance d can be calculated as P = 3.38W / (d*d).

[0102] Based on the above known parameters, the reflectivity of the target object can be solved.

[0103] The reflectivities of human faces with different skin colors are tested separately and the records are as follows:

[0104]

[0105] The reflectivities of human skin with different skin colors and in different environments are measured. Considering the influence of measurement errors, the range of human skin reflectivity in this example is determined to be between 0.35 and 0.50.

[0106] S4: According to the empirical value of human skin reflectivity and combined with the reflectivities of each pixel point calculated, determine the range of the face contour.

[0107] In this example, the reflectivities of human skin with different skin colors and in different environments are measured. Considering the influence of measurement errors, the range of human skin reflectivity in this example is determined to be between 0.35 and 0.50.

[0108] According to empirical statistics, the length of a general human face is from 18.5 cm to 22 cm, and the width is from 12 cm to 14 cm. Therefore, the length and width of a general human face are about 20*13 cm.

[0109] Considering the influence of angles and hair, in this example, the face contour is initially confirmed according to the reflectivities of several points within a range of 0.6 times the size. After confirmation, it is marked and output within a range of 1.5 times the size.

[0110] Taking a certain project as an example, the steps for confirming the face contour range are as follows:

[0111] a. When the reflectivities of several consecutive pixel points are all between 0.35 and 0.50 and the depth difference is within the set range, then mark these points for the next step of processing. Otherwise, continue scanning.

[0112] b. Calculate the actual interval of pixels mapped onto the image based on the depth value of the marked point. In this example, the field of view angle of the camera is 70° horizontally and 50° vertically. Given the depth value of this point as D, then based on the image resolution of 640*480 and the field of view angle of 70°*50°, calculate the actual distance of the pixel interval mapped corresponding to this depth: Let x be the horizontal and y be the vertical.

[0113] The calculation process of the actual interval is as follows:

[0114]

[0115] where x is the horizontal actual interval of pixels mapped onto the image;

[0116] y is the vertical actual interval of pixels mapped onto the image;

[0117] S x is 1 / 2 of the horizontal value of the camera's field of view angle;

[0118] S y is 1 / 2 of the vertical value of the camera's field of view angle;

[0119] D is the depth value of this point;

[0120] P x is 1 / 2 of the horizontal value of the image resolution;

[0121] P y is 1 / 2 of the vertical value of the image resolution.

[0122] In this embodiment:

[0123]

[0124] c. Based on the pixel interval coefficient calculated in the above step, calculate the number of pixels in the face contour confirmation range according to 0.5 times the empirical value of the face size, which is 100*65 mm. When the number of pixels is greater than or equal to 0.5 times the empirical value of the face contour, it is determined that there may be a face area.

[0125] Let the number of horizontal pixels be Xn and the number of vertical pixels be Yn;

[0126]

[0127] where x is the horizontal actual interval of pixels mapped onto the image;

[0128] y is the vertical actual interval of pixels mapped onto the image;

[0129] X n is the number of horizontal pixels of the face contour;

[0130] Y nis the number of vertical pixels of the face contour;

[0131] X j is 1 / 2 of the horizontal empirical value of the face size;

[0132] Y j is 1 / 2 of the vertical empirical value of the face size.

[0133] In this embodiment:

[0134]

[0135] d calculates the number of pixels according to the above steps, performs horizontal and vertical offsets, detects several points within this range, and counts the proportion of the reflectance falling within the skin reflectance range.

[0136] e determines whether this area is the face area according to the set proportion. If so, it is marked and proceeds to the next judgment. If not, it continues to scan from step a.

[0137] S5: Use the image scanning algorithm to sequentially scan the range area of the face contour, determine the face contour coordinates and grayscale values; according to the nearest priority principle, select the grayscale value of the nearest contour for exposure parameter calculation and re-exposure.

[0138] Face contour detection steps:

[0139] Calculate the reflectance according to the grayscale value and depth value, and determine the face contour range based on the empirical range of skin reflectance.

[0140] Taking a certain project as an example, a tester with normal skin color stands 0.58 meters away from the lens. According to the reflectance calculation formula G = K * f * P / (d * d), (G is the grayscale value of this point, unit: nW / mm^2 / LSB; K is the attenuation coefficient, K = k1 + k2 + k3, k1 is the laser cover transmittance, k2 is the lens transmittance, k3 is the sensor conversion rate; P is the laser emission power, unit: W; f is the object reflectance; d is the object distance from the target to the lens, that is, the depth value of this point, unit mm), the skin reflectance of 0.4 is calculated. According to the method of step S4, the approximate contour range of the face is marked.

[0141] S501: Use the scanning algorithm to horizontally scan the grayscale image sequentially to find the position of the human eyes with symmetric grayscales.

[0142] S502: Calculate the horizontal distance of the suspected human eyes with symmetric information found in step S501 according to the depth information and field of view angle, and filter according to the empirical value of the human eye distance. If so, proceed to the next detection. If not, continue to scan.

[0143] Taking a certain project as an example, the tester stands 0.58 meters away from the camera. At this time, the distance between horizontal pixel points is 0.0022*D according to what is obtained in step S4, where D is the corresponding depth value. Substituting it in, the actual pixel interval is 0.0022*0.58 which equals 0.001276 meters, that is, 1.276mm.

[0144] According to the empirical value, the distance between the two eyes of the human body is between 56mm and 70mm. Considering the measurement deviation, in actual use, it is taken as 50mm to 75mm. The minimum number of corresponding pixels is: 50 / 1.27639≈39, and the maximum is: 75 / 1.276≈59.

[0145] Substitute into the suspected eye positions in the previous step, and count whether the number of pixels between the two eyes is within this range. If it is, proceed to the next step; if not, return to step S501 to continue scanning.

[0146] S503: According to the eye positions determined in step S502, based on the depth value, the field of view angle, and the empirical value of the distance between the eyes and the nose on the face, find the height of the nose on the depth map compared to the two cheeks on both sides. If it meets the empirical value, it indicates that this area is the face area, record the gray value and depth value at the nose, and make a mark.

[0147] According to the empirical value, the nose length is 60 to 75mm, and the height is about 20mm. Determine a reasonable range of nose height and length according to actual tests. According to the eye positions determined in the previous step and the set nose length range, calculate the pixel coordinates at the tip of the nose, that is, the horizontal coordinate is the center of the two eyes, and the vertical coordinate is 65mm downward from the center of the two eyes. The number of offset pixels is the distance divided by the pixel interval. According to the pixel interval of 1.276mm obtained in S502, substitute it in to calculate the number of offset pixels: 65 / 1.276≈51. Then calculate the average depth of this coordinate point (the tip of the nose) and the 8 surrounding points, find the difference from the depths at the two cheeks on both sides, and judge whether the difference is within the set nose height range. If it is, it is the facial contour, record the nose depth value and gray value and mark and store them. The coordinates of the two cheeks on both sides are determined. The abscissa is the center coordinates of the left and right eyes, and the ordinate is the ordinate of the tip of the nose. The cheek depth is also the average value of the coordinate point and the 8 surrounding points.

[0148] S504: Repeat steps S501 to S503 until the scanning is completed. Then, according to the recorded face nose depth information, select the gray value of the nose of the face with the smallest depth value and output it to the exposure time control unit.

[0149] The system expands or shrinks and adjusts the exposure parameters according to the ratio of this gray value to the target gray value range (800LSB to 1500LSB). At this time, the influence of overexposed points is no longer considered.

[0150] S6: Output the image frame after automatic exposure based on the face.

[0151] It should be understood that the embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims of this application.

Claims

1. An automatic face exposure method based on a depth camera, characterized in that, It includes the following steps: S1: Expose according to the initial value parameters and obtain an image frame; S2: Perform histogram statistics on the gray values of the image, adjust the exposure parameters of the grayscale image according to the histogram statistics results, and obtain a grayscale image and a depth image with ideal brightness after re-exposure; S3: The point gray value of each pixel point is equal to the product of the attenuation coefficient, the laser emission power, and the object reflectivity divided by the square of the object distance from the target to the lens. The attenuation coefficient is equal to the sum of the laser cover plate transmittance, the lens transmittance, and the sensor conversion rate; According to the depth information of the depth image and the gray value of the grayscale image, calculate the reflectivity of each pixel point in turn; S4: Determine the range area of the face contour based on the empirical value of the human skin reflectivity and the calculated reflectivity of each pixel point; S5: Use the image scanning algorithm to scan the range area of the face contour in turn to determine the face contour coordinates and gray values; According to the nearest priority principle, select the gray value of the nearest contour for exposure parameter calculation and re-exposure; S6: Output the image frame after automatic exposure based on the face.

2. The automatic face exposure method based on a depth camera according to claim 1, wherein, The process of adjusting the exposure parameters of the grayscale image according to the histogram statistics results includes: S201: Divide the obtained grayscale image into pixels according to the gray intervals defined by the histogram, and count the number of pixels in each interval; S202: Count the number of overexposed points. If it is greater than the set overexposed point threshold, reduce the exposure time by a factor of 0.3; Repeat process S202 until the number of overexposed points is less than or equal to the set overexposed point threshold; S203: Judge the largest interval in which 80% of the pixel gray values are distributed in ascending order of gray values; S204: Calculate the exposure time proportionality coefficient from the maximum gray interval to the target gray interval, and judge whether the number of overexposed points exceeds the set overexposed point threshold after prediction according to the exposure time proportionality coefficient. If so, reduce the exposure time proportionality coefficient; If not, keep the exposure time proportionality coefficient; S205: Adjust the exposure time according to the exposure time proportionality coefficient and then take a picture after exposure to obtain a grayscale image with ideal brightness.

3. The automatic face exposure method based on a depth camera according to claim 1 or 2, wherein The process of determining the range area of the face contour is as follows: a. When the reflectivities of continuously several pixel points scanned and detected are all within the range of the empirical value of the human skin reflectivity and the depth difference is within the set range, mark the pixel point and proceed to the next step; Otherwise, continue scanning and detecting; b. Calculate the actual interval of the pixel mapped onto the image according to the depth value of the marked pixel point; c. Based on the calculated actual interval, calculate the face contour according to 0.5 times the empirical value of the face size, and confirm the number of pixels in the range area of the face contour; d. According to the calculated number of pixels, perform horizontal and vertical offsets, and detect several pixel points within this range area, and count the proportion of the reflectivities falling within the range of the empirical value of the human skin reflectivity; e. Judge whether this range area is the range area of the face contour by comparing the counted proportion with the set proportion. If the counted proportion is greater than or equal to the set proportion, judge that this range area is the range area of the face contour, mark it, and proceed to step S5. Otherwise, return to step a and start scanning again.

4. A method for automatic face exposure based on a depth camera according to claim 3, characterized in that, The calculation process of the actual interval is as follows: Among them, x is the horizontal actual interval of the pixel mapped onto the image; y is the vertical actual interval of the pixel mapped onto the image; S x It is 1 / 2 of the horizontal value of the camera field of view angle; S y It is 1 / 2 of the vertical value of the camera field of view angle; D is the depth value of this point; P x It is 1 / 2 of the horizontal value of the image resolution; P y It is 1 / 2 of the vertical value of the image resolution.

5. The automatic face exposure method based on a depth camera according to claim 3, characterized in that, The calculation process of the face contour is as follows: Among them, x is the horizontal actual interval of the pixel mapped onto the image; y is the vertical actual interval of the pixel mapped onto the image; X n is the number of horizontal pixels of the face contour; Y n is the number of vertical pixels of the face contour; X j It is 1 / 2 of the horizontal empirical value of the face size; Y j It is 1 / 2 of the vertical empirical value of the face size.

6. A method for automatic face exposure based on a depth camera according to claim 1 or 4 or 5, characterized in that, Step S5 includes the following steps: S501: Use the scanning algorithm to scan the grayscale image horizontally in sequence to find the suspected eye positions with symmetric grayscales; S502: Calculate the horizontal distance of the suspected eye positions with symmetric grayscales according to the depth information and the field of view angle, and determine whether it is within the empirical value range of the eye distance. If so, proceed to the next detection. If not, return to step S501 to continue scanning; S503: According to the eye positions determined in step S502, based on the depth value, the field of view angle, and the empirical value of the distance between the eyes and the nose on the face, find the height of the nose on the depth map compared to the two cheeks on both sides; If the empirical value is satisfied, it means that this area is the face area, record the grayscale value and the depth value at the nose, and make a mark; S504: Repeat steps S501 to S503 until the scanning is completed; According to the recorded face nose depth information, select the grayscale value of the nose of one face with the smallest depth value and output it to the exposure time control unit.

7. A method for automatically exposing a face based on a depth camera according to claim 6, wherein Step S504 includes: Adjust the exposure parameter by expanding or shrinking it proportionally according to the grayscale value and the target grayscale interval.

Images