Exposure method and device based on face brightness

Abstract

The present invention provides an exposure method and device based on face brightness, which relates to the field of image processing technology, including: S1, obtaining an initial image through an image sensor; S2, obtaining the position information of a face frame in the initial image; S3, calculating the photometric block category information according to the position relationship between the face frame and the photometric block and the intersection ratio of the face in each photometric block; S4, calculating the average brightness value of each photometric block according to the initial image; S5, calculating the face brightness value according to the photometric block category information and the average brightness value of the photometric block; S6, adjusting the exposure and gain according to the difference between the face brightness value and the preset target brightness value to obtain the final face brightness, and outputting the final image. By considering the position and ratio relationship between the face frame and the photometric block, the present invention can more accurately calculate the actual brightness of the face, thereby more accurately adjusting the exposure setting, avoiding overexposure or underexposure, and improving the accuracy of exposure.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an exposure method and device based on face brightness. Background Art

[0002] Face detection is now widely used in many fields such as image processing, machine learning and big data. The accuracy of face detection affects the complexity of subsequent image processing. The more appropriate the brightness of the face, the higher the accuracy of detection. Therefore, it is hoped that the brightness of the face is within a suitable range. The brightness of the face is usually controlled by the automatic exposure module.

[0003] Traditional automatic exposure technology usually divides the image into regions for light measurement and sets the weights of different blocks through methods such as histograms. After face detection is introduced, it is unable to integrate the brightness of the face well, especially when the brightness difference between the face and background blocks is relatively large, the face cannot obtain a good brightness.

[0004] Automatic exposure cannot control the brightness of the face well. The fundamental problem is that the face frame and the metering block do not completely overlap.

[0005] The metering window is an area set in the picture for counting brightness, which can be the entire picture or part of it; the metering block divides the entire metering window into several small blocks, and the average brightness of each block can be counted separately.

[0006] The metering window is fixed. Faces will appear in different positions on the screen and their sizes will also change. The statistical metering blocks will not move with the position of the face. When a face appears in multiple metering blocks and the occupancy ratio is not constant, the brightness of the face cannot be accurately calculated using the metering block brightness. Without hardware changes, the face brightness information cannot be directly counted.

[0007] The inability of the automatic exposure algorithm based on block metering to effectively control the brightness of the face has become a problem that needs to be solved urgently. Summary of the invention

[0008] In order to overcome the deficiencies in the background technology, the present invention discloses an exposure method and device based on face brightness.

[0009] In order to achieve the above-mentioned invention object, the present invention adopts the following technical scheme:

[0010] In a first aspect, the present invention provides an exposure method based on face brightness, comprising the following steps:

[0011] S1, acquiring an initial image through an image sensor;

[0012] S2, obtaining the position information of the face frame in the initial image;

[0013] S3. Calculate the photometric block category information based on the positional relationship between the face frame and the photometric blocks and the intersection ratio of the face in each photometric block; there are multiple photometric blocks; the photometric block category information includes the category of the photometric blocks and the number of photometric blocks of each category; the categories include the first-category photometric blocks, the second-category photometric blocks, the third-category photometric blocks, and the fourth-category photometric blocks;

[0014] S4. Calculate the average brightness value of each photometric block according to the initial image;

[0015] S5. Calculate the face brightness value based on the photometric block category information and the average brightness value of the photometric blocks;

[0016] S6. Adjust the exposure and gain according to the difference between the face brightness value and the preset target brightness value to obtain the final face brightness, and output the final image.

[0017] Specifically, step S3 specifically includes the following steps:

[0018] S31. Calculate the intersection area between each photometric block and the face frame according to their positions; the calculation formula for the intersection area S is:

[0019] ;

[0020] wherein, the upper left coordinate of the photometric block is (x1, y1), the lower right coordinate is (x2, y2); the upper left coordinate of the face frame is (x3, y3), the lower right coordinate is (x4, y4); min is the minimum value operator, and max is the maximum value operator;

[0021] S32. Take the ratio of the intersection area to the area of the corresponding photometric block as the face ratio r;

[0022] S33. Compare the face ratio r with the preset ratio threshold, and divide the photometric blocks into different categories according to different ratio comparison conditions and count the number of photometric blocks of each category; the preset ratio threshold includes the first ratio threshold r1 and the second ratio threshold r2.

[0023] Specifically, comparing the face ratio with the preset ratio threshold and dividing the photometric blocks into different categories according to different ratio comparison conditions is specifically:

[0024] When 0 < r ≤ r1 in the photometric block, it is determined that the photometric block is a first-category photometric block;

[0025] When r1 < r ≤ r2 in the photometric block, it is determined that the photometric block is a second-category photometric block;

[0026] When r2 < r ≤ 1 in the photometric block, it is determined that the photometric block is a third-category photometric block;

[0027] When r in the light metering block is 0 and is not horizontally connected to any light metering block of the second category or the third category, the light metering block is determined to be a light metering block of the fourth category;

[0028] The first category of light metering blocks also includes:

[0029] When r=0 in the light metering block and the horizontally connected light metering block is the second light metering block, the light metering block is determined to be a first category light metering block;

[0030] When r=0 in the light metering block and the horizontally connected light metering block is the third light metering block, the light metering block is determined to be a first category light metering block.

[0031] Specifically, step S2 is as follows: identifying the face frame where the face in the initial image is located through a preset face detection model in the host computer, the host computer is a SOC, the face frame is a rectangle, and obtaining the coordinates of the upper left corner and the lower right corner of the rectangle.

[0032] Specifically, step S5 calculates the face brightness according to the photometric block category information and the average brightness information of the photometric blocks by comparing the number of photometric blocks of each category with the preset number of photometric blocks of each category, and using the preset face brightness adjustment method to calculate the face brightness value The preset face brightness adjustment method varies according to the number of metering categories.

[0033] Specifically, step S5 includes: when the number of the first category of metering blocks is less than the first category of metering block number threshold, and the number of the second category of metering blocks and the third category of metering blocks is 0, using the full-screen average brightness value as the face brightness value, and the calculation formula of the full-screen average brightness value is:

[0034] ;

[0035] in, It is the average brightness value of the metering blocks in the picture, and n is the number of metering blocks.

[0036] Specifically, step S5 further includes: when the number of the first category of metering blocks is greater than or equal to the first category of metering block number threshold, and the number of the second category of metering blocks and the third category of metering blocks is 0, using the brightness of the fourth category of metering blocks and the basic brightness adjustment step to calculate the face brightness, and the calculation formula is:

[0037] ;

[0038] in, is the average brightness value of the fourth type of metering block, is the number of the fourth type of light metering blocks, is the basic brightness adjustment step size, is the first adjustment factor.

[0039] Specifically, step S5 further includes: when the number of the first type of photometric blocks and the number of the second type of photometric blocks are both greater than 0, and the number of the third type of photometric blocks is 0, using the first type of photometric blocks as the compensation blocks for the second type of photometric blocks to calculate the face brightness value, and the calculation formula is:

[0040] ;

[0041] wherein, is the number of the first type of photometric blocks, is the brightness value of the first type of photometric blocks; is the number of the second type of photometric blocks, is the brightness value of the second type of photometric blocks, is the second adjustment factor.

[0042] Specifically, step S5 further includes: when the number of the third type of photometric blocks is greater than 0, using the brightness value of the third type of photometric blocks as the face brightness value, and the calculation formula is:

[0043] ;

[0044] wherein, is the number of the third type of photometric blocks, is the brightness value of the third type of photometric blocks.

[0045] In a second aspect, the present invention provides an exposure device based on face brightness, including the following units:

[0046] An image acquisition unit, configured to acquire an initial image through an image sensor;

[0047] A face frame acquisition unit, configured to acquire the position information of the face frame in the initial image;

[0048] A photometric block classification unit, configured to calculate photometric block category information according to the positional relationship between the face frame and the photometric blocks and the intersection ratio of the face in each photometric block; the photometric blocks are multiple; the photometric block category information includes the category of the photometric blocks and the number of photometric blocks of each category; the categories include the first type of photometric blocks, the second type of photometric blocks, the third type of photometric blocks, and the fourth type of photometric blocks;

[0049] Comparing the face ratio with a preset ratio threshold, and classifying the photometric blocks into different categories according to different ratio comparison conditions specifically as:

[0050] When 0 < r ≤ r1 in the photometric blocks, it is determined that the photometric blocks are the first type of photometric blocks;

[0051] When r1 < r ≤ r2 in the photometric block, the photometric block is determined to be a second - category photometric block;

[0052] When r2 < r ≤ 1 in the photometric block, the photometric block is determined to be a third - category photometric block;

[0053] When r is 0 in the photometric block and it is not horizontally connected to any second - category or third - category photometric block, the photometric block is determined to be a fourth - category photometric block;

[0054] The first - category photometric block further includes:

[0055] When r = 0 in the photometric block and the horizontally - connected photometric block is a second photometric block, the photometric block is determined to be a first - category photometric block;

[0056] When r = 0 in the photometric block and the horizontally - connected photometric block is a third photometric block, the photometric block is determined to be a first - category photometric block;

[0057] Among them, the intersection area between each photometric block and the face frame is calculated, and the ratio of the intersection area to the area of the corresponding photometric block is used as the face occupancy ratio r; the preset occupancy ratio thresholds include a first occupancy ratio threshold r1 and a second occupancy ratio threshold r2;

[0058] An average brightness calculation unit for calculating the average brightness value of each photometric block according to the initial image;

[0059] A face brightness calculation unit for calculating the face brightness value according to the photometric block category information and the average brightness value of the photometric block;

[0060] Compare the number of each category of photometric measurements with the preset number of each type of photometric block, and calculate the face brightness value using a preset face brightness adjustment method , and the preset face brightness adjustment method varies according to the number of each category of photometric measurements;

[0061] When the number of the first - category photometric blocks is less than the first - category photometric block number threshold and the number of the second - category and third - category photometric blocks is 0, use the full - screen average brightness value as the face brightness value, and the calculation formula for the full - screen average brightness value is:

[0062] ;

[0063] Among them, is the average brightness value of the photometric blocks in the picture, and n is the number of photometric blocks;

[0064] When the number of the first category of metering blocks is greater than or equal to the first category metering block number threshold, and the number of the second category of metering blocks and the third category of metering blocks is 0, the brightness of the fourth category of metering blocks and the basic brightness adjustment step are used to calculate the face brightness. The calculation formula is:

[0065] ;

[0066] in, is the average brightness value of the fourth type of metering block, is the number of the fourth type of light metering blocks, is the basic brightness adjustment step size, is the first adjustment factor;

[0067] When the number of the first category of metering blocks and the number of the second category of metering blocks are both greater than 0, and the number of the third category of metering blocks is 0, the first category of metering blocks are used as compensation blocks for the second category of metering blocks to calculate the face brightness value, and the calculation formula is:

[0068] ;

[0069] in, is the number of first category metering blocks, is the brightness value of the first category metering block; is the number of second category metering blocks, is the brightness value of the second category metering block, It is the second regulatory factor;

[0070] When the number of the third-category metering blocks is greater than 0, the brightness value of the third-category metering block is used as the face brightness value. The calculation formula is:

[0071] ;

[0072] in, is the number of third category metering blocks, is the brightness value of the third category metering block;

[0073] The face brightness compensation unit is used to adjust the exposure and gain according to the difference between the face brightness value and the preset target brightness value to obtain the final face brightness and output the final image.

[0074] In a third aspect, the present invention provides an electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0075] The present invention provides an exposure method based on face brightness, comprising: S1, obtaining an initial image through an image sensor; S2, obtaining the position information of a face frame in the initial image; S3, calculating the photometric block category information according to the position relationship between the face frame and the photometric block and the intersection ratio of the face in each photometric block; S4, calculating the average brightness value of each photometric block according to the initial image; S5, calculating the face brightness value according to the photometric block category information and the average brightness value of the photometric block; S6, adjusting the exposure and gain according to the difference between the face brightness value and the preset target brightness value to obtain the final face brightness, and outputting the final image. By considering the position and ratio relationship between the face frame and the photometric block, the present invention can more accurately calculate the actual brightness of the face, thereby more accurately adjusting the exposure setting, avoiding overexposure or underexposure, and improving the accuracy of exposure.

[0076] In addition, since the position and size of the face may vary depending on the shooting scene, the present invention can adjust the exposure according to the specific position and size of the face to better adapt to different shooting environments and light conditions.

[0077] In addition, the present invention divides the metering blocks into four types and performs background brightness compensation according to their types and quantities, which can better balance the brightness difference between the face and the background, while avoiding the background being too bright or too dark, making the overall picture more coordinated, thereby improving the quality of the overall image. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0079] Figure 1 is a schematic flow chart of an exposure method based on face brightness according to an embodiment of the present invention;

[0080] Figure 2 is a schematic diagram of light metering block division according to an embodiment of the present invention;

[0081] Figure 3 is a schematic diagram of a light metering block and a face frame provided according to an embodiment of the present invention;

[0082] Figure 4 is a schematic diagram of the relationship between exposure gain and noise suppression factor provided by an embodiment of the present invention;

[0083] Figure 5 is a schematic diagram of an exposure device based on face brightness provided according to an embodiment of the present invention;

[0084] Figure 6 is a schematic diagram of an exposure device based on face brightness provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0085] The present invention can be explained in detail through the following embodiments. The purpose of disclosing the present invention is to protect all technical improvements within the scope of the present invention. In the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "front", "back", "left", "right", etc. indicating directions or positional relationships, they only correspond to the drawings of the present application for the convenience of describing the present invention, and do not indicate or imply that the device or element referred to must have a specific direction.

[0086] Embodiment 1

[0087] refer to Figure 1 This embodiment provides an exposure method based on face brightness, comprising the following steps:

[0088] S1, acquiring an initial image through an image sensor;

[0089] S2, obtaining the position information of the face frame in the initial image;

[0090] Specifically, the face frame is a rectangle, and the position information is the coordinates of the upper left corner and the lower right corner of the face frame.

[0091] The face frame where the face in the initial image is located is identified by a preset face detection model in the host computer, the host computer is a SOC, the face frame is a rectangle, and the coordinates of the upper left corner and the lower right corner of the rectangle are obtained; in this way, the face position information is obtained.

[0092] The host computer is a SOC (System on Chip), such as the CPU in laptops, tablets and smart phones. The host computer uses machine learning or deep learning methods to train a preset face detection model for face detection. Please refer to Chinese patent application publication number CN116152892A "A real-time face detection and recognition method". The specific type of the host computer is not limited here, and all those that can achieve the above functions are protected.

[0093] Currently, the host computers of laptops, tablets, smartphones and other devices on the market all have face detection functions, which can perform face detection well. Obtaining the face coordinates and position from the host computer helps to better adjust the screen brightness, which can derive many other applications, such as face unlocking and video calling. Only when the face brightness in the picture is moderate, the functions derived from face detection can work.

[0094] S3, calculating the photometric block category information according to the positional relationship between the face frame and the photometric block and the intersection ratio of the face in each photometric block; the photometric block category information includes the category of the photometric block and the number of photometric blocks of each category; the categories include the first category photometric block, the second category photometric block, the third category photometric block and the fourth category photometric block;

[0095] Given the position information of the light metering block and the position information of the face frame, the proportion of the face in the light metering block is calculated. When the face frame is calculated as a whole, due to the diversity of the size and position of the face frame, it cannot be obtained simply through the brightness information of the light metering block. Since the position information of the light metering block is known, the face frame is split according to the positional relationship with the light metering block, and is divided into multiple light metering blocks and the problem of solving the proportion of the face in the light metering block is further converted into the problem of solving the proportion of the face block in a single light metering block. Finally, the proportion of the face block in a single light metering block is converted into solving the intersection area of ​​two rectangles. First, it is necessary to determine whether the two rectangles intersect. If they do not intersect, the intersection area is 0. If they intersect, the intersection part is a rectangular area. It is necessary to find the boundary of the intersecting rectangle and calculate the intersection area.

[0096] In this embodiment, the entire initial image is divided into m*m light-metering blocks, and each light-metering block is numbered.

[0097] In this embodiment, the value of m can be adjusted according to actual requirements, and the specific means of dividing and numbering the metering blocks are not limited. This is the existing technology. As long as the technical means can ensure that the metering blocks and the numbers can be one-to-one corresponding, they are protected and no specific limitation is made here.

[0098] Preferably, in this embodiment, the value of n is 8, and the entire screen of the initial image is equally divided into n parts in both the x direction and the y direction, and a total of 8*8=64 light measuring blocks are divided, and each light measuring block is numbered consecutively from left to right and from top to bottom, starting from 1, and the maximum number is 64, with reference to Figure 2 .

[0099] The intersection area S of each light metering block and the face frame is calculated according to their positions, and the ratio of the intersection area to the area of ​​the corresponding light metering block is taken as the face ratio r. The calculation formula of the intersection area S is:

[0100] ;

[0101] Among them, the coordinates of the upper left corner of the metering block are (x1, y1), and the coordinates of the lower right corner are (x2, y2); the coordinates of the upper left corner of the face frame are (x3, y3), and the coordinates of the lower right corner are (x4, y4); min is the minimum value operator, and max is the maximum value operator;

[0102] Reference Figure 3 Assume that the upper left coordinate of the photometric measurement block is (x1, y1), the lower right coordinate is (x2, y2), the upper left coordinate of the face frame is (x3, y3), and the lower right coordinate is (x4, y4).

[0103] First, check whether the two rectangular blocks intersect: If the photometric measurement block is on the left side of the face frame (i.e., x2 < x3), or the photometric measurement block is on the right side of the face frame (i.e., x1 > x4), and the photometric measurement block is above the face frame (i.e., y2 > y3), or below (i.e., y1 < y4), then the two rectangular blocks do not intersect, that is, there is no overlapping area between the current photometric measurement block and the face frame.

[0104] If there is an intersecting area between the photometric measurement block and the face frame, it is necessary to find the boundaries of the intersecting part. The left boundary of the intersecting part is max(x1, x3), and the right boundary is min(x2, x4); the upper boundary of the intersecting part is min(y1, y3), and the lower boundary is max(y2, y4). The area of the intersecting part is:

[0105] ;

[0106] Among them, min is the minimum value operator, and max is the maximum value operator.

[0107] Figure 3 Specifically, it is a schematic diagram of the photometric measurement block located at the upper left corner of the face frame. Points A and B are the upper left and lower right corners of the photometric measurement block respectively, points C and D are the upper left and lower right corners of the face frame respectively, and the intersecting area between the photometric measurement block and the face frame is marked as CEBF. The area of the rectangular area CEBF can be solved by the above method , and then the face proportion r of the face frame in the photometric measurement block is obtained. The calculation formula is:

[0108] ;

[0109] Compare the face proportion r with the preset proportion threshold, and divide the photometric measurement blocks into different categories according to different proportion comparison conditions and count the number of photometric measurement blocks in each category; the preset proportion threshold includes the first proportion threshold r1 and the second proportion threshold r2;

[0110] Specifically, when 0 < r ≤ r1 in the photometric measurement block, it is determined that the photometric measurement block is a first-category photometric measurement block;

[0111] When r1 < r ≤ r2 in the photometric measurement block, it is determined that the photometric measurement block is a second-category photometric measurement block;

[0112] When r2 < r ≤ 1 in the photometric measurement block, it is determined that the photometric measurement block is a third-category photometric measurement block;

[0113] When r in the light metering block is 0 and is not horizontally connected to any light metering block of the second category or the third category, the light metering block is determined to be a light metering block of the fourth category;

[0114] The first category of light metering blocks also includes:

[0115] When r=0 in the light metering block and the horizontally connected light metering block is the second light metering block, the light metering block is determined to be a first category light metering block;

[0116] When r=0 in the light metering block and the horizontally connected light metering block is the third light metering block, the light metering block is determined to be a first category light metering block.

[0117] In this embodiment, a first ratio threshold and a second ratio threshold are preset, and the types of light metering blocks are divided into four categories according to the relationship between the first ratio threshold and the second ratio threshold of the face ratio:

[0118] When the proportion of faces in a certain light metering block is greater than zero and less than the first proportion threshold, or the proportion of faces in a certain light metering block is equal to zero and the proportion of faces in the horizontally connected light metering blocks is greater than the first proportion threshold, the current light metering block is determined to be a first category light metering block; this situation indicates that the face in the current light metering block is small, or the current light metering block is located near the face, and the brightness of the light metering block is close to the background brightness, which can be used to compensate for the brightness of the face.

[0119] When the proportion of faces in a certain metering block is greater than the first proportion threshold and less than the second proportion threshold, the current metering block is determined to be a second category metering block, and it is considered that there is a face in the current metering block. The brightness of the metering block is significantly different from the brightness of the face, and background compensation is required.

[0120] When the proportion of the face in a certain metering block is greater than the second proportion threshold, the current metering block is determined to be a third category metering block, the face occupies the main area of ​​the current metering block, and the brightness of the current metering block is very different from the brightness of the face, and no background compensation is required.

[0121] If the proportion of faces in a certain metering block is equal to zero, and the proportion of faces in horizontally connected metering blocks is less than the first proportion threshold, the current metering block is determined to be a fourth category metering block, and it is considered that there is no face in the current metering block.

[0122] When the proportion of human faces in a certain light metering block is equal to zero, and the proportion of human faces in the horizontally connected light metering blocks is greater than the first proportion threshold, the current light metering block is also determined to be a first category light metering block. Therefore, the first light metering block also includes the following two situations:

[0123] When r=0 in the light metering block and the horizontally connected light metering block is the second light metering block, the light metering block is determined to be a first category light metering block;

[0124] When r=0 in the light metering block and the horizontally connected light metering block is the third light metering block, the light metering block is determined to be a first category light metering block.

[0125] In this example, it is preferred that the first proportion threshold is set to 5%, and the second proportion threshold is set to 80%. In practical applications, the first proportion threshold and the second proportion threshold can be set according to actual conditions, and this embodiment does not make specific limitations.

[0126] For example, when the face accounts for 2% of a certain metering block, the current metering block is determined to be a first-category metering block. When the face accounts for 20% of a certain metering block, the current metering block is determined to be a second-category metering block. When the face accounts for 90% of a certain metering block, the current metering block is determined to be a third-category metering block. When the face accounts for 0% of a certain metering block and is not horizontally connected to any second-category metering block or third-category metering block, the current metering block is determined to be a fourth-category metering block.

[0127] As described above, in this embodiment, there are a total of 64 light metering blocks. All light metering blocks are traversed and divided into the above 4 categories according to the above method, and the number of light metering blocks in each category is counted.

[0128] S4, calculating the average brightness value of each light-measuring block according to the initial image;

[0129] Specifically, according to the position of each light metering block, the brightness and quantity of all pixels in the block are accumulated, and the brightness accumulation value is divided by the quantity accumulation value to obtain the average brightness value in each light metering block.

[0130] S5, calculating the brightness value of the face according to the photometric block category information and the average brightness value of the photometric block;

[0131] Specifically, the face brightness is calculated according to the photometric block category information and the average brightness information of the photometric blocks as follows: the number of photometric blocks of each category is compared with the number of preset photometric blocks of each category, and the face brightness value is calculated using the preset face brightness adjustment method. The preset face brightness adjustment method varies according to the number of metering categories.

[0132] Specifically, when the number of the first category of light metering blocks is less than the first category of light metering block number threshold, and the number of the second category of light metering blocks and the third category of light metering blocks is 0, the full-screen average brightness value is used as the face brightness value, and the calculation formula of the full-screen average brightness value is:

[0133] ;

[0134] in, It is the average brightness value of the metering blocks in the picture, and n is the number of metering blocks.

[0135] Specifically, when the number of the first category of metering blocks is greater than or equal to the first category of metering block number threshold, and the number of the second category of metering blocks and the third category of metering blocks is 0, the brightness of the fourth category of metering blocks and the basic brightness adjustment step are used to calculate the face brightness, and the calculation formula is:

[0136] ;

[0137] in, is the average brightness value of the fourth type of metering block, is the number of the fourth type of light metering blocks, is the basic brightness adjustment step size, is the first adjustment factor.

[0138] Specifically, when the average brightness of the first category of light metering blocks is greater than or equal to the fourth category of light metering blocks, Take a positive number. When the average brightness of the first category metering block is less than that of the fourth category metering block, Take a negative number.

[0139] Specifically, when the number of the first category of light metering blocks and the number of the second category of light metering blocks are both greater than 0, and the number of the third category of light metering blocks is 0, the first category of light metering blocks are used as compensation blocks for the second category of light metering blocks to calculate the face brightness value, and the calculation formula is:

[0140] ;

[0141] in, is the number of first category metering blocks, is the brightness value of the first category metering block; is the number of second category metering blocks, is the brightness value of the second category metering block, is the second regulatory factor.

[0142] Specifically, when the number of the third category light metering blocks is greater than 0, the brightness value of the third category light metering blocks is used as the face brightness value, and the calculation formula is:

[0143] ;

[0144] in, is the number of third category metering blocks, It is the brightness value of the third category metering block.

[0145] When the brightness difference between the background and the face is relatively large, the global average brightness calculated by setting the weight cannot represent the brightness of the face well. Therefore, this embodiment further proposes a differential metering method to compensate the brightness of the face frame obtained by metering. The brightness value of the face after compensation is compared with the preset target brightness value, and the exposure and gain are adjusted to make the brightness value of the face close to the preset target brightness value. This embodiment calculates the average brightness of the face in four cases according to the number of each type of metering blocks.

[0146] The first case: There are first-category metering blocks in the picture, and the number of first-category metering blocks is less than the number threshold, and there are no second-category metering blocks and third-category metering blocks. At this time, the face occupies a very small proportion in the picture. After increasing the face brightness, the overall change range of the picture will be relatively extreme, so no special face processing is performed. Use the average brightness of the entire picture as the face brightness:

[0147] ;

[0148] in, It is the average brightness value of the metering blocks in the picture, and n is the number of metering blocks in the picture.

[0149] The second case: There are first-category metering blocks in the picture, the number of which is greater than or equal to the number threshold, and there are no second-category metering blocks and third-category metering blocks. At this time, the face occupies a relatively large proportion in the picture, and the picture brightness needs to be adjusted to see the face clearly. Use the brightness of the fourth-category metering block and the basic brightness adjustment step to calculate the face brightness:

[0150] ;

[0151] in, is the average brightness value of the fourth type of metering block, is the basic brightness adjustment step size, is the adjustment factor. When the average brightness of the first category metering block is greater than or equal to the fourth category metering block, Take a positive number. When the average brightness of the first category metering block is less than that of the fourth category metering block, Take a negative number.

[0152] In this embodiment, Set to 8, The absolute value of is set to 1 and the quantity threshold is set to 3.

[0153] The third case: the screen has the first type of metering blocks and the second type of metering blocks, but no third type of metering blocks. At this time, the face occupies a certain proportion in the screen, and the first type of blocks are used as compensation blocks for the second type of blocks. The formula is as follows:

[0154] ;

[0155] in, is the number of first category metering blocks, is the brightness value of the first category metering block; is the number of second category metering blocks, is the brightness value of the second category metering block, is the second regulatory factor.

[0156] In this embodiment, The value range of is between 0 and 0.2. This setting has two main purposes: first, no matter what the relationship between the brightness of the first category metering block and the brightness of the second category metering block is, the brightness of the second category metering block is always reversely compensated to obtain a reasonable face brightness. Second, the degree of reverse compensation is kept within a certain range to prevent the face from being too dark or too bright. Preferably, in this embodiment, The value of is 0.1.

[0157] The fourth case: When there is a third-category metering block on the screen, the brightness of the third-category metering block is directly used as the face brightness. The formula is as follows:

[0158] ;

[0159] in, is the number of third-category metering blocks in the picture. It is the brightness value of the third category metering block.

[0160] In this embodiment, the entire picture is divided into 8x8 light-metering blocks. When a face is detected in the picture, the position of the face is marked. Figure 4 As shown, the dotted boxes with digital marks are different face frame positions. In this embodiment, the first proportion threshold is set to 5%, the second proportion threshold is set to 80%, and the number threshold of the first category metering blocks is set to 3.

[0161] When calculating the face brightness corresponding to face frame 1, the face proportions in photometric blocks 52 and 53 are both 100%, which is greater than the second proportion threshold and is determined to be a photometric block of the third category. Face frame 1 meets the fourth situation, and the average brightness of blocks 51 and 52 is used as the face brightness.

[0162] When calculating the face brightness corresponding to face frame No. 2, the face ratios in photometric blocks 36, 37, and 38 are 60%, 60%, and 4%, respectively. The face ratios of blocks 36 and 37 are between the first ratio threshold and the second ratio threshold, and are determined to be photometric blocks of the second category. The face ratio of block 38 is less than the first ratio threshold, and is determined to be a photometric block of the first category. Photometric block 35 is horizontally connected to the photometric block of the second category, and is determined to be a photometric block of the first category. Face frame No. 2 meets the third situation, and uses the photometric blocks of the first category and the second category to perform reverse compensation to calculate the face brightness.

[0163] When calculating the face brightness corresponding to face frame No. 3, the face proportion in metering blocks 20 and 21 is 4%, which is less than the first proportion threshold. They are determined to be metering blocks of the first category, and the number is less than the number threshold, which meets the first situation. The global average brightness is used as the face brightness.

[0164] When calculating the facial brightness corresponding to face No. 4, the face proportion in metering blocks 14, 15, 22 and 23 is 4%, which is less than the first proportion threshold. They are determined to be metering blocks of the first category, and the number is greater than the number threshold, which meets the second situation. The brightness of the metering block of the fourth category and the basic brightness adjustment step are used to calculate the facial brightness.

[0165] S6. Adjust exposure and gain according to the difference between the face brightness value and a preset target brightness value to obtain a final face brightness, and output a final image.

[0166] Specifically, adjusting the exposure and gain according to the relationship between the face brightness value and the preset target brightness value to obtain the final face brightness specifically includes:

[0167] If the difference between the face brightness value and the preset target brightness value is less than or equal to the preset brightness difference threshold, it is considered that the brightness of the current picture meets the requirement and the picture is not adjusted;

[0168] If the face brightness value is less than the preset target brightness value, and the difference is greater than the preset brightness difference threshold, it is considered that the current image is dark, and the exposure and gain are gradually increased to make the face brightness value gradually approach the preset target brightness value;

[0169] If the face brightness value is greater than the preset target brightness value, and the difference is greater than the preset brightness difference threshold, it is considered that the current image is too bright, and the exposure and gain are gradually reduced to make the face brightness value gradually approach the preset target brightness value;

[0170] It is understandable that in digital image processing, the brightness value is a unitless value, which is based on an 8-bit (1-byte) grayscale representation. The brightness value ranges from 0 to 255, where 0 represents complete underexposure, which is the lower limit of photosensitive pixel imaging; 255 represents complete overexposure, which is the upper limit of photosensitive pixel imaging.

[0171] The preset target brightness value is the target value of the average brightness of the picture, and the preset target brightness value can be adjusted according to actual application requirements; preferably, in this embodiment, the preset target brightness value is set to 128. Under this target brightness value, the brightness of the entire picture will be more balanced, and there will not be too many overexposed areas or too many underexposed areas.

[0172] The preset brightness difference threshold is used to balance the image stability and brightness adjustment accuracy. If the value is too small, the image will be adjusted frequently, resulting in a poor user experience. If the value is too large, the image will be too dark or too bright. You can adjust the threshold according to actual application requirements.

[0173] Preferably, in this embodiment, the preset brightness difference threshold is set to 10, which ensures both the stability of the picture and the appropriate brightness of the picture.

[0174] The present embodiment provides an exposure method based on face brightness, including: S1, obtaining an initial image through an image sensor; S2, obtaining the position information of a face frame in the initial image; S3, calculating the photometric block category information according to the position relationship between the face frame and the photometric block and the intersection ratio of the face in each photometric block; S4, calculating the average brightness value of each photometric block according to the initial image; S5, calculating the face brightness value according to the photometric block category information and the average brightness value of the photometric block; S6, adjusting the exposure and gain according to the difference between the face brightness value and the preset target brightness value to obtain the final face brightness, and outputting the final image. By considering the position and ratio relationship between the face frame and the photometric block, the present embodiment can more accurately calculate the actual brightness of the face, thereby more accurately adjusting the exposure setting, avoiding overexposure or underexposure, and improving the accuracy of exposure.

[0175] In addition, since the position and size of the face may vary depending on the shooting scene, this embodiment can adjust the exposure according to the specific position and size of the face to better adapt to different shooting environments and light conditions.

[0176] In addition, this embodiment divides the metering blocks into four types and performs background brightness compensation according to the types and quantities thereof, thereby better balancing the brightness difference between the face and the background, while avoiding the background being too bright or too dark, making the overall picture more harmonious.

[0177] Thereby improving the overall image quality.

[0178] Embodiment 2

[0179] Please see attached Figure 5 The embodiment of the present invention discloses an exposure device based on face brightness, comprising the following units:

[0180] An image acquisition unit, used for acquiring an initial image through an image sensor;

[0181] A face frame acquisition unit, used to acquire position information of a face frame in the initial image;

[0182] a light metering block classification unit, configured to calculate light metering block category information according to a positional relationship between a face frame and a light metering block and an intersection ratio of a face in each light metering block; the light metering blocks are multiple; the light metering block category information includes the category of light metering blocks and the number of light metering blocks of each category; the categories include first category light metering blocks, second category light metering blocks, third category light metering blocks and fourth category light metering blocks;

[0183] An average brightness calculation unit, used for calculating the average brightness value of each light metering block according to the initial image;

[0184] A face brightness calculation unit, used to calculate the face brightness value according to the photometric block category information and the average brightness value of the photometric block;

[0185] The face brightness compensation unit is used to adjust the exposure and gain according to the difference between the face brightness value and the preset target brightness value to obtain the final face brightness and output the final image.

[0186] An exposure device based on face brightness provided in an embodiment of the present invention can execute an exposure method based on face brightness provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method. It is worth noting that in the device provided in an embodiment of the present invention, the various units and modules included are only divided according to functional logic, but are not limited to the above divisions, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0187] Embodiment 3

[0188] Please see attached Figure 6 , Figure 6 1 is a schematic diagram of the structure of an exposure device based on face brightness according to an embodiment of the present invention. The device 20 includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, the steps in the above method embodiment are implemented. Alternatively, when the processor 21 executes the computer program, the functions of each module / unit in the above device embodiments are implemented.

[0189] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the exposure device 20 based on face brightness. For example, the computer program may be divided into the modules in the second embodiment. For the specific functions of each module, please refer to the working process of the device described in the above embodiment, which will not be repeated here.

[0190] The exposure device 20 based on face brightness may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will appreciate that the schematic diagram is merely an example of an exposure device 20 based on face brightness, and does not constitute a limitation on an exposure device 20 based on face brightness, and may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, the exposure device 20 based on face brightness may also include input and output devices, network access devices, buses, and the like.

[0191] The processor 21 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 21 is the control center of the exposure device 20 based on face brightness, and uses various interfaces and lines to connect various parts of the exposure device 20 based on face brightness.

[0192] The memory 22 can be used to store the computer program and / or module. The processor 21 realizes various functions of the exposure device 20 based on face brightness by running or executing the computer program and / or module stored in the memory 22 and calling the data stored in the memory 22. The memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0193] Wherein, if the module / unit integrated in the exposure device 20 based on face brightness is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor 21, the steps of the above-mentioned method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0194] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present invention. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between the modules, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement the invention without creative work.

[0195] So far, the technical solutions of the present invention have been described in combination with examples, but it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. An exposure method based on face brightness, characterized in that: Including: S1. Obtain an initial image through an image sensor; S2. Obtain the position information of the face frame in the initial image; S3. Calculate the photometric block category information according to the position relationship between the face frame and the photometric blocks and the intersection ratio of the face in each photometric block; there are multiple photometric blocks; the photometric block category information includes the category of the photometric blocks and the number of photometric blocks of each category; the categories include the first-category photometric blocks, the second-category photometric blocks, the third-category photometric blocks, and the fourth-category photometric blocks; Compare the face ratio with a preset ratio threshold, and divide the photometric blocks into different categories according to different ratio comparison conditions, specifically: When 0 < r ≤ r1 in the photometric block, it is determined that the photometric block is a first-category photometric block; When r1 < r ≤ r2 in the photometric block, it is determined that the photometric block is a second-category photometric block; When r2 < r ≤ 1 in the photometric block, it is determined that the photometric block is a third-category photometric block; When r is 0 in the photometric block and it is not horizontally connected to any second-category photometric blocks or third-category photometric blocks, it is determined that the photometric block is a fourth-category photometric block; The first-category photometric blocks further include: When r = 0 in the photometric block and the horizontally connected photometric block is the second photometric block, it is determined that the photometric block is a first-category photometric block; When r = 0 in the photometric block and the horizontally connected photometric block is the third photometric block, it is determined that the photometric block is a first-category photometric block; Among them, calculate the intersection area between each photometric block and the face frame, and use the ratio of the intersection area to the area of the corresponding photometric block as the face ratio r; the preset ratio threshold includes the first ratio threshold r1 and the second ratio threshold r2; S4. Calculate the average brightness value of each photometric block according to the initial image; S5. Calculate the face brightness value according to the photometric block category information and the average brightness value of the photometric blocks; Compare the number of each category of light metering with the number of preset light metering blocks of each category, and use the preset face brightness adjustment method to calculate the face brightness value , the preset face brightness adjustment method varies according to the number of metering in each category; When the number of the first-category photometric blocks is less than the first-category photometric block number threshold and the numbers of the second-category photometric blocks and the third-category photometric blocks are 0, use the full-screen average brightness value as the face brightness value, and the calculation formula for the full-screen average brightness value is: ； in, is the average brightness value of the light metering blocks in the picture, and n is the number of light metering blocks; When the number of the first-category photometric blocks is greater than or equal to the first-category photometric block number threshold and the numbers of the second-category photometric blocks and the third-category photometric blocks are 0, use the fourth-category photometric block brightness and the basic brightness adjustment step to calculate the face brightness, and the calculation formula is: ； in, is the average brightness value of the fourth type of metering block, is the number of the fourth type of light metering blocks, is the basic brightness adjustment step size, is the first adjustment factor; When the numbers of the first-category photometric blocks and the second-category photometric blocks are both greater than 0 and the number of the third-category photometric blocks is 0, use the first-category photometric blocks as the compensation blocks for the second-category photometric blocks to calculate the face brightness value, and the calculation formula is: ； in, is the number of first category metering blocks, is the brightness value of the first category metering block; is the number of second category metering blocks, is the brightness value of the second category metering block, It is the second regulatory factor; When the number of the third-category photometric blocks is greater than 0, use the third-category photometric block brightness value as the face brightness value, and the calculation formula is: ； in, is the number of third category metering blocks, is the brightness value of the third category metering block; S6. Adjust the exposure and gain according to the difference between the face brightness value and the preset target brightness value to obtain the final face brightness, and output the final image.

2. The method according to claim 1, characterized in that Specifically included in step S3: The calculation formula for the intersection area S is: ； Among them, the upper left corner coordinates of the photometric block are (x1, y1), and the lower right corner coordinates are (x2, y2); the upper left corner coordinates of the face frame are (x3, y3), and the lower right corner coordinates are (x4, y4); min is the minimum value operator, and max is the maximum value operator; Compare the face occupancy ratio r with a preset occupancy ratio threshold, and divide the photometric blocks into different categories according to different occupancy ratio comparison conditions, and count the number of photometric blocks in each category.

3. The method according to claim 1, characterized in that Step S2 is specifically: identify the face frame where the face is located in the initial image through a preset face detection model in the host computer. The host computer is a kind of SOC, the face frame is rectangular, and obtain the upper left corner coordinates and the lower right corner coordinates of the rectangle.

4. An exposure device based on face brightness, characterized in that: It includes the following units: An image acquisition unit, configured to acquire an initial image through an image sensor; A portrait frame acquisition unit, configured to acquire the position information of the face frame in the initial image through the host computer; A photometric block classification unit, configured to calculate photometric block category information according to the positional relationship between the face frame and the photometric blocks and the intersection occupancy ratio of the face in each photometric block; there are multiple photometric blocks; the photometric block category information includes the category of the photometric blocks and the number of photometric blocks in each category; the categories include the first category photometric blocks, the second category photometric blocks, the third category photometric blocks, and the fourth category photometric blocks; Comparing the face occupancy ratio with a preset occupancy ratio threshold and dividing the photometric blocks into different categories according to different occupancy ratio comparison conditions specifically is: When 0 < r ≤ r1 in the photometric block, it is determined that the photometric block is a first category photometric block; When r1 < r ≤ r2 in the photometric block, it is determined that the photometric block is a second category photometric block; When r2 < r ≤ 1 in the photometric block, it is determined that the photometric block is a third category photometric block; When r is 0 in the photometric block and it is not horizontally connected to any second category photometric blocks or third category photometric blocks, it is determined that the photometric block is a fourth category photometric block; The first category photometric blocks further include: When r = 0 in the photometric block and the horizontally connected photometric block is the second photometric block, it is determined that the photometric block is a first category photometric block; When r = 0 in the photometric block and the horizontally connected photometric block is the third photometric block, it is determined that the photometric block is a first category photometric block; Among them, calculate the intersection area of the two according to the position of each photometric block and the face frame, and use the ratio of the intersection area to the area of the corresponding photometric block as the face occupancy ratio r; the preset occupancy ratio threshold includes a first occupancy ratio threshold r1 and a second occupancy ratio threshold r2; An average brightness calculation unit, configured to calculate the average brightness value of each photometric block according to the initial image; A face brightness calculation unit, configured to calculate the face brightness value according to the photometric block category information and the average brightness value of the photometric blocks; Compare the number of each category of light metering with the number of preset light metering blocks of each category, and use the preset face brightness adjustment method to calculate the face brightness value , the preset face brightness adjustment method varies according to the number of metering in each category; When the number of the first category photometric blocks is less than the first category photometric block number threshold and the number of the second category photometric blocks and the third category photometric blocks is 0, use the full-screen average brightness value as the face brightness value. The calculation formula of the full-screen average brightness value is: ； in, is the average brightness value of the light metering blocks in the picture, and n is the number of light metering blocks; When the number of the first category of metering blocks is greater than or equal to the first category metering block number threshold, and the number of the second category of metering blocks and the third category of metering blocks is 0, the brightness of the fourth category of metering blocks and the basic brightness adjustment step are used to calculate the face brightness. The calculation formula is: ； in, is the average brightness value of the fourth type of metering block, is the number of the fourth type of light metering blocks, is the basic brightness adjustment step size, is the first adjustment factor; When the number of the first category of metering blocks and the number of the second category of metering blocks are both greater than 0, and the number of the third category of metering blocks is 0, the first category of metering blocks are used as compensation blocks for the second category of metering blocks to calculate the face brightness value, and the calculation formula is: ； in, is the number of first category metering blocks, is the brightness value of the first category metering block; is the number of second category metering blocks, is the brightness value of the second category metering block, It is the second regulatory factor; When the number of the third-category metering blocks is greater than 0, the brightness value of the third-category metering block is used as the face brightness value. The calculation formula is: ； in, is the number of third category metering blocks, is the brightness value of the third category metering block; The face brightness compensation unit is used to adjust the exposure and gain according to the difference between the face brightness value and the preset target brightness value to obtain the final face brightness and output the final image.

Images