A method and apparatus for video image exposure metric and automatic correction

Abstract

Embodiments of the present application provide a method for video image exposure measurement and automatic correction. The method comprises: obtaining YUV format image data; extracting a gray value based on the image data, the gray value being a Y component; extracting a region of interest based on the image data; evaluating an exposure of the region of interest based on the gray value; and correcting the image data based on the exposure. Different corrections can be made to the image for different illuminations in the region of interest of the video picture, and the corrected image is converted to RGB space for color saturation recovery, so that a user obtains a better visual experience. Furthermore, embodiments of the present application provide a device for video image exposure measurement and automatic correction.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of video communication, and more particularly, to a method and device for video image exposure metric and automatic correction. BACKGROUND

[0002] This section is intended to provide background information to facilitate an understanding of embodiments of the application as set forth in the claims. The description herein does not constitute an admission of prior art.

[0003] With the wide application of Internet video communication technology, especially in remote conferencing, remote teaching and training and other scenarios, sometimes the conference picture is dark. In the condition that the external environment cannot be changed, in order to improve the visual effect of the customer, the picture needs to be enhanced to improve the picture quality.

[0004] The existing video conference dark scene enhancement algorithm system does not extract the main attention area of the user in the picture, does not measure and analyze the exposure of the picture image, and does not perform adaptive correction, but uniformly enhances the same parameter. However, due to the changeable video conference scene, the brightness of the conference picture is not the same. Direct exposure evaluation of the whole image introduces great interference in the background area, resulting in inaccurate evaluation results and inaccurate image correction. Without measuring and analyzing the exposure of the picture image, using the same parameter for enhancement will cause over-enhancement in some scenes and insufficient enhancement in some scenes. When the enhancement amplitude is large, due to the Y component enhancement processing, the picture will have a certain color saturation loss, so the best visual effect cannot be obtained.

[0005] The present application designs an image exposure metric and automatic correction algorithm and system, measures and analyzes the exposure of the video picture image, then selects the best correction parameter based on the analysis result to realize the enhancement of the picture, and restores the color saturation to obtain better visual experience. In order to take into account the different perceptions of users, an interface is designed for fine tuning. SUMMARY

[0006] The present application aims to make different picture enhancement processing for image data of different frames in a video process by adaptively correcting the image data based on the measurement of the exposure of the image data, so as to meet the needs of users. Since the existing video conference dark scene enhancement algorithm system does not extract the main attention area of users in a picture in use, and does not measure and analyze the exposure of the picture image for adaptive correction, without extraction of the region of interest, exposure evaluation is directly performed on the whole picture, the background area will introduce great interference, resulting in inaccurate evaluation results and inaccurate correction of the image; the same parameter enhancement will lead to over-enhancement of some scenes and insufficient enhancement of some scenes. Therefore, in order to solve the problems existing in the prior art, there is a great need for an improved video image exposure measurement and automatic correction technical scheme.

[0007] In the present context, embodiments of the present application aim to provide a method and apparatus for video image exposure measurement and automatic correction.

[0008] In a first aspect of the embodiments of the present application, a method for video image exposure measurement and automatic correction is provided, comprising: acquiring YUV format image data; extracting a gray value, i.e. a Y component, based on the image data; extracting a region of interest based on the image data; evaluating the exposure of the region of interest based on the gray value; and correcting the image data based on the exposure.

[0009] In an embodiment of the present application, the extraction of the gray value based on the image data comprises: extracting the gray value of each pixel based on the image data.

[0010] In another embodiment of the present application, the extraction of the region of interest based on the image data comprises: performing face recognition on the image data; and the recognized face region is the region of interest.

[0011] In yet another embodiment of the present application, the gray value comprises the gray value of each pixel point in the region of interest.

[0012] In still another embodiment of the present application, the evaluation of the exposure of the region of interest based on the gray value comprises: establishing a mapping relationship between the gray value and the exposure; acquiring the exposure of each pixel point in the region of interest; and acquiring the mean value of pixel exposure based on the exposure of each pixel point as the exposure of the region of interest.

[0013] In still another embodiment of the present application, the self-adaptive acquisition of the optimal correction value for different exposure levels comprises: establishing a mapping relationship between the exposure levels and the optimal correction values; and self-adaptively acquiring the optimal correction value based on the mapping relationship.

[0014] In still another embodiment of the present application, the correction of the image data based on the optimal correction value comprises: determining the gray value of each pixel point after correction according to the optimal correction value and the gray value of each pixel point; and correcting the image data based on the gray value of each pixel point after correction.

[0015] In still another embodiment of the present application, the correction of the image data further comprises: color saturation recovery after the conversion of the corrected image data to the RGB space.

[0016] In a second aspect of the embodiments of the present application, a device for video image exposure level measurement and automatic correction is provided, which comprises: an acquisition module for acquiring image data; a gray value extraction module for extracting gray values based on the image data; an interesting region extraction module for extracting an interesting region based on the image data; an evaluation module for evaluating the exposure level of the interesting region based on the gray values; and a correction module for correcting the image data based on the exposure level.

[0017] In an embodiment of the present application, the gray value extraction module comprises: a module for extracting the gray value of each pixel based on the image data.

[0018] In another embodiment of the present application, the interesting region extraction module comprises: a module for performing face recognition on the image data; and a module for taking the recognized face region as the interesting region.

[0019] In still another embodiment of the present application, the gray values comprise the gray values of each pixel point in the interesting region.

[0020] In still another embodiment of the present application, the evaluation module comprises: a module for establishing a mapping relationship between the gray values and the exposure levels; a module for acquiring the exposure level of each pixel point in the interesting region; and a module for acquiring the average value of the pixel exposure levels, i.e. the exposure level of the interesting region, based on the exposure level of each pixel point.

[0021] In still another embodiment of the present application, the module for adaptively obtaining the optimal correction value for different exposure includes a module for establishing a mapping relationship between the exposure and the optimal correction value; and a module for adaptively obtaining the optimal correction value based on the mapping relationship.

[0022] In still another embodiment of the present application, the module for correcting the image data based on the optimal correction value includes a module for determining the corrected gray value of each pixel point according to the optimal correction value and the gray value of each pixel point; and a module for correcting the image data based on the corrected gray value of each pixel point.

[0023] In still another embodiment of the present application, the module for correcting the image data based on the optimal correction value includes a module for determining the corrected gray value of each pixel point according to the optimal correction value and the gray value of each pixel point; and a module for correcting the image data based on the corrected gray value of each pixel point.

[0024] According to the method and device for video image exposure measurement and automatic correction, the exposure of a video image is measured and analyzed, and then based on the analysis result, the optimal correction parameter is selected to enhance the image, and the color saturation is recovered to obtain better visual experience. An interface is designed to make fine adjustment to meet different visual experiences of users. BRIEF DESCRIPTION OF DRAWINGS

[0025] The above and other objects, features and advantages of the exemplary embodiments of the present application will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0026] Figure 1 A flow chart of a method for implementing video image exposure measurement and automatic correction according to an embodiment of the present application is schematically shown;

[0027] Figure 2 A graph of the relationship between the gray value and the exposure according to an embodiment of the present application is schematically shown;

[0028] Figure 3 A graph of the relationship between the optimal correction value and the exposure according to another embodiment of the present application is schematically shown;

[0029] Figure 4 A schematic diagram of a device for implementing video image exposure measurement and automatic correction according to an embodiment of the present application is schematically shown;

[0030] In the drawings, identical or corresponding reference signs indicate identical or corresponding parts. DETAILED DESCRIPTION

[0031] The principles and spirits of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are given only for better understanding the present application and enabling those skilled in the art to better implement the present application, and do not limit the scope of the present application in any way. On the contrary, these embodiments are provided for making the present disclosure more thorough and complete, and enabling those skilled in the art to fully convey the scope of the present disclosure.

[0032] Those skilled in the art know that the embodiments of the present application can be implemented as a method and a device. Therefore, the present disclosure can be embodied in the form of entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0033] According to the embodiments of the present application, a method and a device for video image exposure measurement and automatic correction are provided.

[0034] The principles and spirits of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are given only for better understanding the present application and enabling those skilled in the art to better implement the present application, and do not limit the scope of the present application in any way. On the contrary, these embodiments are provided for making the present disclosure more thorough and complete, and enabling those skilled in the art to fully convey the scope of the present disclosure. SUMMARY

[0036] The present inventors find that the existing technology for enhancing pictures has the following defects: the existing video conference dark scene enhancement algorithm system does not extract the main attention area of the user in the picture in use, does not measure and analyze the exposure of the picture image for adaptive correction, but uniformly enhances the same parameter. However, due to the various video conference scenes, the light and dark degrees of the conference pictures are different, the exposure of the whole image is directly evaluated, the background area introduces great interference, the evaluation result is not accurate, and the image cannot be accurately corrected; without the measurement and analysis of the exposure of the picture image, the same parameter is enhanced, which may cause over-enhancement in some scenes and insufficient enhancement in some scenes. When the enhancement amplitude is large, due to the Y component enhancement processing, the picture may have a certain color saturation loss, and thus the best visual effect cannot be obtained.

[0037] In order to overcome the problems in the prior art, the present application provides a method for video image exposure measurement and automatic correction, comprising: acquiring YUV format image data; extracting a gray value, i.e., a Y component, based on the image data; extracting a region of interest based on the image data; evaluating the exposure of the region of interest based on the gray value; and correcting the image data based on the exposure.

[0038] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be specifically introduced below.

[0039] Overview of Application Scenarios

[0040] The embodiments of the present application can be applied to scenarios such as remote conferencing, remote teaching or training, however, it is completely understood by those skilled in the art that the applicable scenarios of the embodiments of the present application are not limited in any aspect by the framework.

[0041] Exemplary Method

[0042] The method for implementing personalized setting of enterprise cloud conference room according to the exemplary embodiments of the present application will be described below in combination with application scenarios, with reference to Figures 1-4 It should be noted that the above-mentioned application scenarios are only shown for the purpose of facilitating understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in any aspect in this regard. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0043] With reference to Figure 1 , a flowchart of a method for implementing video image exposure metric and automatic correction according to one embodiment of the present application is schematically shown. The method comprises the following steps:

[0044] S100, acquiring image data.

[0045] As an example, the acquiring image data comprises obtaining YUV format image data by intercepting each frame of image in the process of video conference.

[0046] S110, extracting gray value based on the image data.

[0047] As an example, the extracting gray value based on the image data comprises extracting the gray value, i.e. Y component value, of each pixel based on the image data.

[0048] S120, extracting region of interest based on the image data.

[0049] As an example, the extracting region of interest based on the image data comprises performing face recognition on the image data; the recognized face region is the region of interest. Specifically, in the video conference, face recognition is performed on the intercepted image data to determine the final face region, and the face region is taken as the region of interest. Since the light and shade of the face is an important factor affecting the visual experience of the user in the process of video conference, face recognition is performed, and exposure metric is performed on the region of interest which is the face region, so that the visual needs of the user can be greatly met.

[0050] In another embodiment, in the video conference, the region of interest is mainly concentrated in the lower middle 1 / 2 region of the picture, and this region can be taken as the region of interest, and exposure metric analysis is performed on this fixed region, so that the calculation amount is small and the algorithm complexity is low.

[0051] S130, evaluate the exposure of the region of interest based on the gray value.

[0052] As an example, the exposure of the region of interest based on the gray value includes: establishing a mapping relationship between the gray value and the exposure; obtaining the exposure of each pixel point of the region of interest; obtaining the mean value of the exposure of each pixel point as the exposure of the region of interest based on the exposure of each pixel point;

[0053] More specifically, since the region of interest in the video conference scene is mainly concentrated in the portrait area, the brightness of the face is the main influencing factor of the user's visual experience, and therefore the content of the image can also be considered to be relatively single, and the exposure of each pixel can be measured based on the gray value (Y) of the image. Through experimental analysis and research, we construct an exposure model to establish a mapping relationship between the gray value of the pixel and the exposure, and the function model is:

[0054]

[0055] In the formula, Y represents the gray value of any pixel, and the value range is [0, 255], and y represents the exposure level. Through experimental test analysis, the best model parameters are given: a = -1.5, the value range of a is (-2.01, -0.99); b = -0.9, the value range of b is (-0.61, -1.19); c = 2.5, the value range of c is (2.01, 2.89). As shown in Figure 2 , a curve graph showing the change between the exposure and the gray value is shown. For example, when the gray value is 100.9, the exposure is 0.4; when the gray value is 191.2, the exposure is -0.4.

[0056] The inverse function of the exposure generation model is obtained, that is, the exposure corresponding to the pixel value Y is obtained, and the exposure of the region of interest is the mean value of the exposure of each pixel:

[0057]

[0058] In the formula, N represents the total pixel amount of the region of interest, and y i represents the exposure evaluated by the i-th pixel. Y i represents the gray value of the i-th pixel

[0059] That is, the exposure of the i-th pixel evaluated in formula (2) can be obtained by solving the inverse function of formula (1). The mean value of the exposure values of all pixels in the region of interest is obtained, that is, the exposure y of the region of interest is evaluated. Through test analysis, the exposure y is in the interval [-0.4, 0.4], the image exposure is appropriate; y < -0.4, the image exposure is insufficient, the image is dark; y > 0.4, the image exposure is excessive, the image is too bright.

[0060] S140, Correct the image data.

[0061] As an example, the correction of the image data includes: adaptively obtaining an optimal correction value for different exposures; and correcting the image data based on the optimal correction value. Specifically, a mapping relationship is established between the exposure and the optimal correction value; the optimal correction value is adaptively obtained based on the mapping relationship; the corrected grayscale value of each pixel is determined based on the optimal correction value and the grayscale value of each pixel; and the image data is corrected based on the grayscale value of each corrected pixel.

[0062] For a specific example, the formula for adaptive image correction is:

[0063]

[0064] Where Y′ represents the grayscale value of each pixel in the corrected image, and Y represents the grayscale value in the image before correction. Let be the optimal correction value adaptively obtained based on the evaluated exposure y, which is a function of the exposure y.

[0065] More specifically, the general idea of ​​this invention is to establish a functional relationship to fit this desired curve by pre-determining the expected curve between the correction value γ and the exposure y—that is, for images with different exposures, when the exposure is appropriate (i.e., the exposure y is in the range of [-0.4, 0.4]), the image does not need adjustment, and the γ correction value should approach 1; when the exposure is insufficient (i.e., the exposure y < -0.4), the image needs to be brightened, and the γ correction value should be greater than 1; when the exposure is excessive (i.e., the exposure y > 0.4), the image needs to be darkened, and the γ correction value should be less than 1. Figure 3 The figure shows the curve relating exposure y and the optimal correction value γ. That is, by evaluating the exposure of each frame of video image, the optimal γ correction value needs to be adaptively obtained based on the evaluated exposure.

[0066] Since the desired curve is between γ and y, and as shown in formula (1), the functional relationship between Y and y has been determined, it is only necessary to construct the mapping relationship between γ and Y to obtain the mapping between γ and y. Using the conformal mapping relationship, the mapping relationship between γ and the gray value Y is constructed as follows:

[0067]

[0068] In the formula, The function is as follows:

[0069]

[0070] wherein: β = arctan(-2b / 255). In the formula, the values of the parameters are a = 0.2, b = 0.3, c = 0.3, p = 0.05, and k = -1.113.

[0071] In combination Figures 2-3 It can be seen that for the region with normal exposure, the value of γ tends to 1, and the correction amplitude of the image is small; when the exposure is not suitable, the appropriate correction value of γ can be obtained according to the exposure, and the image can be corrected by a large amplitude, which meets the expectation of the correction of images with different exposures.

[0072] It can be seen that the optimal correction value γ can be finally determined by the above formulas (1)-(5), and the image data can be appropriately enhanced based on the value of γ.

[0073] Another example, for different exposures y, the optimal correction value of γ can be adaptively obtained by the following piecewise function.

[0074]

[0075] In the formula, when the exposure y≤y2, the image is seriously underexposed, y2 takes the value [-1.31, -1.36], γ1 is the maximum correction value, takes the value [1.70, 1.75], and γ2 takes the value [1.8, 2.0]; when y2<y≤y3, the image is underexposed, y3 = -0.4; when y3<y≤y4, the image is normally exposed, the correction value of γ is 1, and y4 = 0.4; when y4<y≤y5, the image is overexposed, y5 takes the value [1.18, 1.25], and γ5 takes the value [0.7, 0.75]; when y>y5, the image is seriously overexposed, and γ6 takes the value [0.4, 0.6].

[0076] Since the optimal correction value γ is adaptively changed based on different exposures y, that is, in formula (3), in the video conference, the exposure y of the image data and the correction value γ can be different under different illumination environments, so in the video conference under different illumination environments, the image data of different frames is adaptively enhanced or weakened according to the exposure of each frame of image data, and each frame of image data in the video under different illumination environments is not corrected by using the same fixed correction value, which can make the correction of the image data more accurate. The present application evaluates the exposure in the region of interest, and then calculates the optimal correction value and corrects the image data, instead of using a fixed correction value to uniformly correct all images, which avoids the phenomenon of over-enhancement of images with sufficient illumination or insufficient enhancement of images with dark illumination.

[0077] The color saturation recovery is further included after the enhancement processing of the image data; for the correction of the color image, if the RGB channels are processed respectively, color distortion will be caused; the application is based on the YUV color space, only the luminance channel Y is processed, color distortion is avoided, but saturation loss phenomenon is caused to some extent, so that the image looks color deficient.

[0078] Specifically, in order to make the user obtain better visual experience, the RGB is respectively processed with similar amplitude gain according to the gain value of the Y component, color recovery can be realized, and the correction formula is as follows:

[0079]

[0080] In the formula, R'G'B' respectively represent the values after color saturation recovery, R, G and B represent the values before color saturation recovery (namely after the image data enhancement), Y represents the gray value of the original image data, and Y' represents the gray value after the enhancement processing of the image data.

[0081] The application realizes the enhancement of the picture by measuring and analyzing the exposure of the video picture image, and then selects the best correction parameter based on the analysis result, recovers the color saturation to obtain better visual experience. In order to consider different perceptions of the user, an interface is designed for fine tuning.

[0082] Exemplary Device

[0083] After the method of the exemplary embodiment of the application is introduced, next, reference is made to Figure 4 The device for realizing the video image exposure measurement and automatic correction of the exemplary embodiment of the application is shown in the schematic diagram, and the device includes the following modules:

[0084] The acquisition module 400 is used for acquiring image data.

[0085] As an example, the acquisition of the image data includes: obtaining YUV format image data by intercepting each frame of image in the process of the video conference.

[0086] The extraction of the gray value 410 is used for extracting a gray value based on the image data.

[0087] As an example, the extraction of the gray value based on the image data includes: extracting the gray value of each pixel, namely the Y component data, based on the image data.

[0088] The extraction of the region of interest 420 is used for extracting a region of interest based on the image data.

[0089] As an example, the extracting the region of interest based on the image data comprises: performing face recognition on the image data; and a recognized face region is the region of interest. Specifically, in a video conference, a face region is determined by performing face recognition on the captured image data, and the face region is taken as the region of interest. Since the brightness and darkness of a face is an important factor affecting the visual experience of a user in a video conference, performing face recognition and performing exposure measurement on the region of interest of the face region can greatly meet the visual needs of the user.

[0090] In another embodiment, in a video conference, the region of interest is mainly concentrated in the lower middle 1 / 2 region of the picture, and the region of interest is taken as a fixed region for exposure measurement analysis, so that the calculation amount is small and the algorithm complexity is low.

[0091] The evaluation module 430 is configured to evaluate the exposure of the region of interest based on the gray value.

[0092] As an example, the evaluating the exposure of the region of interest based on the gray value comprises: establishing a mapping relationship between the gray value and the exposure; obtaining the exposure of each pixel point of the region of interest; and obtaining the mean value of the exposure of each pixel point, i.e., the exposure of the region of interest, based on the exposure of each pixel point.

[0093] More specifically, in a video conference scenario, the region of interest is mainly concentrated in a portrait region, and the brightness and darkness of a face is an important factor affecting the visual experience of a user, so the content of the image can also be considered to be relatively single, and the exposure of each pixel can be measured based on the gray value (Y) of the image. Through experimental analysis and research, we construct an exposure model to establish a mapping relationship between the gray value of a pixel and the exposure, and the function model is:

[0094]

[0095] In the formula, Y represents the gray value of an arbitrary pixel, and the value range is [0, 255], and y represents the exposure level. Through experimental test analysis, the best model parameters are given: a = -1.5, the value range of a is (-2.01, -0.99); b = -0.9, the value range of b is (-0.61, -1.19); and c = 2.5, the value range of c is (2.01, 2.89). As shown in FIG. 2, a curve graph showing the change between the exposure and the gray value is shown. For example, when the gray value is 100.9, the exposure is 0.4; and when the gray value is 191.2, the exposure is -0.4. Figure 2

[0096] ​The inverse function of the exposure generation model is calculated, and the exposure corresponding to the pixel value Y is obtained. The exposure of the region of interest is the average of the exposure of each pixel:

[0097]

[0098] In the formula, N represents the total pixel amount of the region of interest, y i represents the exposure evaluated by the i-th pixel. Y i represents the gray value of the i-th pixel

[0099] That is, the exposure evaluated by the i-th pixel in formula (2) is obtained by calculating the inverse function of formula (1). The exposure of all pixels in the region of interest is calculated as the average exposure of the region of interest. Through test analysis, the exposure y is in the interval [-0.4, 0.4], the image exposure is appropriate; y<-0.4, the image exposure is insufficient, the image is dark; y>0.4, the image exposure is excessive, the image is too bright.

[0100] The correction module 440 is configured to correct the image data.

[0101] As an example, the correction of the image data includes: adaptively obtaining the optimal correction value for different exposures; and correcting the image data based on the optimal correction value. The mapping relationship between the exposure and the optimal correction value is established; the optimal correction value is adaptively obtained based on the mapping relationship; the gray value of each pixel point after correction is determined according to the optimal correction value and the gray value of each pixel point; and the image data is corrected based on the gray value of each pixel point after correction.

[0102] In a specific example, the adaptive correction formula of the image is:

[0103]

[0104] In the formula, Y' is the total gray value of each pixel in the region of interest after correction, and Y is the gray value in the image before correction. The optimal correction value adaptively obtained based on the evaluated exposure y is a function of the exposure y.

[0105] More specifically, the general idea of the application is to determine the expected curve between the correction value γ and the exposure y in advance, that is, for different exposure images, when the exposure is appropriate, that is, the exposure y is in the interval [-0.4, 0.4], the image does not need to be adjusted, and the γ correction value should tend to 1; when the exposure is insufficient, that is, the exposure y<-0.4, the image needs to be brightened, and the γ correction value should be greater than 1; when the exposure is excessive, that is, the exposure y>0.4, the image is darkened, and the γ correction value should be less than 1, and the expected curve is fitted by establishing a function relationship, as follows:Figure 3 As shown, the curve is the change relationship between exposure y and the best correction value γ. That is, by exposure evaluation of each frame of video image, the best γ correction value needs to be adaptively obtained based on the evaluated exposure,

[0106] Since the determined expected curve is between γ and y, and the function relationship between Y and y has been determined by formula (1), only the mapping relationship between γ and Y needs to be constructed to obtain the mapping between γ and y. The mapping relationship between γ and the gray value Y is constructed by using the conformal mapping relationship:

[0107]

[0108] In the formula, The function is as follows:

[0109]

[0110] Where: β = arctan (-2b / 255). In the formula, the values of the parameters are a = 0.2, b = 0.3, c = 0.3, p = 0.05, and k = -1.113.

[0111] In combination with Figures 2-3 It can be seen that for the area with normal exposure, the γ value tends to 1, and the correction amplitude of the image is small; when the exposure is not suitable, the appropriate correction γ value can be obtained according to the exposure, and the image can be corrected by a large amplitude, which meets the expectation of correcting images with different exposures.

[0112] It can be seen that the best correction value γ can be finally determined by the above formulas (1)-(5), and the image data can be appropriately enhanced based on the γ value.

[0113] Another example, for different exposure y, the following segmented function can be used to adaptively obtain the best γ correction value.

[0114]

[0115] In the formula, when the exposure y≤y2, the image is severely underexposed, y2 takes the value [-1.31, -1.36], γ1 is the maximum correction value, takes the value [1.70, 1.75], and γ2 takes the value [1.8, 2.0]; when y2<y≤y3, the image is underexposed, y3 = -0.4; when y3<y≤y4, the image is normally exposed, the γ correction value is 1, and y4 = 0.4; when y4<y≤y5, the image is overexposed, y5 takes the value [1.18, 1.25], and γ5 takes the value [0.7, 0.75]; when y>y5, the image is severely overexposed, and γ6 takes the value [0.4, 0.6].

[0116] Since the optimal correction value gamma is adaptively changed based on the different exposure y, that is, in the video conference, the exposure y of the image data and the correction value gamma can be different under different illumination environments, so that in the video conference under different illumination environments, the image data of different frames is adaptively enhanced or weakened according to the exposure of each frame of image data, and each frame of image data in the video under different illumination environments is not corrected by using the same fixed correction value, so that the correction of the image data is more accurate. The present application evaluates the exposure in the region of interest, and then calculates the optimal correction value and corrects the image data, instead of using a fixed correction value to uniformly correct all images, thereby avoiding the phenomenon of over-enhancement of images with sufficient illumination or insufficient enhancement of images with dark illumination.

[0117] After the image data is enhanced, color saturation recovery is further included: for the correction of color images, if the RGB channels are processed respectively, color distortion will be caused; the present application is based on YUV color space, and only the luminance channel Y is processed, thereby avoiding color distortion, but causing a certain degree of saturation loss phenomenon, so that the image looks colorless.

[0118] Specifically, in order to enable the user to obtain better visual experience, the RGB is respectively subjected to gain of similar amplitude according to the gain value of Y component, so that the color recovery can be realized, and the correction formula is as follows:

[0119]

[0120] In the formula, R'G'B' respectively represent the values after color saturation recovery, R, G and B represent the values before color saturation recovery (i.e. after image data enhancement), Y represents the gray value of original image data, and Y' represents the gray value after image data enhancement.

[0121] The present application measures and analyzes the exposure of the video picture image, then selects the optimal correction parameter based on the analysis result to realize the enhancement of the picture, and restores the color saturation to obtain better visual experience. In order to take into account the different perceptions of users, an interface is designed for fine tuning.

[0122] It should be noted that although several units / modules or sub-units / modules of the image exposure measurement and automatic correction device are mentioned in the foregoing detailed description, such division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into a plurality of units / modules.

[0123] Further, although operations of the inventive method are described in a particular order in the drawings, this is not required or implied in any way as to order of performance or as to the necessity of all illustrated operations in order to achieve desirable results. Additionally or alternatively, certain steps can be omitted, combined into a single step, and / or separated into multiple steps.

[0124] While the principles and spirit of the application have been described with reference to several specific embodiments, it is to be understood that the application is not limited to the specific embodiments disclosed and that the division of aspects is not meant to imply that features from these aspects cannot be combined to advantage. The application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method for measuring and automatically correcting exposure of video images, characterized in that, The method includes: Acquire YUV format image data; Gray values ​​are extracted from the image data, and the gray values ​​are the Y component. Extract the region of interest based on the image data; After extracting the region of interest (ROI), the exposure of the ROI is evaluated based on the grayscale values. This includes calculating the exposure of each pixel based on its grayscale value within the ROI, and averaging the exposures of all pixels within the ROI to obtain the average exposure, which is the overall exposure of the ROI. The formula for calculating the exposure of each pixel based on the grayscale value of each pixel in the region of interest is as follows: y i Let represent the exposure of the i-th pixel, Yi represent the grayscale value of the i-th pixel, and a, b, and c are constants; The optimal correction value corresponding to the exposure of the region of interest is obtained based on the mapping relationship. The gray value of each pixel after correction is determined based on the optimal correction value and the gray value of each pixel. The image data is corrected based on the gray value of each pixel. The corrected image data is converted from YUV format to RGB format, and a color restoration operation based on a saturation restoration algorithm is performed.

2. The method for measuring and automatically correcting video image exposure according to claim 1, characterized in that, The extraction of the region of interest based on the image data includes: Perform facial recognition on the image data; The identified facial region is the region of interest.

3. The method for measuring and automatically correcting video image exposure according to claim 2, characterized in that, The grayscale value includes the grayscale value of each pixel within the region of interest.

4. The method for measuring and automatically correcting video image exposure according to claim 1 or 3, characterized in that, The step of evaluating the exposure of the region of interest based on the grayscale value includes: Establish a mapping relationship between the grayscale value and the exposure level; Obtain the exposure of each pixel in the region of interest; The average exposure value of each pixel is obtained based on the exposure value of each pixel, and the average exposure value is the exposure value of the region of interest.

5. The method for measuring and automatically correcting video image exposure according to claim 4, characterized in that, The mapping relationship between the grayscale value and the exposure level includes: establishing the mapping relationship based on the following function model: In the formula, Represents the grayscale value of any pixel. y This indicates the exposure level, where a, b, and c are constants.

6. The method for measuring and automatically correcting video image exposure according to claim 1, characterized in that, The process of obtaining the optimal correction value corresponding to the average exposure based on the mapping relationship includes: Establish a mapping relationship between the exposure and the optimal correction value; The optimal correction value is adaptively obtained based on the mapping relationship.

7. The method for measuring and automatically correcting video image exposure according to claim 3 or 5, characterized in that, The process of correcting the image data based on the optimal correction value includes: The corrected grayscale value of each pixel is determined based on the optimal correction value and the grayscale value of each pixel, wherein the correction formula is: in, The grayscale values ​​of each pixel in the corrected image. The grayscale values ​​in the image before correction. For exposure based on assessment y The optimal correction value obtained adaptively is the exposure. y The function; The image data is corrected based on the grayscale value of each pixel after correction.

8. A device for measuring and automatically correcting video image exposure, characterized in that, The device includes: The acquisition module is used to acquire image data; A grayscale value extraction module is used to extract grayscale values ​​based on the image data; The region of interest extraction module is used to extract the region of interest based on the image data; The evaluation module is used to evaluate the exposure of the region of interest based on the grayscale values. This includes calculating the exposure of each pixel based on its grayscale value within the region of interest, and averaging the exposure of all pixels within the region of interest to obtain the average exposure, which is the exposure of the region of interest. The formula for calculating the exposure of each pixel based on the grayscale value of each pixel in the region of interest is as follows: y i Let represent the exposure of the i-th pixel, Yi represent the grayscale value of the i-th pixel, and a, b, and c are constants; The correction module is used to obtain the optimal correction value corresponding to the average exposure based on the mapping relationship, determine the corrected gray value of each pixel based on the optimal correction value and the gray value of each pixel, and correct the image data based on the corrected gray value of each pixel. A module for converting the corrected image data from YUV format to RGB format and performing color restoration operations based on a saturation restoration algorithm.

9. The device for measuring and automatically correcting video image exposure according to claim 8, characterized in that, The module for extracting the region of interest includes: A module for performing face recognition on the image data; The module used to identify the facial region as the region of interest.

10. The device for measuring and automatically correcting video image exposure according to claim 9, characterized in that, The grayscale value includes the grayscale value of each pixel within the region of interest.

11. The apparatus for measuring and automatically correcting video image exposure according to claim 8 or 10, characterized in that, The evaluation module includes: A module used to establish the mapping relationship between the grayscale value and the exposure. A module for obtaining the exposure of each pixel in the region of interest; A module for obtaining the average pixel exposure based on the exposure of each pixel, i.e., the exposure of the region of interest.

12. The device for measuring and automatically correcting video image exposure according to claim 11, characterized in that, The module for establishing the mapping relationship between the grayscale value and the exposure includes: The mapping relationship is established based on the following function model: In the formula, Represents the grayscale value of any pixel. y This indicates the exposure level, where a, b, and c are constants.

13. The device for measuring and automatically correcting video image exposure according to claim 8, characterized in that, The correction module includes: A module used to establish the mapping relationship between the exposure and the optimal correction value; Module for adaptively obtaining the optimal correction value based on the mapping relationship.

14. The device for measuring and automatically correcting video image exposure according to claim 10 or 12, characterized in that, The module for correcting the image data based on the optimal correction value includes: A module for determining the corrected gray value of each pixel based on the optimal correction value and the gray value of each pixel, wherein the correction formula is: in, The grayscale values ​​of each pixel in the corrected image. The grayscale values ​​in the image before correction. For exposure based on assessment y The optimal correction value obtained adaptively is the exposure. y The function; A module for correcting the image data based on the grayscale value of each corrected pixel.

Images