Exposure parameter adjustment method, device and storage medium

Abstract

The present application provides a method, device, and storage medium for adjusting exposure parameters. The method includes obtaining a face frame of a first frame image and a second frame image; the face frame of the first frame image is used to indicate the facial area in the first frame image; the second frame image is the frame image subsequent to the first frame image; correcting the face frame of the first frame image to obtain a corrected face frame; cropping the second frame image based on the corrected face frame to obtain a facial image of the second frame image; and adjusting exposure parameters based on the facial image of the second frame image. This solution first corrects the face frame of the previous frame image and then crops the facial image with the corrected face frame. This ensures that the cropped facial image and the actual facial area in the cropped image are consistent, helping to obtain more accurate facial brightness based on the facial image and more accurate exposure parameters based on facial brightness adjustment.

Description

[0001] This application is a divisional application of the Chinese patent application submitted to the China Patent Office on August 9, 2021, with application number 202110909669.2 and invention name "Method, device and storage medium for adjusting exposure parameters". Technical Field

[0002] The present application relates to the field of camera technology, and in particular to a method, device, and storage medium for adjusting exposure parameters. Background Art

[0003] To capture images that meet brightness requirements, electronic devices often need to adjust the camera's exposure parameters when taking photos. Exposure parameters can include exposure time and gain. For example, when capturing a face, the exposure parameters need to be adjusted to obtain an image with the facial area within a specific brightness range.

[0004] Currently, one method for adjusting exposure parameters when capturing facial images is to use a face detection algorithm to detect the face frame of the previous frame image, then use the face frame of the previous frame image to crop the face image of the current frame image, adjust the exposure parameters of the next frame based on the face image of the current frame image, and capture the next frame image using the exposure parameters of the next frame, and so on, until an image with the brightness of the facial area within a specific brightness range is obtained.

[0005] When the position of the face in the subsequent frame image moves relative to the position of the face in the previous frame image, the facial image of the current frame image cropped in this way often has a large deviation from the actual facial image in the current frame image. Therefore, the accuracy of the exposure parameters adjusted according to the above method is poor. Summary of the Invention

[0006] The present application provides a method, device, and storage medium for adjusting exposure parameters, aiming to obtain more accurate exposure parameters.

[0007] In order to achieve the above objectives, this application provides the following technical solutions:

[0008] A first aspect of the present application provides a method for adjusting exposure parameters, comprising:

[0009] Obtaining a face frame of a first frame image and a second frame image; wherein the face frame of the first frame image is used to indicate a facial area in the first frame image; and the second frame image is a frame image subsequent to the first frame image;

[0010] Correcting the face frame of the first frame image to obtain a corrected face frame;

[0011] Cropping the second frame of image based on the corrected face frame to obtain a face image of the second frame of image;

[0012] The exposure parameters are adjusted according to the facial image of the second frame image.

[0013] As can be seen from the above content, this solution first corrects the face frame of the previous frame image, and then uses the corrected face frame to crop the facial image. This makes the cropped facial image and the actual facial area in the cropped image helpful to obtain more accurate facial brightness based on the facial image, and obtain more accurate exposure parameters based on the facial brightness adjustment.

[0014] In some optional embodiments, the proportion of the background portion in the facial image obtained by cropping with the corrected face frame is smaller than the proportion of the background portion in the facial image obtained by directly cropping with the face frame of the first frame image.

[0015] In some optional embodiments, correcting the face frame of the first frame image may include reducing the size of the face frame of the first frame image to obtain a corrected face frame. It can be seen that the size of the corrected face frame obtained in this way is smaller than the size of the face frame of the first frame image.

[0016] As can be seen from the above, reducing the size of the face frame helps reduce the proportion of background pixels in the cropped face image, thereby reducing the interference of background pixels on the calculated face brightness. This ensures that the face brightness of the current frame image is accurately obtained when subsequently adjusting the exposure parameters, thereby obtaining more accurate exposure parameter adjustment values.

[0017] In some optional embodiments, when reducing the size of the face frame of the first frame image, it can be reduced according to a preset proportional coefficient, so that the ratio of the size of the corrected face frame to the size of the face frame of the first frame image is the proportional coefficient.

[0018] In some optional embodiments, when the size of the face frame of the first frame image is reduced, the position of the center point of the face frame of the first frame image can be kept fixed, so that the position of the center point of the face frame after correction is the same as the position of the center point of the face frame of the first frame image.

[0019] In some optional embodiments, the scaling factor may be determined using a plurality of frames of images captured in advance.

[0020] In some optional embodiments, the multiple image frames used to determine the scaling factor may be consecutive image frames captured before the second image frame. For example, if the second image frame is the N+1th image frame captured, the multiple image frames used to determine the scaling factor may be the N-1th and Nth image frames captured before the second image frame.

[0021] As can be seen from the above, before correcting the face frame of each image frame, a scaling factor can be determined using the most recently captured frames, enabling dynamic updating of the scaling factor. This approach has the advantage of obtaining a scaling factor that matches the degree of change in the facial area between two adjacent frames, such as the first and second frames mentioned above, each time the face frame is corrected. This preserves as many facial pixels as possible, helping to improve the accuracy of the calculated facial brightness.

[0022] In some optional embodiments, before determining the scaling factor using the consecutive frames of images captured before the second frame of image, it is first determined whether the magnitude of the posture change of the first frame of image relative to each of the consecutive frames of image (except the first frame of image) is less than or equal to a preset magnitude of change threshold;

[0023] When the magnitude of the posture change of the first frame image relative to each of the preceding consecutive frames (except the first frame image) is less than or equal to a preset magnitude of change threshold, the scaling coefficient is determined using the consecutive frames imaged before the second frame image;

[0024] The amplitude of the posture change of the first frame image relative to each frame image in the consecutive frames of images is calculated based on the posture data of the camera device when shooting the first frame image and the posture data of the camera device when shooting one frame image in the previous N frames of images.

[0025] As can be seen above, the dynamic update of the scale factor is only performed when the camera's posture has not changed much over the previous few frames. This has the advantage of avoiding using the face frame of two frames with significant changes in the electronic device's posture to determine the scale factor, thus preventing the scale factor from being too small during the dynamic update process.

[0026] In some optional embodiments, when determining the scaling coefficient using multiple frames of pre-taken images, face frames of these images may be specifically used, where the face frame of each frame of image is used to indicate the area of ​​the face in the corresponding image.

[0027] In some optional embodiments, a method for determining a scaling factor using face frames of multiple image frames may include overlapping the face frames of the multiple image frames to obtain an overlapping area of ​​the face frames of the multiple image frames, and then determining the scaling factor based on the overlapping area and the face frame of any one of the multiple image frames.

[0028] In some optional embodiments, a scaling factor is determined based on the overlapping region and the face frame of any one of the multiple image frames. Specifically, the size of the overlapping region and the face frame of any one of the multiple image frames are determined, and the size of the overlapping region is divided by the size of the face frame of any one of the multiple image frames. The resulting ratio is determined as the scaling factor.

[0029] In some optional embodiments, the face frame of the first frame image may be corrected by using the face frames of the two frames before the second frame image. That is, the face frame may be corrected by using the face frame of the first frame image and the face frame of the frame before the first frame image.

[0030] In some optional embodiments, the amplitude of the posture change of the first frame image relative to the previous frame image is less than a preset change amplitude threshold; the amplitude of the posture change of the first frame image relative to the previous frame image is calculated based on the posture data of the camera device when taking the first frame image and the posture data of the camera device when taking the previous frame image.

[0031] As can be seen from the above, the face frame of the second frame is corrected using the face frames of the two frames preceding the second frame only when the posture change between the two frames is small. The advantage of this is that it avoids using the face frames of two frames of images in which the posture change of the electronic device is too large to determine the corrected face frame, thereby preventing the obtained corrected face frame from being too small.

[0032] In some optional embodiments, the face frame is corrected using the face frame of the first frame image and the face frame of the image preceding the first frame image. Specifically, the face frame of the first frame image and the face frame of the image preceding the first frame image are overlapped, and the overlapping area between the two is determined as the corrected face frame.

[0033] As can be seen from the above, reducing the size of the face frame helps reduce the proportion of background pixels in the cropped facial image, thereby reducing the interference of background pixels on the calculated facial brightness. This ensures that the facial brightness of the current frame image is accurately obtained when subsequently adjusting the exposure parameters, thereby obtaining more accurate exposure parameter adjustment values. In some optional embodiments, the face frame of the first frame image is corrected by translating the face frame of the first frame image according to a pre-determined motion vector. The translated face frame is the corrected face frame. The deviation between the position of the corrected face frame obtained in this manner and the position of the face frame of the first frame image matches the aforementioned motion vector.

[0034] As can be seen from the above, on the one hand, translating the face frame based on the motion vector can be applied to face frames of any shape, including rectangles, making this method of face frame correction more widely applicable. On the other hand, correcting the face frame based on the motion vector can obtain a corrected face frame that matches the position of the face in the second frame image. This reduces the proportion of the background in the cropped face image without shrinking the face frame. This not only produces a larger face image but also prevents the face image from including background pixels. This makes the facial brightness calculated from the facial image more accurate, thereby achieving more accurate exposure parameters when adjusting exposure parameters based on facial brightness.

[0035] In some optional embodiments, before translating the face frame of the first frame image, the motion vector may be determined using two frames of image preceding the second frame image, that is, using the first frame image and the frame image preceding the first frame image.

[0036] In some optional embodiments, before using the first frame image and the frame image before the first frame image to determine the motion vector, it is first determined whether the posture change amplitude of the first frame image relative to the previous frame image is less than a preset change amplitude threshold. After determining that the posture change amplitude of the first frame image relative to the previous frame image is less than the preset change amplitude threshold, the first frame image and the frame image before the first frame image are used to determine the motion vector.

[0037] The amplitude of the posture change of the first frame image relative to the previous frame image is calculated based on the posture data of the camera device when shooting the first frame image and the posture data of the camera device when shooting the previous frame image.

[0038] The advantage of adding the above-mentioned determination step is that it can avoid calculating a motion vector with a large deviation, thereby preventing a large deviation from occurring between the corrected face frame and the actual face in the cropped second frame image.

[0039] In some optional embodiments, the method for determining a motion vector using a first frame image and a frame image preceding the first frame image may specifically be: dividing the frame image preceding the first frame image into multiple image blocks, and selecting an image block of the frame image preceding the first frame image as a query block; dividing the first frame image into multiple image blocks, and then selecting an image block that matches the query block from the multiple image blocks of the first frame image as a target block, and finally determining a motion vector based on the query block and the target block.

[0040] In some optional embodiments, the method for determining the motion vector based on the query block and the target block may be to determine the center point of the query block as the starting point of the motion vector and the center point of the target block as the end point of the motion vector, thereby obtaining a motion vector.

[0041] In some optional embodiments, when selecting a query block, an image block located in the center of a face frame in an image frame preceding the first image frame may be selected as the query block. Among the multiple image blocks obtained by segmentation, the image block located in the center of the face frame in the previous image frame is selected; the face frame in the previous image frame is used to indicate the facial region in the image frame preceding the first image frame.

[0042] In some optional embodiments, when selecting a target block, an image block whose deviation value from the query block is less than a preset deviation threshold from multiple image blocks of the first frame image can be selected as the target block; the deviation value between the image block and the query block represents the degree of difference between the image block and the query block.

[0043] In some optional embodiments, the deviation value between the two image blocks may specifically be the mean absolute difference or the mean square error between the two image blocks.

[0044] In some optional embodiments, adjusting the exposure parameters according to the facial image of the second frame may specifically include:

[0045] First, determining the facial brightness of the second frame image according to the facial image of the second frame image;

[0046] The exposure parameters are then adjusted based on the facial brightness of the second frame image.

[0047] In some optional embodiments, adjusting the exposure parameters based on the facial brightness of the second frame image may specifically include:

[0048] If the difference between the facial brightness of the second frame image and the preset standard brightness is outside a preset range, the exposure parameter is adjusted based on the difference between the facial brightness of the second frame image and the standard brightness.

[0049] In some optional embodiments, after adjusting the exposure parameters according to the facial image of the second frame, the method further includes:

[0050] Performing face detection on the second frame image to obtain a face frame of the second frame image; the face frame of the second frame image is used to indicate a face area in the second frame image.

[0051] In some optional embodiments, during the period between capturing the first frame image and capturing the second frame image, the face of the captured user moves, that is, the facial area in the first frame image is different from the facial area in the second frame image.

[0052] A second aspect of the present application provides an electronic device, the electronic device comprising: one or more processors, a memory, and a camera;

[0053] The memory is used to store one or more programs;

[0054] One or more processors are used to execute one or more programs so that the electronic device executes the exposure parameter adjustment method provided in any one of the first aspects of the present application.

[0055] A third aspect of the present application provides a computer storage medium for storing a computer program. When the computer program is executed, it is specifically used to implement the exposure parameter adjustment method provided in any one of the first aspects of the present application.

[0056] The present application provides a method, device, and storage medium for adjusting exposure parameters. The method includes obtaining a face frame of a first frame image and a second frame image; the face frame of the first frame image is used to indicate the facial area in the first frame image; the second frame image is the frame image subsequent to the first frame image; correcting the face frame of the first frame image to obtain a corrected face frame; cropping the second frame image based on the corrected face frame to obtain a facial image of the second frame image; and adjusting exposure parameters based on the facial image of the second frame image. This solution first corrects the face frame of the previous frame image and then crops the facial image with the corrected face frame. This ensures that the cropped facial image and the actual facial area in the cropped image are consistent, helping to obtain more accurate facial brightness based on the facial image and more accurate exposure parameters based on facial brightness adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1a A schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application;

[0058] Figure 1b A schematic diagram of a scenario in which a user browses a web page using an electronic device, as disclosed in an embodiment of the present application;

[0059] Figure 2a A schematic diagram of the software architecture of an electronic device disclosed in an embodiment of the present application;

[0060] Figure 2b A schematic diagram of the operation of an AO module and an automatic exposure module disclosed in an embodiment of the present application;

[0061] Figure 3a A schematic diagram of an implementation method for adjusting exposure parameters of an automatic exposure module disclosed in an embodiment of the present application;

[0062] Figure 3b A schematic diagram of a face frame of an image frame disclosed in an embodiment of the present application;

[0063] Figure 4 This is a flow chart of a method for adjusting exposure parameters disclosed in an embodiment of the present application;

[0064] Figure 5 This is an example diagram of a method for adjusting exposure parameters disclosed in an embodiment of the present application;

[0065] Figure 6 This is a flow chart of a method for determining a proportional coefficient disclosed in an embodiment of the present application;

[0066] Figure 7 This is an example diagram of a method for determining a proportional coefficient disclosed in an embodiment of the present application;

[0067] Figure 8 This is a flow chart of another exposure parameter adjustment method disclosed in an embodiment of the present application;

[0068] Figure 9 This is an example diagram of another exposure parameter adjustment method disclosed in an embodiment of the present application;

[0069] Figure 10 This is a flow chart of another method for adjusting exposure parameters disclosed in an embodiment of the present application;

[0070] Figure 11 This is an example diagram of another exposure parameter adjustment method disclosed in an embodiment of the present application;

[0071] Figure 12 This is a flowchart of a method for calculating a motion vector disclosed in an embodiment of the present application;

[0072] Figure 13 This is an example diagram of a method for calculating a motion vector disclosed in an embodiment of the present application;

[0073] Figure 14 This is a flowchart of another method for adjusting exposure parameters disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The terms "first", "second" and "third" in the specification, claims and drawings of this application are used to distinguish different objects rather than to limit a specific order.

[0075] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0076] The present application embodiment provides an electronic device 100, see Figure 1aThe electronic device 100 may include: a processor 110, an internal memory (also called "memory") 120, an external memory 121, a sensor module 130, a camera 140, a display screen 150, etc. The sensor module 130 may include a gyroscope sensor 130A, etc.

[0077] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0078] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0079] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0080] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0081] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0082] The internal memory 120, which may also be referred to as "memory", may be used to store computer executable program codes, which include instructions. The internal memory 120 may include a program storage area and a data storage area. The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 120, and / or instructions stored in a memory provided in the processor.

[0083] External memory 121 generally refers to external storage. In the present embodiment, external memory refers to storage other than the electronic device's internal memory and the processor's cache memory. This storage is generally non-volatile memory. Common external storage includes hard disks, floppy disks, optical disks, USB flash drives, and Micro SD cards, which are used to expand the storage capacity of the electronic device 100. The external memory can communicate with the processor 110 via an external memory interface or bus to implement data storage. For example, files such as music and videos can be stored in the external memory.

[0084] The sensor module 130 may specifically include a gyroscope sensor 130A. The gyroscope sensor 130A may be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) may be determined by the gyroscope sensor 130A. The gyroscope sensor 130A may be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 130A detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 130A may also be used for navigation and somatosensory game scenes.

[0085] The electronic device 100 can implement a shooting function through an ISP, a camera 140, a video codec, a GPU, a display screen 150, and an application processor.

[0086] The ISP processes data fed back by camera 140. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 140.

[0087] The camera 140 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 140, where N is a positive integer greater than 1.

[0088] Electronic device 100 implements display functionality through a GPU, display screen 150, and an application processor. The GPU is a microprocessor for image processing that connects display screen 150 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0089] The display screen 150 is used to display images, videos, and the like. The display screen 150 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, the electronic device 100 may include one or N display screens 150, where N is a positive integer greater than one.

[0090] The above is a detailed description of the embodiments of the present application using electronic device 100 as an example. It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on electronic device 100. Electronic device 100 may have more or fewer components than shown in the figures, may combine two or more components, or may have a different component configuration. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0091] In some possible embodiments, the internal memory 120 may be used to store one or more programs;

[0092] The processor 110 is used to execute one or more programs stored in the internal memory, so that the electronic device executes the exposure parameter adjustment method provided in any embodiment of the present application.

[0093] In some possible embodiments, the external memory 121 can be used as a computer storage medium to store computer programs. When the stored computer program is executed, it is specifically used to implement the exposure parameter adjustment method provided in any embodiment of the present application.

[0094] Users can Figure 1a The electronic device shown is used to browse web pages, news, articles, etc., and can also be used to play games and watch videos. When a user browses web pages, news, articles, plays games, or watches videos on an electronic device, the user will gaze at the electronic device's display for a long time. To accommodate this, the electronic device will detect the user's prolonged gaze and execute corresponding events, such as keeping the display on or reducing the ringtone volume.

[0095] The electronic device mentioned above may be a mobile phone, a tablet computer or the like.

[0096] Figure 1b The following describes a scenario where a user browses a web page through an electronic device. This scenario is used as an example to describe how an electronic device can detect when a user has been looking at the display for a long time and execute corresponding events.

[0097] See also Figure 2a The simplest image front end (IFE lit) unit is an integrated unit in the image signal processor. The image output by the camera will reach the IFE lit integrated unit, which will store the image output by the camera in a secure buffer in the memory.

[0098] The automatic exposure module is a logic unit of the controller and is obtained by the controller running the automatic exposure (AE) algorithm.

[0099] The AO (always on) module is also a logical unit of the controller, generated by the controller running the AO (always on) solution. The AO solution refers to an intelligent perception solution based on an AO camera (always on camera). It typically includes functions such as gaze recognition, device owner recognition, and gesture recognition, and is typically characterized by long-term low-power operation.

[0100] The camera driver is also a logic unit of the controller, which is used to configure camera parameters and turn the camera on or off.

[0101] The display screen of the electronic device displays the web page, and the user looks at the display screen of the electronic device to view the web page. Figure 2a As shown, the electronic device's front camera operates, executing step S1 to capture a user's facial image. The minimalist image front-end unit executes step S2 to read the facial image and, based on a security mechanism, stores the facial image in a secure buffer in memory. The AO module executes step S3-1 to obtain image data of the facial image stored in the secure buffer in memory and, by analyzing the image data, determines whether the user is looking at the display. If the AO module determines that the user is looking at the display, it executes step S4 to control the electronic device's display to remain on.

[0102] The quality of the facial image captured by the camera restricts the accuracy of the AO module in determining whether the user's eyes are looking at the display screen. In particular, when the brightness of the facial image captured by the camera is high or low, the AO module will have a large error in determining whether the user's eyes are looking at the display screen. Figure 2aIn the process, the automatic exposure module obtains the image data of the face image stored in the memory according to step S3-2; uses the image data to calculate the image brightness of the face image, compares the calculated image brightness with the standard brightness, and obtains the comparison result; adjusts the exposure parameters of the camera according to the comparison result, generally the exposure time and gain, and obtains the exposure time adjustment value and the gain adjustment value. The automatic exposure module also executes step S4, transmits the calculated exposure time adjustment value and gain adjustment value to the AO module, and the AO module then adjusts the exposure time and gain adjustment value according to the comparison result. Figure 2a As shown in step S5, the exposure time adjustment value and the gain adjustment value are sent to the camera driver, and the camera driver follows Figure 2a As shown in step S7, the camera is configured to operate with the exposure duration adjustment value and the gain adjustment value.

[0103] The above-mentioned standard brightness can be pre-configured in the AO module. The standard brightness can be configured as the brightness of the facial image when the AO module determines that the error in whether the user's eyes are looking at the display is minimized, or as the brightness of the facial area of ​​the facial image when the AO module determines that the error in whether the user's eyes are looking at the display is minimized. For example, the standard brightness can be set to 522.

[0104] The following combination Figure 2b , describes how the AO module analyzes image data to determine whether the user's eyes are looking at the display screen, and how the automatic exposure module adjusts the exposure parameters of the camera.

[0105] Each time the automatic exposure module obtains a frame of image data, it uses this frame of image data to perform an exposure parameter adjustment process. The exposure parameters include the exposure duration and gain mentioned above. The following uses the first frame of the face image (also called image frame 1) as an example to illustrate the exposure parameter adjustment process:

[0106] See also Figure 2bThe image sequence includes multiple frames captured by a camera, such as frames 1, 2, 3, 4, ...n, wherein the camera initially operates with a common exposure duration and gain. Generally, the common exposure duration and gain can be pre-set. The automatic exposure module sequentially acquires image data for each frame in the image sequence according to the order in which the images were stored. For the first frame (also referred to as frame 1), the automatic exposure module uses the image data of frame 1 to calculate the image brightness of frame 1 and compares the image brightness of frame 1 with a standard brightness to obtain a comparison result. If the comparison result indicates that the difference between the image brightness of frame 1 and the standard brightness is less than a preset value (e.g., ±10%), the automatic exposure module performs no operation, and the camera continues to operate with the original exposure duration and gain, which are the pre-set common exposure duration and gain. If the comparison result indicates that the difference between the image brightness of frame 1 and the standard brightness is not less than the preset value, the automatic exposure module adjusts the camera's exposure duration and gain based on the comparison result, obtaining an exposure duration 1 adjustment value and a gain 1 adjustment value. The auto-exposure module adjusts the exposure duration and gain by 1 and transmits them to the camera driver via the AO module. The camera driver configures the camera to capture images using the exposure duration and gain adjusted by 1.

[0107] Because the auto-exposure module and camera driver execute a single process, which lags behind the camera's capture of an image, assume that image frames 2 and 3 were captured by the camera using the original exposure duration and gain. The auto-exposure module uses the image data of frames 2 and 3 according to the aforementioned processing method to calculate the exposure duration 1 adjustment value and the gain 1 adjustment value. The camera driver also configures the camera to capture the image using the exposure duration 1 adjustment value and the gain 1 adjustment value. Image frame 4 is captured by the camera configured to capture the image using the exposure duration 1 adjustment value and the gain 1 adjustment value. The auto-exposure module also uses the aforementioned processing method to calculate the exposure duration 2 adjustment value and the gain 2 adjustment value using the image data of frame 4. The camera driver then configures the camera to capture the image using the exposure duration 2 adjustment value and the gain 2 adjustment value. This process repeats until the auto-exposure module determines that the difference between the image brightness of the image frame and the standard brightness is less than a preset value, such as ±10%.

[0108] The AO module also sequentially acquires image data for each frame of the image sequence, following the order in which the images were stored. For each frame of image data acquired by the AO module, the module executes the following process to determine whether the human eye is looking at the display screen in each frame. The following uses the AO module's processing of image frame 1 as an example.

[0109] The AO module compares the image data of image frame 1 with the sample feature library. Based on the comparison results, it assigns a confidence level to image frame 1. This confidence level represents the probability that the human eye in image frame 1 is looking at the display screen. The AO module determines whether the confidence level of image frame 1 is less than a threshold. If the confidence level of image frame 1 is not less than the threshold, it is determined that the human eye in image frame 1 is looking at the display screen. If the confidence level of image frame 1 is less than the threshold, it is determined that the human eye in image frame 1 is not looking at the display screen.

[0110] In some embodiments, the sample feature library includes feature data of images of a person looking at a display screen. This feature data is determined by obtaining a large number of sample images, including sample images of the person looking at the display screen and sample images of the person not looking at the display screen, and using the image data of each sample image for learning to obtain feature data representing the image of the person looking at the display screen. Both the sample images of the person looking at the display screen and the sample images of the person not looking at the display screen refer to facial images captured by the front-facing camera of the electronic device.

[0111] The AO module determines that there is a frame of image within a certain period of time in which human eyes are looking at the display screen, and then executes corresponding events such as controlling the display screen of the electronic device to not turn off, reducing the ringtone volume, etc.

[0112] A face image can be divided into two parts: the face and the background. The image quality of the background has a weaker impact on the accuracy of gaze detection results, while the image quality of the face has a more significant impact on the accuracy of gaze detection results.

[0113] Therefore, in order to reduce the amount of calculation and obtain the appropriate exposure parameters more quickly, the automatic exposure module can adjust the exposure parameters according to the facial brightness of the image frame, and stop adjusting the exposure parameters when the difference between the facial brightness of the image frame and the standard brightness is less than a preset value.

[0114] See Figure 3a The method for adjusting exposure parameters according to the brightness of a facial image may include the following steps:

[0115] After the automatic exposure module obtains the image frame 1 , it performs the following steps S301 to S303 on the image frame 1 .

[0116] S301, comparing image brightness with standard brightness.

[0117] The image brightness is calculated based on the image data of image frame 1. Image frame 1 may be the first image frame captured by the camera.

[0118] The image brightness of a certain image frame can be equal to the average brightness value of all pixels in the image frame. The brightness value Bri of each pixel in the image frame can be calculated by substituting the red pixel value R, green pixel value G and blue pixel value B of the pixel, that is, the RGB pixel value of the pixel into the following formula:

[0119] Bri=0.3×R+0.6×G+0.1×B

[0120] If the difference between the image brightness and the standard brightness is less than the preset value, the method ends; if the difference between the image brightness and the standard brightness is not less than the preset value, step S302 is executed.

[0121] Figure 3a In the example, the difference between the image brightness of the image frame 1 and the standard brightness is not less than the preset value, so step S302 is executed.

[0122] S302, adjusting exposure parameters.

[0123] In step S302, the automatic exposure module adjusts the exposure parameters based on the difference between the image brightness of image frame 1 and the standard brightness, obtaining an adjusted exposure parameter value. The specific execution process of step S302 can be found in the aforementioned exposure parameter adjustment process and will not be further described. The adjusted exposure parameter value obtained after executing step S302 is transmitted to the camera driver, which configures the camera to capture images using the adjusted exposure parameter value.

[0124] The exposure parameter adjustment value may include an exposure duration adjustment value and a gain adjustment value. After the automatic exposure module obtains the exposure parameter adjustment value from each frame of image, it may send the exposure parameter adjustment value to the camera driver. The camera driver then captures one or more image frames according to the configured camera and the exposure parameter adjustment value.

[0125] Exemplarily, after the automatic exposure module executes step S302, it sends the obtained exposure parameter adjustment value to the camera driver. At this time, the camera is ready to shoot image frame 4, so the camera driver configures the camera to shoot image frame 4 according to the exposure parameter adjustment value obtained in step S302.

[0126] S303, face detection.

[0127] After performing face detection on image frame 1, a face frame for image frame 1 is obtained. The face frame for image frame 1 can be stored in the memory of the electronic device. For an image frame, if a face is present in the image frame, face detection can obtain the face frame for the image frame. The face frame for the image frame is used to indicate the area within the image frame where the face is located.

[0128] In an electronic device, a face frame for a facial image can be stored in the electronic device's memory in the form of coordinate data for multiple vertices. Each coordinate data point corresponds to a specific pixel in the facial image. Connecting the pixels corresponding to these coordinate data points with line segments yields a polygonal frame, which serves as the face frame for the facial image. Therefore, in any embodiment of the present application, obtaining the face frame for an image frame is equivalent to reading the coordinate data for each vertex of the face frame.

[0129] by Figure 3b For example, Figure 3b -1 indicates the image frame 1 captured by the camera. Figure 3b -1 After face detection by the face detection module, the coordinate data is obtained: P1 (X1, Y1), P2 (X2, Y1), P3 (X2, Y2), P4 (X1, Y2). In image frame 1, the pixel points corresponding to the above coordinate data are determined and the corresponding pixel points are connected to obtain the following: Figure 3b -2 shown in the face frame 1.

[0130] In some possible embodiments, step S303 may be implemented by a face detection module. The face detection module is a logical unit of the controller, which is generated by the controller running a face detection algorithm. The face detection module may be a submodule configured within the automatic exposure module or an independent module.

[0131] Face detection technology refers to the process of determining whether an image or video contains a face and locating the face's position and size. The prerequisite for implementing face detection is to build a face detector. In this application, the face detector can be constructed in a variety of ways. For example, the face detection module can pre-build a face detector in the following way and use the face detector to implement face detection:

[0132] Face detection is achieved through Haar-Like features and the Adaboost algorithm. In this method, Haar-Like features are used to represent faces. Each Haar-Like feature is trained to obtain a weak classifier. The Adaboost algorithm is used to select multiple weak classifiers that best represent faces to construct a strong classifier. Several strong classifiers are connected in series to form a cascaded classifier, namely a face detector.

[0133] After the automatic exposure module obtains the image frame 2, steps S304, S305, S306 and S307 are sequentially performed on the image frame 2.

[0134] S304, cropping the facial image.

[0135] The automatic exposure module crops the facial image of the image frame 2 using the facial frame of the image frame 1 .

[0136] See Figure 3b in Figure 3b -3. After obtaining the face frame of image frame 1 and image frame 2, the automatic exposure module can determine the face frame of image frame 1 in image frame 2.

[0137] Afterwards, the automatic exposure module calculates whether each pixel of image frame 2 is within the face frame 1 detected from image frame 1 based on the face frame of image frame 1. This allows the module to read each pixel of image frame 2 that is within the face frame 1. The image composed of these pixels in image frame 2 that are within face frame 1 is the facial image cropped from image frame 2 using face frame 1. See [Image frame 2 image] for details. Figure 3b ,in Figure 3b-4 It is the facial image obtained by cropping the face frame 1 from the image frame 2.

[0138] S305, comparing the facial brightness with the standard brightness.

[0139] When step S305 is executed for image frame 2, the facial brightness used for comparison is the facial brightness of image frame 2. The facial brightness of image frame 2 can be calculated using the image data of the cropped facial image of image frame 2. The specific calculation method can refer to the method for calculating image brightness based on the image data of image frame 1 described above and will not be repeated here.

[0140] Exemplarily, the facial brightness of image frame 2 may be equal to the average brightness of all pixels in the facial image of image frame 2 .

[0141] After executing step S305, if the difference between the facial brightness and the standard brightness is less than the preset value, the method ends; if the difference between the facial brightness and the standard brightness is not less than the preset value, step S306 is executed.

[0142] In this embodiment, the difference between the facial brightness of the image frame 2 and the standard brightness is not less than the preset value, so step S306 is executed.

[0143] S306, adjusting exposure parameters.

[0144] In step S306, the automatic exposure module adjusts the exposure parameter according to the difference between the face brightness and the standard brightness of image frame 2 to obtain an exposure parameter adjustment value. The specific execution process of step S306 can refer to the above-mentioned exposure parameter adjustment process.

[0145] Similar to step S302 , the exposure parameter adjustment value obtained in S306 is sent to the camera driver, and the camera driver configures the camera to capture one or more image frames according to the exposure parameter adjustment value obtained in S306 .

[0146] Exemplarily, when the exposure parameter adjustment value obtained in step S306 is sent to the camera driver, the camera is ready to capture image frame 5, so the camera driver configures the camera to capture image frame 5 according to the exposure parameter adjustment value obtained in step S306.

[0147] S307, face detection.

[0148] The specific implementation of step S307 is the same as step S303, that is, using the face detection module to perform face detection on the specific image frame to obtain the face frame of the specific image frame. The difference between step S307 and step S303 is that the image frame detected in step S303 is image frame 1, and the image frame detected in step S307 is image frame 2.

[0149] like Figure 3a As shown, starting from image frame 2, the automatic exposure module crops each image frame using the face frame of the previous image frame to obtain a face image of the image frame, and then determines the face brightness of the image frame based on the face image of the image frame. Thereafter, if the difference between the face brightness of the image frame and the standard brightness is less than a preset value, the method ends. If the difference between the face brightness of the image frame and the standard brightness is not less than the preset value, the exposure parameters are adjusted using the face image of the image frame. After the exposure parameters are adjusted, face detection is performed on the image frame to obtain a face frame of the image frame. This process is repeated until the difference between the face brightness of a certain image frame and the standard brightness is less than the preset value.

[0150] The above method of adjusting exposure parameters based on the brightness of the facial image has the following problems:

[0151] See Figure 3b ,exist Figure 3b -1 to Figure 3b-4 In the example shown, the user's face is stationary during the shooting process, and the position of the face in image frame 1 is substantially consistent with the position of the face in image frame 2. Therefore, the face frame of image frame 1 can be used to more accurately crop the face image of image frame 2.

[0152] However, in some shooting scenes, the user's face (or the electronic device used for shooting) may be in motion during the shooting process, resulting in inconsistent facial positions in the two adjacent frames of the shot. Figure 3b , Figure 3b middle Figure 3b-5 to Figure 3b-8 is an example of an image captured in motion, where Figure 3b-5 represents image frame 1, Figure 3b-6 represents the face frame of image frame 1 obtained by face detection of image frame 1, Figure 3b-7Represents image frame 2.

[0153] See also Figure 3b-7 After the face frame of image frame 1 is determined in image frame 2, it can be seen that there is a deviation between the area where the face in image frame 2 is actually located and the area within the face frame of image frame 1.

[0154] See also Figure 3b-8 , Figure 3b-8 Based on Figure 3b-7 The face frame of image frame 1 and the cropped face image of image frame 2 are shown. It can be seen that in motion, the facial image obtained by cropping image frame 2 using the face frame of image frame 1 does not accurately match the actual face in image frame 2. Instead, it contains part of the face in image frame 2 and part of the background. Therefore, the facial brightness calculated based on the image data of the cropped facial image is inaccurate, ultimately resulting in poor accuracy of the exposure parameters adjusted based on the facial brightness.

[0155] Furthermore, in the scenario shown in Figure 1 where the electronic device detects that a user is staring at the display screen for a long time, in order to reduce the power consumption of the electronic device, the AO module generally configures an extremely low frame rate for the camera driver. Therefore, the shooting time interval between the two adjacent frames of images is longer, and the deviation of the position of the face in the two adjacent frames of images is greater, resulting in the final exposure parameter adjustment value calculated based on the difference between the facial brightness and the standard brightness being more inaccurate.

[0156] For example, the frame rate configured by the AO module may be 5 frames per second. At this frame rate, the camera captures a face image frame every 200 milliseconds (ms).

[0157] In order to solve the problems existing in the above-mentioned method of adjusting exposure parameters according to the brightness of the face image, the present application provides a method for adjusting exposure parameters. The method for adjusting exposure parameters provided by the present application is described in detail below with reference to an embodiment.

[0158] Example 1

[0159] See Figure 4 , a method for adjusting exposure parameters provided in this embodiment may include the following steps.

[0160] S401: The automatic exposure module obtains the face frame of the previous frame image and the current frame image.

[0161] The automatic exposure module can specifically read the face frame of the previous frame image and the current frame image from the memory.

[0162] The current frame image refers to the frame image currently used by the automatic exposure module to adjust the exposure parameters. For example, when the automatic exposure module needs to use the N+1 frame image captured by the camera to adjust the exposure parameters, the N+1 frame image is the current frame image.

[0163] The previous frame image refers to the frame image captured by the camera before the current frame image. For example, if the current frame image is the N+1th frame image, the previous frame image is the Nth frame image.

[0164] See Figure 5 , Figure 5 This is an example of the automatic exposure module using the N+1 frame image to adjust the exposure parameters. Figure 5 a represents the Nth frame image above, Figure 5 b represents the face frame of the Nth frame image obtained by face detection of the Nth frame image, Figure 5 d represents the N+1th frame image.

[0165] The face frame of the Nth frame image is obtained by performing face detection on the Nth frame image by the face detection module before executing step S401 , that is, before obtaining the face frame of the Nth frame image and the N+1th frame image.

[0166] contrast Figure 5 a and Figure 5 d It can be seen that there is a deviation in the facial area between the Nth frame image and the N+1th frame image.

[0167] S402: The automatic exposure module calculates the corrected size using the scale factor and the original size.

[0168] The original size refers to the size of the face frame of the previous frame image in step S401, specifically including the length L1 and width W1 of the face frame of the previous frame image.

[0169] Corrected dimensions include corrected length and corrected width.

[0170] When the face frame of the current frame image only includes coordinate data of vertices but does not include length data and width data, the automatic exposure module can calculate the length L1 and width W1 of the face frame using the coordinate data included in the face frame.

[0171] For example, the face frame of the previous frame includes coordinate data for four vertices P1, P2, P3, and P4, where the coordinate data for each vertex is: P1 (X1, Y1), P2 (X2, Y1), P3 (X2, Y2), and P4 (X1, Y2). The automatic exposure module subtracts the horizontal coordinates of two coordinate data with the same vertical coordinate to obtain a width W1. In this example, W1 is equal to the absolute value of the difference between X1 and X2. The automatic exposure module subtracts the vertical coordinates of two coordinate data with the same horizontal coordinate to obtain a length L1. In this example, L1 is equal to the absolute value of the difference between Y1 and Y2.

[0172] For example, the origin of the horizontal and vertical coordinates in an image frame can be the pixel at the bottom left corner of the image. The values ​​of the horizontal and vertical coordinates indicate the column and row of the corresponding pixel in the image. For example, P1(X1, Y1) means that P1 is the pixel at the X1th column and Y1th row in the image.

[0173] See Figure 5 b. After the automatic exposure module obtains the face frame of the Nth frame image, the length L1 and width W1 of the face frame of the Nth frame image can be calculated according to the above method.

[0174] After obtaining the length L1 and width W1 of the face frame corresponding to the previous frame image, the automatic exposure module multiplies the length L1 by the first scaling factor Sy to obtain a corrected length L2, where L2 is equal to L1 multiplied by Sy. The automatic exposure module then multiplies the width W1 by the second scaling factor Sx to obtain a corrected width W2, where W2 is equal to W1 multiplied by Sx. The first scaling factor and the second scaling factor are collectively referred to as scaling factors. The scaling factors can be dynamically determined by the automatic exposure module during the execution of the face frame correction method of this embodiment, or can be pre-configured in the automatic exposure module, or can be pre-configured in the AO module and provided to the automatic exposure module by the AO module.

[0175] S403: The automatic exposure module determines a corrected face frame according to the corrected size and the original size.

[0176] After obtaining the corrected width W2 and the corrected length L2, the automatic exposure module subtracts W2 from W1 to obtain a width change ΔW, and subtracts L2 from L1 to obtain a length change ΔL. The automatic exposure module then calculates a corrected face frame based on the width change ΔW and the length change ΔL, as well as the coordinate data of the four vertices in the face frame of the previous frame. The face frame corresponding to the corrected face frame is called the corrected face frame.

[0177] In some possible implementations, the automatic exposure module determines the corrected face frame according to the width change ΔW and the length change ΔL, and the coordinate data of the four vertices in the face frame of the previous frame image as follows:

[0178] For the face frames P1(X1, Y1), P2(X2, Y1), P3(X2, Y2), and P4(X1, Y2) of the previous frame, assuming that X1 is greater than X2 and Y1 is greater than Y2, the automatic exposure module obtains the width change ΔW and length change ΔL and calculates the corrected coordinates using the following formula:

[0179] X10=X1-ΔW÷2

[0180] X20=X2+ΔW÷2

[0181] Y10=Y1-ΔL÷2

[0182] Y20=Y2+ΔL÷2

[0183] X10, X20, Y10, and Y20 represent the corrected coordinates. Finally, the corrected coordinates are used to replace the corresponding coordinates in the face frame of the previous frame image. That is, X1 is replaced by X10, X2 is replaced by X20, Y1 is replaced by Y10, and Y20 is replaced by Y2. This results in the following corrected face frame, where P5 to P8 represent the vertices corresponding to the corrected face frame:

[0184] P5(X10, Y10), P6(X20, Y10), P7(X20, Y20), P8(X10, Y20).

[0185] The above-mentioned method of correcting the face frame according to the width change and the length change is equivalent to fixing the center point of the face frame before correction, and then reducing the width and length of the face frame according to the width change and the length change to obtain the corrected face frame.

[0186] Steps S402 and S403 are equivalent to the process of correcting the face frame of the previous frame image according to the proportional coefficients Sx and Sy.

[0187] See Figure 5 b and Figure 5 c. After determining the width change ΔW and length change ΔL based on the corrected size and the original size, the automatic exposure module keeps the coordinates of the center point of the face frame of the Nth frame unchanged and moves the face frame according to the width change ΔW and length change ΔL. Figure 5 The vertices P1 to P4 of the face frame of the Nth frame image shown in b are obtained Figure 5 The vertices P5 to P8 after movement shown in c, Figure 5 The polygonal frame defined by the moved vertices in c is the corrected face frame obtained in step S403.

[0188] In other possible implementations of the present application, the automatic exposure module may also fix any point of the face frame before correction, for example, fix the vertex of the upper left corner of the face frame before correction, and then reduce the width and length of the face frame according to the width change and length change to obtain the corrected face frame.

[0189] contrast Figure 5 b and Figure 5 c It can be seen that compared with the face frame of the Nth frame image, through steps S401 and S403, a corrected face frame with a length L2, a width W2, and a center point consistent with the face frame of the Nth frame image can be obtained.

[0190] S404: The automatic exposure module crops the current frame image with the corrected face frame to obtain a face image of the current frame image.

[0191] After obtaining the corrected face frame, the automatic exposure module determines the corrected face frame in the current frame image, and then extracts the pixels in the current frame image that are located within the corrected face frame. The image composed of these pixels is the cropped facial image of the current frame image.

[0192] See Figure 5 d and Figure 5 e. After the corrected face frame is determined in the N+1th frame image, the pixels within the corrected face frame can be extracted from the N+1th frame image. The image composed of these pixels within the corrected face frame is Figure 5 The facial image of the cropped N+1th frame image shown in e.

[0193] from Figure 5 It can be seen from e that the facial image obtained by cropping the corrected face frame only contains Figure 5 The part of the face in the N+1 frame image shown in a hardly contains Figure 5 The background of a.

[0194] S405: The automatic exposure module adjusts exposure parameters according to the facial image of the current frame image.

[0195] The specific execution process of step S405 can be found in Figure 3a Step S305 and step S302 in the illustrated embodiment will not be described in detail.

[0196] The method for correcting the face frame provided in this embodiment has the following beneficial effects:

[0197] When cropping the current frame based on the face frame of the previous frame, reducing the size of the face frame helps reduce the proportion of background pixels in the cropped facial image, thereby reducing the interference of background pixels on the calculated facial brightness, ensuring that the facial brightness of the current frame image is accurately obtained when subsequently adjusting the exposure parameters.

[0198] The proportionality coefficient used in the first embodiment can be determined as follows.

[0199] Please refer to Figure 6 A method for determining a proportionality coefficient provided in an embodiment of the present application may include the following steps:

[0200] The method for determining the proportional coefficient provided in this embodiment can be obtained by Figure 2a The automatic exposure module or AO module in the electronic device with intelligent sensing function shown in the figure can be executed, or it can be performed by a different Figure 2a The electronic device shown in the figure is executed by another electronic device, and the other electronic device can transmit the proportional coefficient to the electronic device after determining the proportional coefficient. Figure 2a The electronic device shown makes Figure 2a The automatic exposure module in the electronic device shown can execute the correction method provided in the aforementioned embodiment based on the proportional coefficient.

[0201] S601: Obtain face frames of multiple frames of images.

[0202] In some possible embodiments, the above method for determining the scale factor may be performed using two frames of images.

[0203] See Figure 7 , Figure 7 This is an example diagram of determining the scale factor using two frames of images.

[0204] Figure 7 middle, Figure 7 a and Figure 7 b represents the two frames of images captured. Figure 7 c and Figure 7 d is the face frame of the two frames of images obtained in step S601, where Figure 7 c is for Figure 7 aThe face frame Q1 obtained after face detection, Figure 7 d is for Figure 7 d. The face frame Q2 obtained after face detection, where the face frame Q1 includes the following coordinate data:

[0205] P1(X1, Y1), P2(X2, Y1), P3(X2, Y2), P4(X1, Y2).

[0206] The face frame Q2 includes the following coordinate data:

[0207] P5(X3, Y3), P6(X4, Y3), P7(X4, Y4), P8(X3, Y4).

[0208] S602: Determine overlapping areas of face frames of multiple image frames.

[0209] The overlap region is the area where the face frames of multiple frames intersect. Similar to the face frames, the overlap region can also be represented by vertex coordinate data.

[0210] For any two rectangular face frames, the overlapping area of ​​the two face frames can be determined as follows:

[0211] Determine the upper left corner vertex and lower right corner vertex of the two face frames respectively, and obtain two upper left corner vertices and two lower right corner vertices. Then, select the maximum value of the horizontal coordinates of the two upper left corner vertices as the first horizontal coordinate, select the minimum value of the vertical coordinates of the two upper left corner vertices as the first vertical coordinate, select the minimum value of the horizontal coordinates of the two lower right corner vertices as the second horizontal coordinate, and select the maximum value of the vertical coordinates of the two lower right corner vertices as the second vertical coordinate. Using the first horizontal coordinate, the first vertical coordinate, the second horizontal coordinate, and the second vertical coordinate, four coordinate data can be determined. The vertices corresponding to these four coordinate data are the vertices of the overlapping area of ​​the two face frames.

[0212] The method to determine the upper left corner vertex of the face frame is:

[0213] The smaller of the two horizontal coordinates included in the coordinate data of the face frame is selected, and the larger of the two vertical coordinates included in the coordinate data of the face frame is selected. The vertex determined by the smaller horizontal coordinate and the larger vertical coordinate is the upper left corner vertex of the face frame.

[0214] The method to determine the lower right corner vertex of the face frame is:

[0215] The larger of the two horizontal coordinates included in the coordinate data of the face frame is selected, and the smaller of the two vertical coordinates included in the coordinate data of the face frame is selected. The vertex determined by the larger horizontal coordinate and the smaller vertical coordinate is the lower right corner vertex of the face frame.

[0216] See Figure 7 e, in obtaining Figure 7 The face frame Q1 shown in c and Figure 7 d, P1 is determined as the upper left corner vertex of the face frame Q1, P3 is determined as the lower right corner vertex of the face frame Q1, P5 is determined as the upper left corner vertex of the face frame Q2, and P7 is determined as the lower right corner vertex of the face frame Q2.

[0217] Then, compare the horizontal coordinates X1 and X3 of P1 and P5, select the maximum value max(X1, X3), and record it as Xc1; compare the vertical coordinates Y1 and Y3 of P1 and P5, select the minimum value min(Y1, Y3), and record it as Yc1; compare the horizontal coordinates X2 and X4 of P3 and P7, select the minimum value min(X2, X4), and record it as Xc2; compare the vertical coordinates Y2 and Y4 of P3 and P7, select the maximum value max(Y2, Y4), and record it as Yc2.

[0218] from Figure 7 As can be seen from Figure 5, the rectangle enclosed by the straight lines corresponding to Xc1, Xc2, Yc1, and Yc2, and the overlapping area Q3 of the face frame Q1 and the face frame Q2, can be represented by the following coordinate data in the electronic device:

[0219] Pc1 (Xc1, Yc1), Pc2 (Xc2, Yc1), Pc3 (Xc2, Yc2), Pc4 (Xc1, Yc2).

[0220] S603: Calculate the ratio of the size corresponding to the overlapping area to the size corresponding to the face frame of any of the above frames to obtain a proportional coefficient.

[0221] The overlapping area obtained by stacking multiple rectangular face frames is also a rectangular area. In step S603, the length of the overlapping area is divided by the length of any stacked face frame. The obtained ratio is the first proportional coefficient Sy. The width of the overlapping area is divided by the width of any stacked face frame. The obtained ratio is the second proportional coefficient Sx.

[0222] See Figure 7 After determining the overlapping area Q3 between face frames Q1 and Q2, the coordinate data representing the overlapping area is used to calculate the length Lc = Yc1 - Yc2 and the width Wc = Xc2 - Xc1 of the overlapping area Q3. Finally, the length Lc of the overlapping area Q3 is divided by the length L2 of the face frame Q2 to obtain the first proportionality factor Sy. The width Wc of the overlapping area is divided by the width W2 of the face frame Q2 to obtain the second proportionality factor Sx.

[0223] Understandably, Figure 6 The method for determining the overlapping area of ​​two face frames shown in FIG can be used to determine the overlapping area of ​​any two rectangular frames including the face frame. For the specific process, see Figure 6 The above-mentioned embodiments will not be described in detail.

[0224] Although Figure 7 The example uses face frames of two frames of images as an example, but the method for determining the proportional coefficient provided in this embodiment can also use face frames of three or more frames of images to determine the proportional coefficient.

[0225] Taking three frames of images, image 1, image 2 and image 3, as an example, we can use the face frame of image 1 and the face frame of image 2 to determine the first overlapping area according to the above method. Since the first overlapping area also corresponds to a rectangular frame, we can continue to press Figure 6 The method shown uses the first overlapping area and the face frame of image 3 to determine a second overlapping area. The resulting second overlapping area is the overlapping area of ​​the face frames of image 1, image 2, and image 3. Similarly, calculations for more than three frames can be performed using the above example method, and will not be repeated here.

[0226] In some possible implementations, Figure 2a The automatic exposure module of the electronic device shown, or Figure 2b AO module of the electronic device shown, or and Figure 2a The electronic device shown in the figure is different from other electronic devices, which can obtain the face frame of multiple frames of images in advance, and then press Figure 6 The method shown calculates the proportional coefficient and pre-stores the proportional coefficient in the automatic exposure module. Thereafter, each time the automatic exposure module corrects the face frame according to the above embodiment, the pre-stored proportional coefficient is used for correction.

[0227] In other possible implementations, the automatic exposure module may further calculate a scaling factor for each face frame correction using the face frames of S previously captured frames, thereby dynamically updating the scaling factor. S is a preset integer greater than or equal to 2.

[0228] To dynamically update the scaling factor, the automatic exposure module maintains a data buffer in memory for caching the face frames of S frames. Starting from the first frame captured by the camera, each time the automatic exposure module uses the face detection module to identify the face frame for a frame, it stores the face frame in the data buffer. If the automatic exposure module finds the data buffer full before storing the face frame, it deletes the oldest stored face frame and stores the face frame of the current frame. This way, when the automatic exposure module adjusts the exposure parameters for frame N (where N is greater than S), the data buffer contains the face frames for frames Ns through N-1.

[0229] Based on the above data buffer, the automatic exposure module reads the face frames of S frames from the data buffer before adjusting the exposure parameters of the Nth frame image (N is greater than S), and performs the face frame adjustment based on the face frames of these S frames. Figure 6 The method for determining the scale coefficient is shown in FIG. 1 , thereby obtaining the scale coefficient applicable to the N-th frame image, and then using the scale coefficient applicable to the N-th frame image to correct the face frame of the N-1-th frame to obtain the corrected face frame.

[0230] Optionally, in the dynamic update scheme of the scaling coefficient, if the data buffer has not yet stored the face frame of the S frame, the automatic exposure module may use the pre-stored scaling coefficient to correct the face frame when adjusting the exposure parameters.

[0231] In the correction method provided in the above embodiment, the benefits of dynamically updating the proportional coefficient are:

[0232] In multiple frames captured continuously by a camera, the change in facial position between two adjacent frames is not fixed. This dynamic update method ensures that each time the face frame is corrected, a proportional coefficient that matches the change in facial position between two adjacent frames is obtained. This method preserves as many facial pixels as possible while minimizing the background content in the cropped image data, helping to improve the accuracy of calculated facial brightness.

[0233] Furthermore, in the dynamic update scheme for the proportional coefficient, the automatic exposure module can clear the data in the data cache based on the magnitude of the change in the phone's posture. Each time, before storing the face frame of a frame into the data cache, the automatic exposure module can obtain the posture data of the electronic device when the frame was captured. Then, using the posture data from the time the frame was captured and the posture data from the time the target image was captured, the module calculates the magnitude of the posture change of the frame relative to the target image. If the posture change is no greater than a preset threshold, the face frame of the frame is directly stored. If the posture change is greater than the preset threshold, all face frames currently stored in the data cache are deleted, and the face frame of the frame is then stored.

[0234] The posture data of the electronic device when shooting a frame of image can be collected by the aforementioned gyro sensor 130A and written into the memory. The posture data of the electronic device can include the tilt angle of the electronic device. Accordingly, the posture change amplitude can be the angle difference of the tilt angles included in the two posture data.

[0235] In some optional embodiments, the target image includes a frame of image corresponding to each face frame currently stored in the data buffer.

[0236] In some other optional embodiments, the target image may also include only a frame of image corresponding to any face frame currently stored in the data buffer. For example, the target image may be a frame of image corresponding to a face frame stored earliest in the data buffer.

[0237] For example, assuming that the data cache stores face frames from image frames 1 to 3, when the automatic exposure module wants to store the face frame of image frame 4, the automatic exposure module respectively determines the posture change amplitude of image frame 4 relative to image frame 1, the posture change amplitude of image frame 4 relative to image frame 2, and the posture change amplitude of image frame 4 relative to image frame 3. If each posture change amplitude is not greater than a change amplitude threshold, the automatic exposure module stores the face frame of this frame of image into the data cache. If any posture change amplitude is greater than the change amplitude threshold, the automatic exposure module deletes the face frames from image frames 1 to 3 currently stored in the data cache, and then stores the face frame of image frame 4.

[0238] If the posture of the electronic device changes significantly between two frames, the overlapped area of ​​the facial frames in the two frames will obviously be smaller, resulting in a too-small scale factor determined using the facial frames from these two frames. By cleaning the data cache based on the posture change of the electronic device, we can avoid using the facial frames from two frames with significantly different postures to determine the scale factor, thus preventing the generation of too-small scale factors during dynamic updates.

[0239] Example 2

[0240] The second embodiment of the present application also provides a method for adjusting exposure parameters. Figure 8 The exposure parameter adjustment method provided in this embodiment includes the following steps.

[0241] S801: The automatic exposure module obtains the face frames of the previous two frames and the current frame.

[0242] The previous two frames are the two frames captured before the current frame. The current frame is captured by the camera and stored in the memory. The face frames of the previous two frames are detected by the face detection module and stored in the memory. In step S801, the automatic exposure module can read the above data from the memory.

[0243] See Figure 9 , Figure 9 This is an example of the exposure parameter adjustment method provided in this embodiment. Figure 9 a and Figure 9 b represents the first two frames of images in step S801, Figure 9 c represents the current frame image, Figure 9 d is obtained through face detection Figure 9 a's face frame Q1, Figure 9 e represents the face frame Q2 of Figure b obtained through face detection.

[0244] For example, if Figure 9 The current frame image shown in c is the Nth frame image, N is greater than or equal to 3, then Figure 9 a is the N-2th frame image, Figure 9 b is the N-1th frame image.

[0245] S802: The automatic exposure module determines the overlapping area of ​​the face frames of the first two frames of image as the corrected face frame.

[0246] The method for the automatic exposure module to determine the overlapping area of ​​the face frame of the first two frames can be found in Figure 6 The method described in step S602 of the embodiment shown.

[0247] See Figure 9 f and Figure 9 g. After obtaining the face frames of the first two frames, the automatic exposure module determines the following: Figure 9 f shows the overlapping area of ​​the face frame Q1 and the face frame Q2, and the overlapping area is determined as Figure 9 g shows the corrected face frame Q3.

[0248] S803: The automatic exposure module crops the current frame image using the corrected face frame to obtain a face image of the current frame image.

[0249] See Figure 9 h, Figure 9 h is the automatic exposure module Figure 9 The corrected face frame shown in g Figure 9 The facial image obtained by cropping the current frame image of c.

[0250] S804: The automatic exposure module adjusts exposure parameters according to the facial image of the current frame image.

[0251] The specific execution process of steps S803 and S804 can be found in Figure 4 Steps S404 and S405 of the illustrated embodiment are not described in detail here.

[0252] The method for correcting the face frame provided in this embodiment has the same beneficial effects as the method for correcting the face frame according to a specific scale factor provided in the previous embodiment.

[0253] In some possible embodiments, before calculating the overlap area using the face frames of the previous two frames, the automatic exposure module first obtains the posture data of the electronic device when the previous two frames were captured. Based on the posture data corresponding to the previous two frames, the module calculates the amplitude of the posture change of the electronic device during the previous two frames. If the amplitude of the posture change is less than or equal to a preset amplitude change threshold, the module continues to calculate the overlap area using the face frames of the previous two frames. If the amplitude of the posture change is greater than the amplitude change threshold, the module does not calculate the overlap area. If the amplitude of the posture change is greater than the amplitude change threshold, the automatic exposure module can directly use the face frame of the previous frame to crop the current frame, or can correct the face frame of the previous frame according to a preset scale factor and then use the face frame corrected by the scale factor to crop the current frame.

[0254] Continue Figure 9 In the example shown, the automatic exposure module uses Figure 9 d and Figure 9 Before calculating the overlapped area of ​​the face frame shown in e, first calculate the shooting area according to the method of calculating the posture change range mentioned above. Figure 9 a The image shown in Figure 9 b, the electronic device's posture change amplitude, if the posture change amplitude is less than or equal to the preset change amplitude threshold, the automatic exposure module will Figure 9 d and Figure 9 The face frame shown in e is determined Figure 9 If the posture change of the corrected face frame shown in h is greater than the change threshold, the automatic exposure module directly uses Figure 9 The face frame shown in e Figure 9 The image shown in c can be cropped, or the image can be cropped according to a preset ratio coefficient. Figure 9 e, and then use the face frame corrected by the proportional coefficient to align Figure 9 The image shown in c is cropped.

[0255] Example 3

[0256] Another embodiment of the present application provides a method for adjusting exposure parameters. Figure 10 The exposure parameter adjustment method provided in this embodiment may include the following steps.

[0257] S1001, the automatic exposure module obtains the face frame of the previous frame image, the previous two frames of image and the current frame image.

[0258] In a specific embodiment, after the camera starts capturing images, the automatic exposure module reads the first frame of image from the memory, adjusts the exposure parameters based on the first frame of image, and finally uses the face detection module to identify the face frame of the first frame of image.

[0259] The automatic exposure module reads the second frame image from the memory, corrects the face frame of the first frame according to a preset scale factor to obtain a corrected face frame of the first frame, crops the second frame image using the corrected face frame of the first frame to obtain a face image of the second frame image, adjusts the exposure parameters based on the face image of the second frame image, and finally uses the face detection module to recognize the face frame of the second frame image.

[0260] Thereafter, starting from the third frame, the automatic exposure module reads one frame of image from the memory each time, and then reads the previous two frames of image from the memory, and reads the face frame of the previous frame of image from the memory.

[0261] See Figure 11 ,in Figure 11 a and Figure 11 b represents the first two frames of images, Figure 11 c represents the current frame image, Figure 11 d represents the value obtained through face detection Figure 11 b is a face frame Q1, the vertices of the face frame Q1 include P1 to P4, and the face frame Q1 can be represented by the following coordinate data:

[0262] P1(X1, Y1), P2(X2, Y1), P3(X2, Y2), P4(X1, Y2).

[0263] For example, Figure 11 c can be the K+1th frame image, Figure 11 a is the K-1th frame image, Figure 11 b is the K-th frame image, Figure 11 d represents the face frame of the K-th frame image obtained through face detection, where K is greater than or equal to 2.

[0264] S1002: The automatic exposure module calculates a motion vector using the first two frames of image.

[0265] The specific implementation method of calculating the motion vector using the first two frames of image can be found below. Figure 12 The embodiment shown.

[0266] See Figure 11 , Figure 11 This is an example of the exposure parameter adjustment method of this embodiment. The automatic exposure module obtains Figure 11 a and Figure 11 After the two frames of images shown in b, use Figure 11 a and Figure 11 b is calculated Figure 11 The motion vector V1 is shown.

[0267] S1003: The automatic exposure module corrects the face frame of the previous frame image according to the motion vector to obtain a corrected face frame.

[0268] When executing step S1003, the automatic exposure module may add each coordinate data corresponding to the face frame of the previous frame image to the aforementioned motion vector to obtain updated coordinate data. After each coordinate data of the face frame of the previous frame image is updated, the polygonal frame represented by each updated coordinate data is the corrected face frame obtained based on the motion vector correction.

[0269] See Figure 11 .based on Figure 11 d shows the coordinate data of the face frame: P1 (X1, Y1), P2 (X2, Y1), P3 (X2, Y2), P4 (X1, Y2). The automatic exposure module adds P1 and the motion vector V1 to obtain updated coordinate data P10. The automatic exposure module adds P2 and the motion vector V1 to obtain updated coordinate data P20. The automatic exposure module adds P3 and the motion vector V1 to obtain updated coordinate data P30. The automatic exposure module adds P4 and the motion vector V1 to obtain updated coordinate data P40. Based on the updated coordinate data P10 to P40, it is possible to determine Figure 11 e shows the corrected face frame Q10.

[0270] For example, if the motion vector V1 is recorded as V1(Vx1, Vy1), the updated coordinate data is:

[0271] P10(X1+Vx1, Y1+Vy1), P20(X2+Vx1, Y1+Vy1), P30(X2+Vx1, Y2+Vy1), P40(X1+Vx1, Y2+Vy1).

[0272] from Figure 11 As can be seen, step S1003 is equivalent to translating the face frame of the previous frame image according to the calculated motion vector. The translation direction is the direction of the motion vector, and the translation distance is the length of the motion vector. The face frame obtained after the translation is the corrected face frame.

[0273] S1004: The automatic exposure module crops the current frame image using the corrected face frame to obtain a face image of the current frame image.

[0274] Continuing with the previous example, the automatic exposure module obtains Figure 11 After the corrected face frame Q10 is formed, the corrected face frame Q10 is used to correct the face. Figure 11 The current frame image shown in c is cropped to obtain Figure 11 f is the facial image of the current frame image.

[0275] S1005: The automatic exposure module adjusts exposure parameters according to the facial image of the current frame image.

[0276] The specific execution process of steps S1004 and S1005 can be found in Figure 4 Steps S404 and S405 of the illustrated embodiment are not described in detail here.

[0277] The exposure parameter adjustment method provided in this embodiment has the following advantages:

[0278] On the one hand, the automatic exposure module can adjust the coordinate data contained in the face frame according to the motion vector. The method for correcting the face frame in Example 3 can correct both face frames with rectangular shapes and face frames with shapes other than rectangular shapes. Therefore, the exposure parameter adjustment method provided in Example 3 can adjust the exposure parameters for face frames of any shape, and has a wider range of applicability.

[0279] On the other hand, since the motion vector reflects the movement direction and distance of the face in the current frame image relative to the face in the previous frame image, the face frame can be corrected according to the motion vector to obtain a corrected face frame that matches the position of the face in the current frame image. Without reducing the face frame, the proportion of the background part in the cropped face image is reduced, so that a larger area face image can be obtained and the face image can be prevented from including pixels of the background part. This makes the face brightness calculated based on the face image more accurate, and thus more accurate exposure parameters are obtained when adjusting the exposure parameters based on the face brightness.

[0280] In some possible implementations, when the automatic exposure module corrects the face frame based on motion estimation, it is not limited to using the first two frames of images to calculate the motion vector, but may use images captured earlier to calculate the motion vector.

[0281] Continuing with the above example, when the face frame of the Kth frame needs to be corrected, the automatic exposure module may also calculate the motion vector using the Kth frame image and the K-2th frame image.

[0282] In some possible implementations, the automatic exposure module may choose whether to calculate a motion vector based on the magnitude of the change in the posture of the electronic device. Each time before calculating the motion vector using the first two frames of imagery, the automatic exposure module first uses the posture data of the electronic device when the first two frames of imagery were captured to calculate the magnitude of the change in the posture of the electronic device during the period between the first two frames of imagery. If the magnitude of the change in posture is greater than a threshold value, the automatic exposure module does not calculate the motion vector. If the magnitude of the change in posture is less than or equal to the threshold value, the automatic exposure module continues to calculate the motion vector according to the method of the aforementioned embodiment.

[0283] Among them, when the automatic exposure module does not calculate the motion vector because the posture change amplitude is greater than the change amplitude threshold, the automatic exposure module can correct the face frame of the previous frame with a preset proportional coefficient, and then use the corrected face frame obtained by the proportional coefficient to crop the current frame image.

[0284] The benefits of choosing whether to calculate motion vectors based on the magnitude of posture changes are:

[0285] If the posture of the electronic device changes significantly between two frames, the position of the face will shift significantly between the two frames. Consequently, the motion vector calculated from these two frames will not accurately reflect the direction and distance of the face's movement. By selecting whether to calculate the motion vector based on the magnitude of the posture change, you can avoid excessive deviations in the calculated motion vectors, preventing a significant misalignment between the corrected face frame and the actual face in the cropped image.

[0286] See Figure 12 A method for calculating a motion vector using two frames of images may include the following steps:

[0287] S1201: Determine a query block in a reference frame image.

[0288] The reference frame image is shot earlier than the target frame image. In this embodiment, the reference frame image is the earlier shot of the two frames used to calculate the motion vector, and the target frame image is the later shot of the two frames used to calculate the motion vector.

[0289] In S1201 , the reference frame image may be divided into a plurality of image blocks according to a certain division method, and then the automatic exposure module designates an image block from the plurality of image blocks as a query block.

[0290] Exemplarily, the automatic exposure module may divide the reference frame image into a plurality of 10×10 image blocks, and then select an image block located in the center of the face frame of the reference frame image as the query block.

[0291] See Figure 13 ,in, Figure 13 a represents the reference frame image, Figure 13 b represents the target frame image. For example, Figure 13 a can be the K-1th frame image, Figure 13 b can be the Kth frame image as the target frame image.

[0292] Automatic exposure module acquisition Figure 13 After a frame of image shown in a, the face frame Q2 of this frame of image is read from the memory, and according to the face frame Q2, the face frame Q2 is selected. Figure 13In the image shown in a, the image block B1 located in the center of the face frame Q2 is used as the query block.

[0293] S1202: Search for a target block that matches the query block in the target frame image.

[0294] Similar to S1201, in S1202, the target frame image may also be divided into multiple image blocks. The automatic exposure module then searches for an image block from the multiple image blocks of the target frame image as the target block. Generally, the image blocks of the target frame image and the reference frame image are of the same size. In the above example, the target frame image may be divided into multiple 10×10 image blocks.

[0295] A specific implementation of S1202 is as follows:

[0296] First, an image block with the same position as the query block in the target frame image is selected, and the deviation value between the selected image block and the query block is calculated using the pixel data in the selected image block and the pixel data in the query block. If the deviation value is less than or equal to the preset deviation threshold, the currently selected image block is determined as the target block matching the query block. If the deviation value is greater than the preset deviation threshold, a new image block that has not been selected before is selected from multiple image blocks adjacent to the currently selected image block, and the deviation value between the selected new image block and the query block is calculated. This process is repeated until the target block is selected.

[0297] In some other optional implementations, the target block may also be defined as an image block in the target frame image that has the smallest deviation value from the query block.

[0298] In addition to the above-mentioned search method starting from the position of the query block, other search methods can be used when executing step S1202. For example, each image block of the target frame image can be searched one by one starting from the image block in the upper left corner of the target frame image. Details will not be repeated here.

[0299] The deviation value between two image blocks reflects the degree of difference in the contents of the two image blocks. The smaller the deviation value, the closer the contents of the two image blocks are.

[0300] The deviation value between the two image blocks may be the mean absolute difference (MAD) of the two image blocks, or may be the mean squared difference (MSD) of the two image blocks.

[0301] The calculation formula of the MAD of two image blocks is as follows:

[0302]

[0303] The calculation formula of the MSD of two image blocks is as follows:

[0304]

[0305] In the above formula, C ij Represents the grayscale value of the pixel in the i-th row and j-th column of an image block, R ij Represents the grayscale value of the pixel at row i and column j in another image block. M represents the number of rows of the two image blocks, and N represents the number of columns of the two image blocks.

[0306] See Figure 13 b, automatic exposure module Figure 13 After the reference frame image shown in a determines the query block, calculate one by one Figure 13 The deviation value of each image block and query block in the target frame image shown in b is calculated and found Figure 13 The deviation value between the image block B2 shown in b and the query block is less than or equal to the deviation threshold, so B2 is determined as the target block.

[0307] S1203: Calculate a motion vector based on the first position data and the second position data.

[0308] The first position data refers to the coordinate data of a point in the query block in the reference frame image. For example, the first position data may be the coordinate data of the center point of the query block in the reference frame image.

[0309] The second position data refers to the coordinate data of a certain point in the target block in the target frame image. For example, the second position data may be the coordinate data of the center point of the target block in the target frame image.

[0310] In a planar image, a motion vector can be represented by the difference between the horizontal coordinate and the vertical coordinate between the starting point and the end point of the vector. In step S1203, the starting point of the motion vector is the point corresponding to the first position data, and the end point is the point corresponding to the second position data.

[0311] See Figure 13 c. After determining the query block B1 and the target block B2, the automatic exposure module determines the coordinate data (Bx1, By1) of the center point of the query block B1 as the first position data, and the coordinate data (Bx2, By2) of the center point of the target block B2 as the second position data. Then, the motion vector V1 (Vx1, Vy1) is calculated based on the coordinate data (Bx1, By1) and (Bx2, By2), where:

[0312] Vx1=Bx2-Bx1; Vy1=By2-By1.

[0313] Understandably, Figure 12The illustrated embodiment is an optional method for calculating a motion vector using two frames of image data in the present application. In other possible embodiments of the present application, the automatic exposure module may also calculate a motion vector based on motion estimation methods such as region-matching methods and optical-flow methods. The present application does not limit the method for calculating the motion vector.

[0314] It should be noted that the above-mentioned embodiments 1 to 3 all use the images captured when the user's face is in motion as an example to illustrate the exposure parameter adjustment method provided in the present application. However, the exposure parameter adjustment method provided in the embodiments of the present application can also adjust the exposure parameters according to the images captured when the user's face is in a static state. The exposure parameter adjustment method provided in the embodiments of the present application does not limit whether the face is static or moving when the image is captured.

[0315] Example 4

[0316] This embodiment provides a method for adjusting exposure parameters. Figure 14 , the adjustment method may include the following steps.

[0317] After the camera is started, it continuously captures images at certain intervals and saves each captured frame in the memory.

[0318] After the automatic exposure module obtains the image frame 1 , it performs the following steps S1401 to S1403 on the image frame 1 .

[0319] S1401, comparing image brightness with standard brightness.

[0320] If the comparison finds that the difference between the image brightness of image frame 1 and the standard brightness is less than the preset value, the method of this embodiment ends. If the comparison finds that the difference between the image brightness of image frame 1 and the standard brightness is not less than the preset value, the automatic exposure module executes steps S1402 and S1403.

[0321] S1402, adjust exposure parameters.

[0322] S1403, face detection.

[0323] After performing face detection on the image frame 1, a face frame of the image frame 1 is obtained. The face frame of the image frame 1 can be stored in a memory of the electronic device.

[0324] For the specific execution process of steps S1401 to S1403, please refer to Figure 3a Steps 301 to 303 of the illustrated embodiment.

[0325] After the automatic exposure module stores the face frame of image frame 1 into the memory, it reads image frame 2 from the memory and then sequentially performs steps S1404, S1405, S1406, S1407 and S1408 on image frame 2.

[0326] S1404, correct the face frame.

[0327] In step S1404, the face frame of the image preceding the currently read image is corrected. When step S1404 is executed for image frame 2, the face frame of image frame 1 is corrected. After correction, the automatic exposure module obtains the corrected face frame.

[0328] The specific implementation of step S1404 can refer to the relevant steps of obtaining the corrected face frame in embodiments 1 to 3, and will not be repeated here.

[0329] S1405: Crop the facial image.

[0330] In this embodiment, the automatic exposure module uses the corrected face frame obtained in step S1404 to crop the facial image.

[0331] S1406, comparing the facial brightness with the standard brightness.

[0332] For the specific execution process of steps S1405 and S1406, please refer to Figure 3a Steps 304 and 305 of the illustrated embodiment will not be described in detail here.

[0333] After executing step S1406, if the difference between the facial brightness and the standard brightness is less than the preset value, the method ends; if the difference between the facial brightness and the standard brightness is not less than the preset value, step S1407 is executed.

[0334] In this embodiment, the difference between the facial brightness of the image frame 2 and the standard brightness is not less than the preset value, so step S1407 is executed.

[0335] S1407: Adjust exposure parameters.

[0336] In step S1407, the automatic exposure module adjusts the exposure parameter according to the difference between the face brightness and the standard brightness of image frame 2 to obtain an exposure parameter adjustment value. The specific execution process of step S1407 can refer to the above-mentioned exposure parameter adjustment process.

[0337] Similar to step S1402 , the exposure parameter adjustment value obtained in S1407 is sent to the camera driver, and the camera driver configures the camera to capture one or more image frames according to the exposure parameter adjustment value obtained in S1407 .

[0338] Exemplarily, when the exposure parameter adjustment value obtained in step S1407 is sent to the camera driver, the camera is ready to capture image frame 5, so the camera driver configures the camera to capture image frame 5 according to the exposure parameter adjustment value obtained in step S1407.

[0339] S1408: Face detection.

[0340] like Figure 14 As shown, starting from image frame 2, the automatic exposure module corrects the face frame of the previous image frame for each image frame obtained to obtain a corrected face frame. The corrected face frame is then used to crop the image frame to obtain a facial image of the image frame. The facial brightness of the image frame is then determined based on the facial image of the image frame. Thereafter, if the difference between the facial brightness of the image frame and the standard brightness is less than a preset value, the method ends. If the difference between the facial brightness of the image frame and the standard brightness is not less than the preset value, the exposure parameters are adjusted using the facial image of the image frame. After the exposure parameters are adjusted, face detection is performed on the image frame to obtain a face frame of the image frame. This process is repeated until the difference between the facial brightness of a certain image frame and the standard brightness is less than the preset value.

[0341] After comparing that the difference between the facial brightness of a certain image frame and the standard brightness is less than a preset value, the automatic exposure module can send the exposure parameters used by the camera when shooting this image frame to the camera driver, so that the camera driver configures the camera to continue shooting images according to the exposure parameters used when shooting this image frame.

[0342] After image frame N, if the automatic exposure module finds that the difference between the face brightness of a certain image frame and the standard brightness is not greater than the preset value, the automatic exposure module repeats the above process and continues to adjust the exposure parameters.

[0343] Figure 14 The image frame 1 shown includes but is not limited to the first frame of image captured after the camera is started.

[0344] In some possible embodiments, if a frame of image does not obtain a corresponding face frame after passing through the face detection module, the automatic exposure module may regard the first frame of image after this frame of image as image frame 1, and continue Figure 14 If a frame of image does not obtain a corresponding face frame after passing through the face detection module, the reason may be that the frame of image does not contain a face, or the face image is too blurry to be detected, or the face image is incomplete (for example, only the side face is captured).

[0345] For example, if the Xth frame image does not obtain the corresponding face frame after passing through the face detection module, the automatic exposure module can regard the subsequent X+1th frame image as image frame 1 and continue Figure 14 The process shown.

[0346] In some possible embodiments, the automatic exposure module ends Figure 14 After the process shown in FIG. 1 , if it is determined that the difference between the image brightness of a certain frame and the standard brightness is not less than the preset value, the frame can be determined as image frame 1 and the process is started again. Figure 14 The process shown.

[0347] The method for adjusting exposure parameters provided in this embodiment has the following beneficial effects:

[0348] By correcting the face frame of the previous frame, a corrected face frame that is closer to the face of the image in the current frame can be obtained, reducing the proportion of the background part in the cropped facial image data, improving the accuracy of the calculated facial brightness, and making the exposure parameter adjustment value determined based on the facial brightness more accurate.

[0349] It should be noted that the method for adjusting exposure parameters provided in Example 1 of the present application is applicable not only to the scenario where the electronic device detects that a user has been looking at the display screen for a long time, but also to other scenarios where exposure parameters need to be adjusted. This embodiment of the present application does not limit the scenario in which exposure parameters are adjusted.

[0350] An embodiment of the present application further provides a computer storage medium for storing a computer program. When the computer program is executed, it is specifically used to implement the exposure parameter adjustment method provided in any embodiment of the present application.

[0351] An embodiment of the present application also provides a computer program product, comprising a plurality of executable computer commands. When the computer commands of the product are executed, they are specifically used to implement the exposure parameter adjustment method provided in any embodiment of the present application.

Claims

1. A method for adjusting exposure parameters, characterized in that: include: Control display screen display; Obtaining a first frame of image and a first face frame, wherein the first face frame is used to indicate a face area in the first frame of image; obtaining a facial image of a second frame based on a second facial frame, wherein the second frame is a subsequent frame of the first frame; and the second facial frame is obtained by correcting the first facial frame; The exposure parameters are adjusted according to the facial image of the second frame image.

2. The exposure parameter adjustment method according to claim 1, wherein: The second frame image is cropped based on the second facial frame to obtain a facial image of the second frame image.

3. The adjustment method according to claim 2, characterized in that: The ratio of the background portion in the facial image of the second frame is smaller than the ratio of the background portion in the comparison facial image; the comparison facial image is an image obtained by cropping the second frame image based on the first facial frame.

4. The adjustment method according to claim 1, characterized in that: The size of the second face frame is smaller than the size of the first face frame.

5. The adjustment method according to claim 4, characterized in that: A ratio of a size of the second face frame to a size of the first face frame is a preset proportional coefficient.

6. The adjustment method according to claim 4 or 5, characterized in that: The position of the center point of the second face frame is the same as the position of the center point of the first face frame.

7. The adjustment method according to claim 5, characterized in that: The proportional coefficient is determined based on N frames of images; N is greater than or equal to 2.

8. The adjustment method according to claim 7, characterized in that: The N frames of images are N consecutive frames of images captured before the second frame of image.

9. The adjustment method according to claim 8, characterized in that: The amplitude of the posture change of the first frame image relative to each frame image in the N frames of images is less than or equal to a preset change amplitude threshold; the amplitude of the posture change of the first frame image relative to one frame image in the N frames of images is calculated based on the posture data of the camera device when shooting the first frame image and the posture data of the camera device when shooting one frame image in the N frames of images.

10. The adjustment method according to any one of claims 7 to 9, characterized in that: The scaling factor is determined according to the face frame of the N frames of images; the face frame of the N frames of images is used to indicate the area of ​​the face in the N frames of images.

11. The adjustment method according to claim 10, characterized in that: The scaling coefficient is determined according to an overlapping area of ​​face frames of the N frames of images and a face frame of any one of the N frames of images.

12. The adjustment method according to claim 11, characterized in that: The proportional coefficient is the ratio of the first size to the second size; the first size is the size of the overlapping area; and the second size is the size of the face frame of any one of the N frames of images.

13. The adjustment method according to claim 4, characterized in that: The second face frame is determined based on the first face frame and the face frame of the previous frame image; the face frame of the previous frame image is used to indicate the face area in the previous frame image; the previous frame image is the previous frame image of the first frame image.

14. The adjustment method according to claim 13, characterized in that: The amplitude of the posture change of the first frame image relative to the previous frame image is less than a preset change amplitude threshold; the amplitude of the posture change of the first frame image relative to the previous frame image is calculated based on the posture data of the camera device when taking the first frame image and the posture data of the camera device when taking the previous frame image.

15. The adjustment method according to claim 13 or 14, characterized in that: The second face frame is an overlapping area of ​​the first face frame and the face frame of the previous frame image.

16. The adjustment method according to any one of claims 1 to 3, characterized in that: The deviation between the position of the first face frame and the position of the second face frame matches a pre-acquired motion vector.

17. The adjustment method according to claim 16, characterized in that: The motion vector is determined based on the first frame image and a previous frame image; the previous frame image is a frame image previous to the first frame image.

18. The adjustment method according to claim 17, characterized in that: The amplitude of the posture change of the first frame image relative to the previous frame image is less than a preset change amplitude threshold; the amplitude of the posture change of the first frame image relative to the previous frame image is calculated based on the posture data of the camera device when taking the first frame image and the posture data of the camera device when taking the previous frame image.

19. The adjustment method according to claim 17 or 18, characterized in that: The motion vector is determined based on a query block and a target block; the query block is any image block obtained by dividing the previous frame image; the target block is an image block obtained by dividing the first frame image and matching the query block.

20. The adjustment method according to claim 19, characterized in that: The starting point of the motion vector is the center point of the query block; the end point of the motion vector is the center point of the target block.

21. The adjustment method according to claim 19, characterized in that: The query block is an image block located in the center of a face frame of the previous frame image among a plurality of image blocks obtained by dividing the previous frame image; the face frame of the previous frame image is used to indicate a facial area in the previous frame image.

22. The adjustment method according to claim 19, characterized in that: The target block is an image block among the multiple image blocks obtained by dividing the first frame image, and the deviation value between the target block and the query block is less than a preset deviation threshold; the deviation value between the image block and the query block represents the degree of difference between the image block and the query block.

23. The adjustment method according to claim 22, characterized in that: The deviation value is the mean absolute difference or the mean square error.

24. The adjustment method according to any one of claims 1 to 5, characterized in that: The adjusting the exposure parameters according to the facial image of the second frame image includes: determining the facial brightness of the second frame of image according to the facial image of the second frame of image; The exposure parameter is adjusted based on the facial brightness of the second frame image.

25. The adjustment method according to claim 24, characterized in that: The adjusting the exposure parameter based on the facial brightness of the second frame image includes: If the difference between the facial brightness of the second frame image and a preset standard brightness is outside a preset range, the exposure parameter is adjusted based on the difference between the facial brightness of the second frame image and the standard brightness.

26. The adjustment method according to any one of claims 1 to 5, characterized in that: After adjusting the exposure parameters according to the facial image of the second frame image, the method further includes: Performing face detection on the second frame image to obtain a face frame of the second frame image; the face frame of the second frame image is used to indicate a face area in the second frame image.

27. The adjustment method according to any one of claims 1 to 5, characterized in that: The facial area in the first frame image is different from the facial area in the second frame image.

28. The method for adjusting exposure parameters according to any one of claims 1 to 5, wherein: The step of adjusting the exposure parameters according to the facial image of the second frame image includes: The camera is controlled to capture a user's facial image using an exposure parameter adjustment value, where the exposure parameter adjustment value is obtained by adjusting the exposure parameter according to the facial image of the second frame.

29. The method for adjusting exposure parameters according to claim 28, wherein: After controlling the camera to shoot the user's face image with the exposure parameter adjustment value, the method further includes: Detecting that a person is looking at the display screen; Control electronic devices to keep the screen on.

30. The method for adjusting exposure parameters according to any one of claims 1 to 5, wherein: The method is applied to an electronic device, the electronic device including a front camera, the front camera being an AO (always on) camera. Before obtaining the first frame image and the first face frame, the method further includes: The front camera is controlled to operate in a low power consumption mode.

31. An electronic device, characterized in that: The electronic device includes: one or more processors, a memory, and a camera; The memory is used to store one or more programs; The one or more processors are used to execute the one or more programs, so that the electronic device executes the exposure parameter adjustment method as described in any one of claims 1 to 30, and is used to control the camera to capture images.

32. The electronic device according to claim 31, wherein The electronic device further includes a front camera, which is an AO (always on) camera. When the front camera operates in a low power consumption mode, the exposure parameter adjustment method according to any one of claims 1 to 30 is executed.

33. A computer storage medium, characterized in that Used to store a computer program, which, when executed, is specifically used to implement the exposure parameter adjustment method according to any one of claims 1 to 30.

Images