Automatic exposure adjustment method, electronic equipment and storage medium

Abstract

The invention discloses an automatic exposure adjustment method, electronic equipment and a storage medium, and relates to the technical field of electronics, in the method, the electronic equipment can obtain a first image, and if a face is recognized in the first image, background target brightness, face target brightness and a brightness weight are determined; and updating the brightness weight by using the position of the target face in the first image, and if the target face is close to the edge, determining that all faces in the first image are close to the edge. The closer the target face position is to the edge, the smaller the updated brightness weight is. And fusing the background target brightness and the face target brightness by using the updated brightness weight to obtain the updated background target brightness, so that the effect of the face target brightness on the updated background target brightness can be reduced, and the brightness fluctuation is reduced. When the exposure parameter is adjusted according to the current photometric brightness and the updated background target brightness, the brightness fluctuation caused by unstable edge face detection can be reduced, and the consistency of the preview brightness and the photographing brightness is improved.

Description

An automatic exposure adjustment method, electronic device and storage medium Technical Field

[0001] This application relates to the field of electronic technology, and in particular to an automatic exposure adjustment method, electronic device, and storage medium. Background Technology

[0002] When shooting in scenes with people, the algorithm identifies the face area (i.e., the human face). Currently, when a face is detected, the AE algorithm uses the face as the metering subject to adjust the exposure parameters, adjusting the brightness of the face area to a suitable level. However, when the face is located at the edge of the image, due to factors such as the person looking up or down, or environmental influences, the face detection algorithm cannot accurately identify the face. That is, sometimes it recognizes the face, and sometimes it does not, resulting in inconsistent brightness in the image, poor brightness consistency in the preview image, and poor brightness consistency in continuous shooting. Summary of the Invention

[0003] This application provides an automatic exposure adjustment method, electronic device, and storage medium that can improve the brightness consistency of preview images and the brightness consistency of images during continuous shooting when there are faces accidentally captured at the edge of the image, resulting in no obvious brightness differences between images. The technical solution is as follows:

[0004] In a first aspect, embodiments of this application provide an automatic exposure adjustment method applied to an electronic device. In this method, the electronic device can acquire a first image. When at least one face is detected in the first image, it determines the background target brightness, the face target brightness, and a brightness weight. Using the position of the target face in the first image, it updates the brightness weight, where the target face is the face closest to the center point of the first image among the at least one face. Using the updated brightness weight, it fuses the background target brightness and the face target brightness to obtain an updated background target brightness. Based on the brightness of each pixel in the first image, it determines the current metering brightness, i.e., the overall image metering brightness. Finally, it adjusts the exposure parameters based on the current metering brightness and the updated background target brightness.

[0005] In the above technical solution, when a face is detected, the electronic device can determine the location of the target face. The target face is the face closest to the center of the image. If the target face is close to the edge, then all faces in the first image are close to the edge; therefore, the target face is used as the benchmark. The closer the target face is to the edge, the smaller the updated brightness weight. When fusing background target brightness and face target brightness, the effect of face target brightness on the updated background target brightness can be reduced. Compared to the case where a face is detected but its location is not considered, the updated background target brightness is closer to the original background target brightness, thereby reducing the fluctuation of the updated background target brightness. This reduces brightness fluctuations caused by unstable edge face detection and improves the consistency between preview brightness and captured brightness.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, updating the brightness weights using the position of the target face in the first image includes: determining a distance weight using the position of the target face in the first image; and updating the brightness weights based on the distance weights. Specifically, the closer the target face is to the edge in the first image, the smaller the distance weight, and the smaller the updated brightness weight.

[0007] Combining the first aspect and the aforementioned implementation methods, in some implementations of the first aspect, the distance weight is determined using the position of the target face in the first image. This includes: determining the distance from the center point of the target face to the center point of the first image, and the distance from the center point of the first image to the origin; and determining the distance weight based on these distances. In this implementation, the ratio between the distance from the center point of the target face to the center point of the first image and the distance from the center point of the first image to the origin is determined. This ratio represents the face position; a larger ratio indicates that the face is closer to the image edge, and a smaller ratio indicates that the face is closer to the image center. The relationship between the preset distance ratio and the distance weight can be determined using a lookup table and linear interpolation methods.

[0008] Combining the first aspect and the aforementioned implementation methods, in some implementations of the first aspect, the distance weight is determined using the position of the target face in the first image. This includes: determining the distance from the center point of the target face to the center point of the first image, and the distance from the center point of the first image to the origin; determining the face contrast corresponding to the first image; and determining the distance weight based on the distance from the center point of the target face to the center point of the first image, the distance from the center point of the first image to the origin, and the face contrast. Since faces are darker in backlight, they usually need to be brightened. However, the distance weight for faces further out is smaller, reducing the brightening effect. If the distance weight from the center to the edge is reduced too much, the brightness of faces outside the center will be too low in backlight shooting. Therefore, this implementation sets different distance weight regions and selects the distance weight region based on the face contrast. This reduces the brightness attenuation of faces at the edges in backlight shooting, thereby reducing brightness fluctuations and preventing faces outside the center from being too low in brightness.

[0009] Combining the first aspect and the aforementioned implementation methods, in some implementation methods of the first aspect, determining the face contrast corresponding to the first image includes: determining the current face brightness corresponding to at least one face and the current background brightness of the first image; and determining the face contrast based on the current face brightness and the current background brightness of the first image. Specifically, the face contrast is the ratio of the current background brightness to the current face brightness, and a face contrast greater than a threshold indicates backlighting.

[0010] Combining the first aspect and the aforementioned implementation methods, in some implementations of the first aspect, determining the current face brightness corresponding to at least one face includes: when there is only one face, determining the face position weight based on the distance from the center point of the first image to the origin and the distance from the center point of the current face to the center point of the first image; and determining the current face brightness based on the face position weight and the average pixel brightness of the current face. By increasing the current face brightness through the face position weight, the contrast is relatively reduced, thus balancing and reducing brightness fluctuations and the attenuation of brightness at the edges of faces.

[0011] Combining the first aspect and the aforementioned implementation methods, in some implementations of the first aspect, determining the current face brightness corresponding to at least one face includes: when there are multiple faces, determining the face position weight corresponding to each face based on the distance from the center point of each face to the center of the first image; and determining the current face brightness based on the face position weight corresponding to each face and the average pixel brightness corresponding to each face. In this way, in multi-face scenarios, repeated changes in image brightness caused by differences in the brightness of multiple faces can be avoided.

[0012] Combining the first aspect and the above implementation methods, in some implementations of the first aspect, after determining the distance weight, the method further includes: determining the area of ​​the target face and determining a face area coefficient based on the area of ​​the target face; and correcting the distance weight based on the face area coefficient. Then, the brightness weight is updated using the distance weight, including: updating the brightness weight using the corrected distance weight. By correcting the position weight based on the area of ​​the target face, the position weight of a target face that is closer to the edge and has a smaller area is reduced, thus weakening the effect of the face target brightness.

[0013] Combining the first aspect and the aforementioned implementation methods, in some implementations of the first aspect, the method further includes: determining the orientation of the electronic device, including both landscape and portrait orientations; and shifting the initial image center point in a specified direction based on the orientation of the electronic device to obtain a first image center point. For most people, regardless of whether shooting in portrait or landscape mode, the usual shooting habit is to place the face slightly above the preview image. Therefore, in some implementations, if a face is detected, the image center can be shifted upwards to more accurately determine the position of the target face. Here, "upwards" refers to the user's perspective, and the actual shift is based on the coordinate system and the orientation of the electronic device.

[0014] In a second aspect, embodiments of this application provide an electronic device, including: one or more processors; one or more memories; the memories storing one or more programs, which, when executed by the processors, cause the electronic device to perform the method described in any of the first aspects above.

[0015] Thirdly, embodiments of this application provide an apparatus included in an electronic device, which has the function of implementing the behaviors of the electronic device in the above aspects and possible implementations thereof. The functions can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a display module or unit, a detection module or unit, a processing module or unit, etc.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described in any of the first aspects above.

[0017] Fifthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the method described in any of the first aspects above.

[0018] The technical effects achieved by the second, third, fourth, and fifth aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0019] Figure 1 shows a schematic diagram of a scenario provided by an embodiment of this application;

[0020] Figure 2 shows a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0021] Figure 3 shows a software structure block diagram of an electronic device provided in an embodiment of this application;

[0022] Figure 4 shows an interactive schematic diagram of the cooperation between the various software structures in Figure 3 to implement the method of this application;

[0023] Figure 5 shows a schematic diagram of a face location provided in an embodiment of this application;

[0024] Figure 6 shows another example of face location provided in the embodiments of this application;

[0025] Figure 7 shows a schematic diagram of an example shooting posture provided in an embodiment of this application;

[0026] Figure 8 shows a schematic diagram of the posture of an electronic device provided in an embodiment of this application;

[0027] Figure 9 shows a schematic diagram of the structure of a device provided in an embodiment of this application;

[0028] Figure 10 shows a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0030] To facilitate understanding, some terms involved in the embodiments of this application will be explained first.

[0031] 1. Metering: When the terminal performs exposure processing on the captured image, it needs to first meter the captured image to obtain the metered brightness. The metered brightness is used to indicate the brightness of the shooting scene, and then calculate the appropriate exposure parameters so that an image with appropriate brightness can be obtained when shooting.

[0032] 2. Ambient Light Value: Used to measure the brightness of a scene. It can be determined by shutter speed, aperture, ISO, and statistical information. The smaller the ambient light value, the darker the scene, and vice versa.

[0033] 3. Automatic Exposure (AE) Algorithm: The AE algorithm acquires the metered brightness and compares it with the target brightness, automatically adjusting exposure parameters (including ISO, aperture, and exposure time) to achieve an actual brightness close to the target brightness, ensuring the image captured by the camera is neither too dark nor too bright. "Close to the target brightness" generally means that the difference between the metered brightness and the target brightness is less than a preset threshold.

[0034] 4. Face Detection: Identifying faces in an image. In this embodiment, the AE algorithm adjusts the exposure based on the brightness characteristics of the subject to ensure that the subject is properly exposed.

[0035] 5. Metering weight: This refers to the relative importance of different areas or features in the overall metering calculation when calculating exposure. For example, in some metering modes, the camera may place more emphasis on the brightness of the image center or specific areas, thus affecting the overall exposure decision.

[0036] 6. Gravity of electronic devices: The changes in rotation angle of electronic devices on three mutually perpendicular coordinate axes, including pitch, yaw, and roll angles.

[0037] The following is a schematic diagram of a scenario involved in an embodiment of this application.

[0038] Electronic devices have multiple applications installed, such as file management, email, weather, calculator, contacts, phone, and camera. Users can launch the camera application. It is understood that there are multiple ways to launch the camera application. For example, it can be launched by touching the camera application icon, or by voice or swipe. For instance, when the electronic device is locked, the user can instruct the device to launch the camera application by swiping right on the screen. Alternatively, if the electronic device is locked and the lock screen includes a camera application icon, the user can instruct the device to launch the camera application by clicking the icon. Or, when the electronic device is running another application that has permission to access the camera application, the user can instruct the device to launch the camera application by clicking the corresponding control. For example, when the electronic device is running an instant messaging application, the user can instruct the device to launch the camera application by using the camera function controls. This application does not limit the specific method of launching the camera application.

[0039] After the electronic device launches the camera application, it can display the shooting interface shown in Figure 1. This shooting interface includes a preview box 11, shooting controls, shooting mode options, album shortcut controls, camera flip controls, and some shooting function controls, etc. Among them, the preview box 11 displays the image captured by the electronic device in real time through the camera.

[0040] When shooting in scenes with people, the algorithm identifies the face area (i.e., the face). Currently, when the face is detected, the AE algorithm adjusts the exposure parameters based on the face as the metering subject to adjust the brightness of the face area appropriately. When the face is located at the edge of the preview frame 11 (e.g., the area where box a is located in Figure 1, i.e., the image edge), if the face is sometimes obscured and sometimes revealed, or if the face is partially obscured, or if the person is making movements such as looking up, looking down, or turning their head, the face detection algorithm cannot accurately identify the face. That is, sometimes it recognizes the face and sometimes it does not. When the face is recognized, the preview image is brighter (as shown in Figure 1(a)) and when the face is not recognized, the preview image is darker (as shown in Figure 1(b)), resulting in inconsistent brightness in the preview. When the user takes multiple images in succession, it will also cause the brightness of the multiple images to be output consecutively to be inconsistent.

[0041] In view of this, this application provides an automatic exposure adjustment method that, when a face is detected, determines whether the face is at the edge, and then adjusts the influence of face brightness on exposure by adjusting the position weight. The position weight is set to be lower for faces that are further away from the edge, so as to achieve the purpose of brightness attenuation, thereby reducing the brightness fluctuation caused by unstable edge face detection and improving the consistency between preview brightness and photo brightness.

[0042] The automatic exposure adjustment method provided in this application can be applied to electronic devices with shooting functions, such as mobile phones, smart screens, tablets, wearable electronic devices, in-vehicle electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and projectors. This application does not limit the type of electronic device.

[0043] Figure 2 shows a schematic diagram of the hardware structure of an electronic device 100 applicable to this application.

[0044] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0045] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0046] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0047] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0048] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0049] For example, the processor 110 can execute the following method provided in the embodiments of this application: when shooting, if a face is recognized in the image, the position of the target face can be determined; the brightness weight is updated using the position of the target face, and the updated brightness weight is used to fuse the overall target brightness and the face target brightness to obtain the updated overall target brightness; the exposure parameters are adjusted based on the updated overall target brightness.

[0050] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0051] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device 100 to display images.

[0052] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0053] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0054] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0055] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0056] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0057] The ISP is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193. In this embodiment, the ISP includes an AE statistics module, which can count the AE data of the original image.

[0058] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0059] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0060] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0061] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0062] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate for based on the angle, and allows the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization. The distance sensor 180F is used to measure distance. In some embodiments, during shooting, the electronic device 100 can use the distance sensor 180F to measure distance for fast focusing. The ambient light sensor 180L can be used to automatically adjust the white balance during shooting.

[0063] The accelerometer 180E can detect the magnitude of acceleration of the electronic device 100 in various directions (typically three axes). When the electronic device 100 is stationary, it can detect the magnitude and direction of gravity. The accelerometer 180E can also be used to identify the posture of the electronic device and determine whether the electronic device is in landscape or portrait mode.

[0064] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0065] The hardware structure of electronic device 100 has been described in detail above. The software system of electronic device 100 will be introduced below.

[0066] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0067] Figure 3 is a schematic diagram of the software system of the electronic device 100 according to an embodiment of the present invention.

[0068] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the software system may include an application layer, an application framework layer, a hardware abstraction layer (HAL), a driver layer, and a hardware layer.

[0069] The application layer can include a series of application packages. As shown in Figure 3, application packages can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0070] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0071] For example, the application framework layer may include a camera access interface; the camera access interface can provide an access interface for managing the camera, and can also provide an interface for accessing the camera.

[0072] The Hardware Abstraction Layer (HAL) is used to abstract hardware. By calling the HAL interface, connections can be established between the application layer and application framework layer above the HAL and the driver layer and hardware layer below, enabling camera data transmission and function control.

[0073] For example, the hardware abstraction layer may include the camera hardware abstraction layer, the camera algorithm library, and other hardware device abstraction layers;

[0074] The camera hardware abstraction layer can include an AE control module and a face detection module. The face detection module can identify whether there are faces in the scene, and when faces are present, it outputs information such as the face location and number of faces.

[0075] The AE control module includes the AE algorithm. Based on the face recognition results and AE statistics, the AE control module can adjust the exposure parameters so that there is no obvious brightness difference in the preview and the image when there is a face with edges.

[0076] The driver layer is used to provide drivers for different hardware devices. For example, the driver layer may include camera drivers, image signal processor drivers, display drivers, and sensor drivers.

[0077] The hardware layer may include displays, cameras, image signal processors (ISPs), accelerometers, and other hardware devices.

[0078] The camera includes an image sensor, which outputs the raw image. The accelerometer is used to acquire attitude information of the electronic device to determine its orientation.

[0079] An image signal processor (ISP) may include an AE statistics module. The AE statistics module can analyze AE data. In addition, the ISP can also analyze autofocus (AF) data and auto-white balance (AWB) data, among others.

[0080] Specifically, after the user activates the camera, the camera requests a preview image and sends this request to the camera hardware abstraction layer (HAL) via the camera access interface. The HAL then sends this request to the camera via the underlying driver. The camera acquires the raw image and sends it to the ISP (Internet Service Provider). Upon receiving the raw image, the ISP processes it to generate a preview image, which is then uploaded to the camera application. The camera application can then call the display driver to display the preview image.

[0081] On the other hand, the AE statistics module in the ISP performs AE data statistics and sends AE data to the AE control module; the ISP also sends the processed raw image to the camera hardware abstraction layer. After the camera hardware abstraction layer receives a frame of raw image, the face detection module performs face detection based on the frame of raw image and outputs the face detection result to the AE control module.

[0082] Upon face detection, the AE control module calculates the current metering luminance (curLuma) and target luminance (targetLuma) based on the face location and AE data statistics. If the difference between the target luminance and the metering luminance is within a preset range, they are considered close. If the difference is outside the preset range, the AE control module reconfigures the exposure parameters based on the current exposure of the original image and then sends these parameters to the camera. The camera acquires a new original image, and the above process continues, adjusting the exposure parameters until convergence occurs, meaning the target luminance and metering luminance are close.

[0083] The automatic exposure adjustment method provided in the embodiments of this application will be described in detail below.

[0084] After acquiring the original image, the system identifies faces and confirms their positions. Based on the face positions, it determines the current metering brightness and target brightness, and then adjusts the exposure parameters. Figure 4 is a flowchart illustrating an automatic exposure adjustment method provided in an embodiment of this application. This method is applied in an electronic device, which implements it through the interaction between the various modules shown in Figure 3.

[0085] S401, the camera application receives a request to launch the camera application.

[0086] Understandably, a user can trigger an electronic device to launch the camera app. Correspondingly, the camera app on the electronic device can receive this request to launch the camera app.

[0087] S402, the camera application sends a request to the camera to obtain a preview image.

[0088] Understandably, the camera can be activated after receiving a request to acquire a preview image. Furthermore, once activated, the camera can continuously acquire initial image data.

[0089] S403, the camera sends the first image data to the ISP.

[0090] After receiving the first image data, the ISP can perform 3A data statistics. Among them, the AE statistics module is responsible for the statistics of AE data.

[0091] S404, the AE statistics module, compiles AE data.

[0092] AE data can include brightness statistics and brightness histograms, such as the brightness of each pixel.

[0093] S405, the ISP sends the first image data to the camera hardware abstraction layer.

[0094] The first image data includes AE data and other data, such as timestamps. This first image data is sent to the AE control module and face detection module in the hardware abstraction layer. The ISP also sends a preview image generated from the first image to the camera, allowing the camera to display this preview image.

[0095] S406, the face detection module performs face recognition on the first image.

[0096] In this embodiment of the application, no specific limitation is made on the face recognition algorithm.

[0097] S407, the face detection module sends the recognition result after face recognition to the AE control module. Correspondingly, the AE control module receives the recognition result.

[0098] In this embodiment, the recognition result after face recognition can be represented by fields. For example, field "1" can be used to indicate that a face has been recognized, and field "0" can be used to indicate that no face has been recognized. If a face is recognized, the face detection module can also identify the number of faces and the location information of each face. Then, when sending the recognition result to the AE control module, it also sends the number of faces and the location information of each face to the AE control module.

[0099] In some implementations, the position information of a face can be represented by the position information of a face bounding box. For example, as shown in Figure 5, with the top left corner pixel of the first image as the origin O, the horizontal distance w from the top left corner of the face bounding box 51 to the origin O, the vertical distance h from the top left corner of the face bounding box 51 to the origin O, the width x (i.e., occupying x pixels horizontally), and the height y (i.e., occupying y pixels vertically), then the position information of the face can be represented as (w, h, x, y).

[0100] The subject detection module can send the face recognition result to the AE control module. For example, when a face is recognized, the face recognition result can be represented as "result:1; face1,(w,h,x,y); face2,(w,h,x,y)". When no face is recognized, the face recognition result can be represented as "result:0".

[0101] After receiving the recognition result, the AE control module determines whether a face has been recognized based on the content included in the result and executes different processing procedures. When the recognition result indicates that no face has been recognized, for example, the recognition result is "result:0", the AE control module executes the corresponding processing procedure for non-face scenes and ends the current process. When the recognition result indicates that a face has been recognized, for example, the recognition result is "result:1; face1,(w,h,x,y); face2,(w,h,x,y)", the AE control module can determine that a face has been recognized, and can determine the number of faces and the position of each face from the recognition result. Furthermore, it can determine the distance weight based on the position of the target face, and determine the final target brightness when a face is present based on the distance weight.

[0102] Regarding the target brightness, whether it is a single face or multiple faces, there is only one target brightness. In this embodiment, the target brightness of the portrait scene is obtained by weighting the background target brightness and the face target brightness, as shown in the following formula (1).

[0103] targetLuma=sceneTarget*(1-faceWeight)+faceTarget*faceWeight (1)

[0104] Where, targetLuma represents the final target brightness, i.e., the updated background target brightness. sceneTarget represents the background target brightness before the update, faceTarget represents the face target brightness, and faceWeight represents the brightness weight applied to the background target brightness and the face target brightness.

[0105] There is a correlation between ambient brightness and the brightness of the face target. After acquiring the first image, the AE control module can determine the ambient brightness using shutter speed, aperture, ISO, and statistical information. Once the ambient brightness is determined, the brightness of the face target can be determined by looking up a table based on the ambient brightness. There is also a correlation between ambient brightness and background target brightness; the background target brightness can be determined by looking up a table based on the ambient brightness.

[0106] After determining the ambient brightness, the AE control module can also calculate an initial brightness weight, denoted as faceWeight', based on the ambient brightness. In this embodiment, the distance weight distWeight is determined based on the face position, and the distance weight is applied to the initial brightness weight faceWeight'. The brightness weight faceWeight related to the face position is shown in equation (2) below. That is, the brightness weight is updated using the distance weight to obtain the updated brightness weight.

[0107] faceWeight=faceWeight'*distWeight (2)

[0108] The purpose of the distance weighting in this application is to reduce the brightness weight by decreasing the distance weight when the face is near the edge, thereby reducing the impact of the face target brightness on the updated background target brightness. The AE control module can determine the distance weight (distWeight) in different ways, which will be explained in detail below.

[0109] In the first implementation, the distance weight is determined only based on the distance ratio. Step S409 is executed after step S408, and steps S415-S421 are executed after step S409.

[0110] S408, if a face is detected, the AE control module determines the ratio between the distance from the center point of the target face to the center point of the image and the distance from the center point of the image to the origin.

[0111] Regarding the target face, this embodiment distinguishes between single-face and multi-face cases. The AE control module can determine whether the number of faces is greater than 1 based on the recognition result. If the number of faces is equal to 1, that is, only one face is recognized in the first image, the current face is taken as the target face. If the number of faces is greater than 1, that is, multiple faces are recognized in the first image, the face closest to the center of the image is taken as the target face.

[0112] Referring to Figure 6(a), an image pixel coordinate system is established in the first image. The image origin can be located at the upper right corner of the image, or the upper left corner, etc. The coordinates of the image origin O can be denoted as (x0, y0), and the coordinates of the image center point P can be denoted as P(x1, y1). The face center point can be represented by the center point of the face bounding box. The AE control module can determine the face center point based on the face position information in the recognition result. The coordinates of the face center point in the pixel coordinate system can be denoted as (x2, y2). The distance from the face center point to the image center point is represented by cur_dist.

[0113] So, when the number of faces is equal to 1, the distance from the center point of the target face to the center point of the image is cur_dist. target It can be represented by Euclidean distance, as shown in equation (3) below.

[0114] cur_dist target =(x0-x2) 2 +(y0-y2) 2 (3)

[0115] The distance maxdist from the center point of the image to the origin can also be represented by Euclidean distance, as shown in equation (4) below.

[0116] maxdist = (x0 - x1) 2 +(y0-y1) 2 (4)

[0117] Referring to Figure 6(b), when the number of faces is greater than 1, the AE control module can determine the distance from the center point of each face to the center point of the image, obtaining multiple distance values, such as cur_dist[1], cur_dist[2], cur_dist[3], and cur_dist[4]. From these multiple distance values, the minimum distance min_dist (e.g., cur_dist[2]) is determined, which is the distance from the center point of the face closest to the center of the image to the center point of the image. At this time, min_dist is used as the distance from the center point of the target face to the center point of the image, cur_dist. targetAlternatively, the AE control module can determine the coordinates of the center point of each face, compare the coordinates of the center points of each face to determine the coordinates of the target face, and calculate the distance cur_dist from the center point of the target face to the center point of the image based on the Euclidean distance formula. target .

[0118] Furthermore, the distance cur_dist from the center point of the target face to the center point of the image can be determined. target The ratio distRatio between the distance maxdist from the center point of the image to the origin is shown in equation (5) below.

[0119]

[0120] The ratio between the distance from the center point of the target face to the center point of the image and the distance from the center point of the image to the origin is referred to as the distance ratio. In this embodiment, the distance ratio distRatio represents the face position. A larger distance ratio indicates that the face is closer to the image edge, and a smaller distance ratio indicates that the face is closer to the image center.

[0121] S409, the AE control module determines the distance weight distWeight based on the ratio between the distance from the center of the target face to the center of the image and the distance from the center of the image to the origin.

[0122] The distance ratio is divided into multiple regions, and the interval of each region can be adjusted. This application does not limit the specific number of regions. The appropriate smooth interval can be determined through training with a large amount of data. For example, the distance ratio is divided into 10 regions, as shown in the following formula (6). The distance ratio is 0.1 or below as one region, 0.1 to 0.2 as one region, 0.2 to 0.3 as one region, and so on.

[0123] distRatioZone[9]={0.1,0.2, 0.3, 0.4, 0.5, 0.6,0.7,0.8,0.9} (6)

[0124] For each distance ratio zone (distRatioZone), a corresponding distance weight zone (distWeightZone) is set as shown in equation (7). The distance weight (distWeight) can then be mapped using a lookup table based on the distance ratio. The distance weight decreases as the distance ratio increases. A distance weight of 1 corresponds to a distance ratio of 0.1, 0.2, 0.3, and 0.9, respectively. The distance weight zone corresponding to distance ratio zones 0.1–0.2 is 1; the distance weight zone corresponding to distance ratio zones 0.4–0.5 is 1–0.9; the distance weight zone corresponding to distance ratio zones 0.5–0.6 is 0.9–0.8, and so on.

[0125] distWeightZone[9]={1, 1, 1, 1,0.9, 0.8, 0.7,0.6,0.5} (7)

[0126] After receiving the recognition result, if a face is recognized, the AE control module can first determine the distance ratio corresponding to the first image. Based on the distance ratio region and the distance weight region mentioned above, the distance ratio is linearly interpolated to determine the corresponding distance weight distWeight, as shown in equation (8). Then, based on equation (2) above, the brightness weight is updated using the distance weight to obtain the updated brightness weight. Based on equation (1) above, the updated brightness weight is used to fuse the background target brightness and the face target brightness to obtain the updated background target brightness.

[0127]

[0128] Where distRatio represents the distance ratio of the current target face, and i represents the region corresponding to the current distance ratio. For example, distRatioZone[1] = {0.1, 0.2}, i ∈ [0, 9].

[0129] In summary, when no face is detected, the background target brightness (sceneTarget) is the final target brightness. When a face is detected, the smaller the distance ratio (distRatio), meaning the closer the target face is to the image center, the greater the distance weight; conversely, the larger the distance ratio (distRatio), meaning the closer the target face is to the image edge, the smaller the distance weight. Thus, when determining the updated background target brightness (targetLuma), if the target face is at the image edge, the influence of the face target brightness on the updated background target brightness (targetLuma) can be reduced, making the updated background target brightness (targetLuma) closer to the original background target brightness (sceneTarget), thereby reducing fluctuations in the final target brightness. If the target face is close to the center, the position weight within a certain range around the center point is all 1, which also ensures appropriate face brightness.

[0130] In the second implementation, considering the backlight shooting scene, the distance weight is determined based on the distance ratio and face contrast. Steps S410-S412 are executed after step S408, and steps S415-S421 are executed after step S412.

[0131] S410, determine the current face brightness corresponding to at least one face and the current background brightness of the first image.

[0132] Specifically, this application embodiment distinguishes between single-face and multi-face scenarios. When the number of faces is equal to 1, the current face brightness is determined according to S4101-S4102. When the number of faces is greater than 1, the current face brightness is determined according to S4105-S4105.

[0133] S4101, when the number of faces is 1, the AE control module determines the face position weight facePosWeight based on the distance max_dist from the center point of the image to the origin of the image and the distance cur_dist from the current center point of the face to the center point of the image, as shown in the following formula (9).

[0134]

[0135] S4102, the AE control module determines the current face brightness based on the average pixel brightness of the face bounding box and the face position weight facePosWeight, as shown in equation (10) below.

[0136] curfaceLuma=faceLuma*facePosWeight (10)

[0137] Here, `curfaceLuma` represents the current face brightness, and `faceLuma` represents the average pixel brightness of the face bounding box. It can be understood that the face bounding box contains multiple pixels, each corresponding to a brightness value, and the average pixel brightness of the face bounding box is the average brightness of these multiple pixels. The AE control module can determine the average pixel brightness of the face bounding box based on brightness statistics.

[0138] S4103, when the number of faces is greater than 1, the AE control module determines the face position weight corresponding to each face based on the distance from the center point of each face to the center point of the image.

[0139] Specifically, the minimum distance min_dist is determined from multiple distance values. The face position weight corresponding to each face is determined based on the ratio of the distance from the center point of each face to the center point of the image to the minimum distance min_dist, as shown in the following formula (11).

[0140]

[0141] Where cur_dist[k] represents the distance from the center point of the k-th face to the center point of the image, and facePosWeight[k] represents the face position weight corresponding to the k-th face, where k is greater than 1 and is a positive integer.

[0142] S4104, the AE control module determines the current face brightness based on the face position weight corresponding to each face and the average pixel brightness of each face.

[0143] When the number of faces is greater than 1, the brightness of the current face in the first image is obtained by weighting the brightness of multiple faces, as shown in equation (12). In this way, in a multi-face scenario, the image brightness can be avoided due to the different brightness of multiple faces.

[0144]

[0145] Where curfaceLuma represents the current face brightness, and faceLuma[k] represents the average pixel brightness of the k-th face.

[0146] In some implementations, face position weights can be omitted.

[0147] S411, determine the face contrast based on the current face brightness and the current background brightness of the first image.

[0148] Considering the possibility of backlighting during shooting, this embodiment also sets a parameter for face contrast, which indicates whether the current shooting is backlit or frontlit. When the face contrast is greater than a threshold, it is considered backlit shooting, and the higher the face contrast, the darker the captured face. The face contrast corresponding to the first image can be determined based on the current background brightness and the current face brightness of the first image, as shown in the following formula (13).

[0149]

[0150] Here, `faceContrast` represents the face contrast, `Luma` represents the current background brightness, and `curfaceLuma` represents the current face brightness. The current background brightness is the average brightness of all pixels in the first image.

[0151] S412, the AE control module determines the distance weight distWeight based on the face contrast and the ratio between the distance from the center of the target face to the center of the image and the distance from the center of the image to the origin.

[0152] The face contrast can be divided into multiple regions. The specific number of regions and the range of each region can be adjusted, and this application does not limit this. For example, the face contrast can be divided into 8 regions, as shown in the following formula (14).

[0153] faceContrastZone[7]={10, 50, 80,100,200, 800,1000} (14)

[0154] For each face contrast zone (faceContrastZone), a different distance weight zone (distWeightZone) is set. Therefore, based on the distance ratio and face contrast, the distance weight table can be expanded into two dimensions, as shown in equation (15) below, where the row direction corresponds to the face contrast and the column direction corresponds to the distance ratio. When the contrast is 10, the distance weight zone (distWeightZone) in the first row is selected for calculation. When the contrast is 50, the distance weight zone (distWeightZone) in the second row is selected for calculation, and so on.

[0155] distWeightZone[7][9]={1, 1, 1, 0.9, 0.8, 0.7,0.6,0.5,0.3} (15)

[0156] ={1,1,1,0.9,0.8,0.7,0.6,0.5,0.35}

[0157] ={1,1,1,0.9,0.8,0.7,0.6,0.5,0.4}

[0158] ={1,1,1,1,0.9,0.8,0.7,0.6,0.5}

[0159] ={1,1,1,1,0.9,0.8,0.7,0.6,0.6}

[0160] ={1,1,1,1,0.9,0.8,0.8,0.7,0.6}

[0161] ={1,1,1,1,1,0.9,0.8,0.6,0.7}

[0162] After determining the face contrast and distance ratio corresponding to the first image, the distance weight region is obtained by looking up the face contrast. Then, based on the distance weight region and the aforementioned distance ratio region, the distance ratio corresponding to the first image is linearly interpolated to determine the distance weight distWeight corresponding to the first image, as shown in equation (8) above. Then, based on equation (2) above, the brightness weight is updated using the distance weight to obtain the updated brightness weight. Based on equation (1) above, the updated brightness weight is used to fuse the background target brightness and the face target brightness to obtain the updated background target brightness.

[0163] When a face is detected, the closer the target face is to the image center, the greater the distance weight; the closer it is to the image edge, the smaller the distance weight, meaning the distance weight gradually decreases from the center to the edge. This way, when determining the updated background target brightness, if the face is located at the edge, the influence of the face target brightness on the updated background target brightness can be reduced, making the updated background target brightness closer to the original brightness and thus reducing brightness fluctuations.

[0164] Because faces are relatively dark in backlit conditions, they typically need to be brightened. However, the distance weight for faces further out is smaller, reducing the brightening effect. If the distance weight from the center to the edge is reduced too much, the brightness of off-center faces will be too low in backlit shots. Therefore, in this embodiment, different distance weight regions are set. The AE control module selects the distance weight region based on face contrast, reducing the brightness attenuation of edge faces in backlit shots. This reduces brightness fluctuations while preventing off-center faces from being too low in brightness.

[0165] In the third implementation, considering the face area, the position weight is corrected using a face area coefficient, based on the first two implementations.

[0166] S413, the AE control module determines the area of ​​the target face and determines the face area coefficient based on the area of ​​the target face.

[0167] S414, the AE control module uses the face area coefficient to correct the distance weight distWeight.

[0168] If the target face is off-center from the image and its area is greater than or equal to a preset threshold, then the target face is likely the focus of the shot. If the target face's area is less than the preset threshold, the target face is considered to be a face that has mistakenly entered the shooting area, and the distance weight should be reduced. Based on this, the AE control module can use a face area coefficient to correct the distance weight distWeight.

[0169] Specifically, a face area ratio is set, which is the ratio of the area of ​​the target face to the area of ​​the entire image. The face area ratio is divided into multiple regions. The specific number of regions and the interval of each region can be adjusted, and this application does not limit this. For example, the face area ratio is divided into 8 regions, as shown in the following formula (16).

[0170] facePercZone[7]={0.005,0.008,0.01,0.02,0.03,0.04,0.05} (16)

[0171] For each face area region facePercZone, a face area coefficient region facePercRatioZone is divided as shown in the following formula (17). When the face area ratio is 0.005, the face area coefficient is 0.5; when the face area ratio is 0.008, the face area coefficient is 0.6, and so on.

[0172] facePercRatioZone[7]={0.5,0.6,0.7,0.8,0.9,1,1} (17)

[0173] After determining the distance weight distWeight, the AE control module can determine the face area ratio of the target face. Based on the area ratio region and the face area coefficient region, the current face area coefficient facePercRatio is obtained through linear interpolation. Then, the distance weight is corrected using the current face area coefficient, as shown in equation (18).

[0174] distWeight ＇ =distWeight*facePercRatio (18)

[0175] Where facePercRatio represents the current face area coefficient, and distWeight ＇This represents the corrected position weight. distWeight is the position weight calculated based on the distance ratio distRatio and the distance weight zone distWeightZone in the two implementation methods mentioned above.

[0176] Then, the corrected position weight distWeight ＇ Applying the initial brightness weight faceWeight', we obtain the brightness weight faceWeight related to the face position, as shown in equation (19) below. That is, the brightness weight is updated using the corrected distance weight. Then, based on the above equation (1), the updated brightness weight is used to fuse the background target brightness and the face target brightness to obtain the updated background target brightness.

[0177] faceWeight=faceWeight'*distWeight' (19)

[0178] In summary, even faces located at the edge can be a focus of the shot. By adjusting the position weight based on the area of ​​the target face, the closer a target face is to the edge and the smaller its area, the smaller its position weight will be, and thus the effect of the face target brightness will be weaker.

[0179] The above method uses the position of the target face in the first image to determine the distance weights, taking into account different shooting scenarios, making the calculated distance weights more accurate.

[0180] S415 determines the updated background target brightness based on distance weights.

[0181] After determining the distance weight, the brightness weight is updated using the distance weight; the updated brightness weight is then used to fuse the background target brightness and the face target brightness to obtain the updated background target brightness.

[0182] S416, determine the current metering brightness based on the brightness of each pixel in the first image.

[0183] The ISP can divide the first image into multiple blocks, each containing multiple pixels. For example, if the first image is 640*480 pixels, dividing it into blocks reduces its size to 64*48 pixels. Each block can be considered a single pixel, and the brightness of a block can be the average brightness of all pixels within that block. Furthermore, the AE control module can determine the grayscale value corresponding to each block based on the AE data of the first image.

[0184] A general metering weight table is pre-stored. The general metering weight of the central pixel of the image is high, and the general metering weight of the surrounding pixels is low. The general metering weight table can be 16*16 in size or expanded to 64*48. For example, the general metering weight of the central pixel is 100, and it decreases to 90, 80, 70 and so on towards the surrounding pixels.

[0185] The AE control module can obtain the general metering weight of each block from the general metering weight table. The current metering brightness is determined according to the general metering weight, as shown in equation (20) below.

[0186]

[0187] curLuma is the current metering brightness, k represents the kth block, meteringWeight[k] is the general metering weight of the kth block, blockLuma[k] is the grayscale value of the kth block, and M represents the total number of blocks, i.e., k∈[0,M].

[0188] In summary, the AE control module obtains the current metering brightness and the target brightness, and then adjusts the exposure parameters based on the current metering brightness and the target brightness.

[0189] The S417 AE control module adjusts the exposure parameters based on the current metering brightness and the updated background target brightness.

[0190] The AE control module first determines the target exposure based on the current light intensity and the target brightness, as shown in the following formula (21).

[0191]

[0192] Where targetExpoValue is the target exposure value to be adjusted in subsequent images, curExpoValue is the exposure value corresponding to the statistics of the first image, targetLuma is the background target brightness, and curLuma is the current metering brightness. The target exposure value corresponds to the background target brightness.

[0193] After determining the target exposure, the exposure to be adjusted next can be determined based on the current exposure of the first image, as shown in equation (22).

[0194] nextExpoValue=lastExpoValue + (targetExpoValue - lastExpoValue) *step (22)

[0195] Where nextExpoValue is the exposure value to be adjusted next, lastExpoValue is the exposure value of the previous frame (i.e., the first image), and step is the adjustment step size.

[0196] Then, the AE control module reconfigures the exposure parameters according to the exposure amount to be adjusted next time, and sends the new exposure parameters to the camera, so that the camera can acquire a new image.

[0197] S418, the AE control module sends reconfigured exposure parameters to the camera.

[0198] S419: The camera acquires a second image based on the reconfigured exposure parameters.

[0199] S420: The camera sends a second image to the ISP.

[0200] S421, the ISP sends a preview image of the second image to the camera. The camera then displays the preview image of the second image.

[0201] When the target face is close to the edge of the image, due to the smaller distance weight, the updated background target brightness is smaller compared to the case where the face is recognized but its position is not considered. As a result, the target exposure is smaller, and the exposure to be adjusted next time is smaller. That is, the exposure of the first image and the exposure of the second image to be adjusted next time are close. The brightness difference between the first image and the second image obtained before and after adjusting the exposure is small. Therefore, the method provided in this application embodiment can reduce the brightness fluctuation caused by unstable face detection and improve the consistency of preview brightness.

[0202] When taking photos, users can issue multiple shooting commands in quick succession, such as rapidly clicking the shooting control multiple times, thereby capturing multiple images in succession. Since the captured images are identical to the preview images, the brightness consistency of the multiple images captured in succession is good, with no obvious brightness differences.

[0203] The aforementioned image center point is the center point of the first image. For most people, whether shooting in portrait or landscape mode, the usual shooting habit is to place the face slightly above the preview image, as shown in Figure 7. Therefore, in some implementations, if a face is detected, the image center can be shifted upwards. The following steps can be performed between steps S407 and S408:

[0204] (1) Acquire attitude information of the electronic device. The AE control module can acquire attitude information from the G-sensor.

[0205] (2) Calculate the rotation angle of the electronic device based on the attitude information, as shown in the following formula (23).

[0206] roll= -factor * acos(Xg / sqrt(Xg * Xg + Yg * Yg)) (23)

[0207] In this system, the electronic device is positioned in a three-dimensional coordinate system, with the X-axis representing width and the Y-axis representing height. `roll` is the rotation angle around the X-axis. The `acos` function calculates the cosine of the angle between the electronic device and the positive X-axis, and `factor` is a constant (180 / π) used to convert radians to degrees. `(Xg, Yg)` represents the coordinates of a point on the electronic device.

[0208] (3) Determine the coordinates of the offset image center point based on the rotation angle of the electronic device.

[0209] Based on the rotation angle, the orientation of the electronic device can be determined as landscape or portrait. Specifically, the roll angle is normalized to four directions of the electronic device, each direction corresponding to a posture. (-45, 45) represents a 0° rotation, (-135, -45) represents a 90° rotation, (45, 135) represents a 270° rotation, and other cases represent a 180° rotation. For example, as shown in Figure 8, postures at 0° and 180° can be considered landscape, and postures at 90° and 270° can be considered portrait. Combining the landscape and portrait orientations, the original image center point can be shifted a certain distance in the width or height direction to determine the coordinates of the shifted image center point.

[0210] For example, the top left corner of the electronic device when rotated 0° is taken as the origin of the coordinate system. The coordinates of the original image center point can be denoted as (0.5*width, 0.5*height). At 0°, a certain distance can be shifted in the negative Y-axis direction; for example, the coordinates of the shifted image center point could be (0.5*width, 0.3*height). At 90°, a certain distance can be shifted in the negative X-axis direction; for example, the coordinates of the shifted image center point could be (0.3*width, 0.5*height). At 180°, a certain distance can be shifted in the positive Y-axis direction; for example, the coordinates of the shifted image center point could be (0.5*width, 0.7*height). At 270°, a certain distance can be shifted in the positive X-axis direction; for example, the coordinates of the shifted image center point could be (0.7*width, 0.5*height). It should be understood that the above is only an illustrative example; the orientations at 0° and 180° can also be used as portrait orientation, and the orientations at 90° and 270° as landscape orientation. For a specific offset value, a suitable offset distance can be determined through training with a large amount of data.

[0211] In the embodiment shown in Figure 4 above, the image center point can be either the center point before or after the offset. By offsetting the center point, the target face can be avoided from being misidentified as an edge face, thus reducing the positional weight of the target face. Since the positional weight within a certain range near the center point is all 1, the accuracy of the positional weight after the offset can be guaranteed.

[0212] In summary, the embodiments of this application provide an automatic exposure adjustment method. For faces at the edge, the method adjusts the brightness of the face target based on the face position to reduce the brightness fluctuations caused by unstable face detection and improve the consistency between the preview brightness and the brightness of the photograph.

[0213] Figure 9 is a schematic diagram of an automatic exposure adjustment device provided in an embodiment of this application. This exposure adjustment device can be an electronic device as described in this application embodiment, or a chip or chip system within an electronic device. As shown in Figure 9, the automatic exposure adjustment device 900 may include a processing unit 901. The processing unit 901 is used to support the automatic exposure adjustment device 900 in performing the aforementioned processing steps.

[0214] In one implementation, the automatic exposure adjustment device 900 further includes a display unit 902. The display unit 902 is used to display a preview image.

[0215] In one implementation, the automatic exposure adjustment device 900 further includes a storage unit 903. The storage unit 903 and the processing unit 901 are connected via a circuit. The storage unit 903 may include one or more memories, which can be devices in one or more devices or circuits used to store programs or data. The storage unit 903 can exist independently and be connected to the processing unit 901 via a communication bus. Alternatively, the storage unit 903 can be integrated with the processing unit 901.

[0216] Storage unit 903 may store computer-executable instructions for methods in an electronic device, so that processing unit 901 executes the methods in the above embodiments. Storage unit 903 may be a register, cache, or random access memory (RAM), or it may be read-only memory (ROM) or other types of static storage devices that can store static information and instructions.

[0217] Figure 10 is a schematic diagram of the structure of a chip provided in an embodiment of this application. As shown in Figure 10, the chip 1000 includes one or more processors 1001, a communication line 1002 and a communication interface 1003. Optionally, the chip 1500 also includes a memory 1004.

[0218] In some implementations, memory 1004 stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof.

[0219] The methods described in the embodiments of this application can be applied to or implemented by the processor 1001. The processor 1001 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 1001 or by instructions in the form of software. The processor 1001 may be a general-purpose processor (e.g., a microprocessor or conventional processor), a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates, transistor logic devices, or discrete hardware components. The processor 1001 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0220] The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in mature storage media in the art, such as random access memory, read-only memory, programmable read-only memory, or electrically erasable programmable read-only memory (EEPROM). This storage medium is located in memory 1004, and processor 1001 reads information from memory 1004 and, in conjunction with its hardware, completes the steps of the above method.

[0221] The processor 1001, memory 1004 and communication interface 1003 can communicate with each other via communication line 1002.

[0222] In the above embodiments, the instructions stored in the memory for execution by the processor can be implemented in the form of a computer program product. This computer program product can be pre-written into the memory, or it can be downloaded and installed into the memory as software.

[0223] This application also provides a computer program product comprising one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. For example, available media may include magnetic media (e.g., floppy disks, hard disks, or magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0224] This application provides an electronic device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to perform the method described above.

[0225] This application provides a chip. The chip includes a processor, which is used to call a computer program in memory to execute the technical solutions in the above embodiments. Its implementation principle and technical effects are similar to those in the related embodiments described above, and will not be repeated here.

[0226] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms. Additionally, the functional units in the various embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0227] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above. The computer-readable storage medium stores a computer program or instructions. When executed by a processor, the computer program or instructions implement the methods described above. The methods described in the above embodiments can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted over the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0228] As one possible design, computer-readable media may include compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM, or other optical disc storage; computer-readable media may include disk storage or other disk storage devices. Furthermore, any connecting cable may also be appropriately referred to as computer-readable media. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. As used herein, disks and optical discs include optical discs (CD), laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0229] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0230] In the foregoing description, specific details such as particular system architectures and techniques have been set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted to avoid unnecessary detail that could obscure the description of this application.

[0231] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0232] It should also be understood that, in the description of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, referring to any combination and all possible combinations of one or more of the related listed items, and including these combinations, for example, A and / or B can mean: A alone, A and B together, and B alone.

[0233] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0234] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0235] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An automatic exposure adjustment method, characterized in that, The method, applied to an electronic device, includes: the electronic device acquiring a first image; when at least one face is identified in the first image, determining a background target brightness, a face target brightness, and a brightness weight; updating the brightness weight using the position of the target face in the first image, wherein the target face is the face among the at least one face closest to the center point of the first image; fusing the background target brightness and the face target brightness using the updated brightness weight to obtain an updated background target brightness; determining a current metering brightness based on the brightness of each pixel in the first image; and adjusting exposure parameters based on the current metering brightness and the updated background target brightness.

2. The method according to claim 1, characterized in that, The step of updating the brightness weight using the position of the target face in the first image includes: determining a distance weight using the position of the target face in the first image; and updating the brightness weight according to the distance weight.

3. The method according to claim 1, characterized in that, The step of determining the distance weight using the position of the target face in the first image includes: determining the distance from the center point of the target face to the center point of the first image, and the distance from the center point of the first image to the origin; and determining the distance weight based on the distance from the center point of the target face to the center point of the first image and the distance from the center point of the first image to the origin.

4. The method according to claim 1, characterized in that, The step of determining the distance weight using the position of the target face in the first image includes: determining the distance from the center point of the target face to the center point of the first image, and the distance from the center point of the first image to the origin; determining the face contrast corresponding to the first image; and determining the distance weight based on the distance from the center point of the target face to the center point of the first image, the distance from the center point of the first image to the origin, and the face contrast.

5. The method according to claim 4, characterized in that, Determining the face contrast corresponding to the first image includes: determining the current face brightness corresponding to the at least one face and the current background brightness of the first image; and determining the face contrast based on the current face brightness and the current background brightness of the first image.

6. The method according to claim 5, characterized in that, Determining the current face brightness corresponding to the at least one face includes: when there is only one face, determining the face position weight based on the distance from the center point of the first image to the origin and the distance from the center point of the current face to the center point of the first image; and determining the current face brightness based on the face position weight and the average pixel brightness of the current face.

7. The method according to claim 5, characterized in that, Determining the current face brightness corresponding to the at least one face includes: when there are multiple faces, determining the face position weight corresponding to each face based on the distance of each face to the center of the first image; and determining the current face brightness based on the face position weight corresponding to each face and the average pixel brightness corresponding to each face.

8. The method according to any one of claims 2 to 6, characterized in that, After determining the distance weight, the method further includes: determining the area of ​​the target face and determining a face area coefficient based on the area of ​​the target face; correcting the distance weight based on the face area coefficient; and updating the brightness weight using the distance weight, which includes: updating the brightness weight using the corrected distance weight.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: determining the posture of the electronic device, the posture of the electronic device including landscape and portrait modes; and shifting the initial image center point in a specified direction according to the posture of the electronic device to obtain the first image center point.

10. An electronic device, characterized in that, include: One or more processors; one or more memories; The memory stores one or more programs that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 9.

12. A computer program product, characterized in that, Includes a computer program that, when run, causes a computer to perform the method as described in any one of claims 1 to 9.

Images