Image processing method and related equipment

Abstract

The embodiment of the invention provides an image processing method and related equipment, and relates to the technical field of terminals. The EV0target can be adjusted through the brightness of the main body target, and the problems of black scene overexposure and white scene underexposure are reduced. Specifically, in response to an operation for starting a camera application, a first frame of original image is collected; obtaining first 3A statistical data, wherein the first 3A statistical data comprises two or more different types of 3A statistical data; under the condition that the electronic equipment shoots a target scene, obtaining first main body target brightness according to the first 3A statistical data; obtaining first target brightness according to the first main body target brightness; and using the camera to collect a second frame of original image based on the first target brightness. Therefore, in the target scene, the electronic equipment can obtain the target brightness adaptive to the target scene through the main body target brightness; and the exposure value of the next frame is continuously adjusted by using the target brightness of the previous frame, so that a high-quality image without overexposure and underexposure is obtained, and the photographing experience of a user is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image processing method and related equipment. Background Art

[0002] Electronic devices can provide a variety of services. For example, electronic devices can provide users with services such as taking photos and recording videos based on camera applications.

[0003] In a possible implementation, when an electronic device is used to capture an image of a black scene, a dark scene, or a white scene, there may be a large color difference between the captured object in the image and the actual captured object; for example, the main object in a black scene is too bright, and the main object in a white scene is too dark. The image quality obtained by the electronic device is poor, which reduces the user's photography experience. Summary of the invention

[0004] The embodiments of the present application provide an image processing method and related equipment, which are applied to the field of terminal technology; in shooting scenarios, the problems of overexposure of images in black scenes and underexposure of images in white scenes can be reduced.

[0005] In a first aspect, an embodiment of the present application proposes an image processing method; the method includes: in response to an operation for starting a camera application, using a camera to capture a first frame of original image; obtaining a first 3A statistical data in the first frame of original image; when the electronic device shoots a target scene, obtaining a first subject target brightness according to the first 3A statistical data, the first 3A statistical data including two or more different categories of 3A statistical data; obtaining a first target brightness according to the first subject target brightness; based on the first target brightness, using a camera to capture a second frame of original image; obtaining a second 3A statistical data in the second frame of original image; when the electronic device shoots a target scene, obtaining a second subject target brightness according to the second 3A statistical data; obtaining a second target brightness according to the second subject target brightness.

[0006] After the electronic device starts the camera application, the camera can collect the original image in real time and send the processed image for display. Among them, the shooting scene can be a target scene, and the target scene can be a black scene or a white scene in the embodiment of the present application. In the target scene, the electronic device can execute the image processing method provided in the embodiment of the present application.

[0007] The first subject target brightness may be predicted by the AI ​​AE model in the embodiment of the present application based on the first 3A statistical data, and the target brightness may correspond to the EV0 target (also referred to as the AE target); the first target brightness may be the EV0 target obtained by referring to the first subject target brightness. The second subject target brightness may be predicted by the AI ​​AE model in the embodiment of the present application based on the second 3A statistical data, and the second target brightness may be the EV0 target obtained by referring to the second subject target brightness.

[0008] In an embodiment of the present application, the electronic device can use the first target brightness to adjust the exposure value during the second frame acquisition process, thereby achieving the effect of using the 3A statistical data of the previous frame of the original image to adjust the exposure brightness of the next frame of the original image.

[0009] Specifically, in a possible implementation, EV0 target is a preset value obtained based on the sensitivity ISO in the 3A statistical data, which is suitable for normal scenes, but in black scenes, electronic devices will still use the preset value; when the reflectivity of the black area is low, converging the brightness of the image to the preset value will result in overexposure; similarly, in white scenes, converging the brightness of the image to the preset value will result in underexposure.

[0010] In the embodiment of the present application, the electronic device uses a variety of 3A statistical data to predict a subject target brightness, and the EV0 target can be adjusted according to the subject target brightness; for example, in a black scene, the subject target brightness output by the AI ​​AE model based on the 3A statistical data is low, and the EV0 target is lowered, so that the EV0 target in the embodiment of the present application is less than the preset value, thereby reducing the problem of overexposure in black scenes. In a white scene, the subject target brightness output by the AI ​​AE model based on the 3A statistical data is high, and the EV0 target is increased, so that the EV0 target in the embodiment of the present application is greater than the preset value, thereby reducing the problem of underexposure in white scenes.

[0011] In this way, when the shooting scene is a target scene, the electronic device can obtain the target brightness adapted to the target scene through the subject target brightness; and use the target brightness of the previous frame to continuously adjust the exposure value of the next frame, thereby obtaining a high-quality image that is neither overexposed nor underexposed, thereby improving the user's shooting experience.

[0012] Optionally, the target scene includes a black scene or a white scene, wherein the black scene includes a shooting scene in which a proportion of a black area is greater than a first threshold, and the white scene includes a shooting scene in which a proportion of a white area is greater than a second threshold.

[0013] The first threshold value may be the same as or different from the second threshold value. In this way, the electronic device may use the image processing method provided by the embodiment of the present application in a black scene or a white scene, thereby obtaining a high-quality image in which the black scene is not overexposed and the white scene is not underexposed.

[0014] Optionally, the first subject target brightness is positively correlated with the first 3A statistical data; the first target brightness is positively correlated with the first subject target brightness; the second subject target brightness is positively correlated with the second 3A statistical data; and the second target brightness is positively correlated with the second subject target brightness.

[0015] It can be understood that in the embodiment of the present application, the subject target brightness is positively correlated with the 3A statistical data; the target brightness is positively correlated with the subject target brightness; so that the electronic device can obtain the EV0 target through a variety of 3A statistical data. For example, in a black scene, a lower subject target brightness can be obtained, thereby reducing the EV0 target. In a white scene, a higher subject target brightness can be obtained, thereby increasing the EV0 target. In this way, the mapping relationship between the 3A statistical data and the subject target brightness and the mapping relationship between the subject target brightness and the EV0 target can be used to control the electronic device to obtain an image that is not overexposed in a black scene, and an image that is not underexposed in a white scene.

[0016] Optionally, after obtaining the first target brightness according to the first subject target brightness, it also includes: displaying a first frame image, the first frame image is obtained after the electronic device processes the first frame original image; wherein the image format of the first frame original image is different from the image format of the first frame image; after obtaining the second target brightness according to the second subject target brightness, it also includes: displaying a second frame image; the second frame image is obtained after the electronic device processes the second frame original image; wherein the image format of the second frame original image is different from the image format of the second frame image; the brightness of the first frame image is different from the brightness of the second frame image; the brightness of the second frame image is related to the brightness of the first frame original image and the first target brightness.

[0017] This process can correspond to Figure 3 In the preview process, the electronic device can use 3A statistical data to adjust the exposure of the next frame, or process the original image and display a preview image on the display.

[0018] Exemplarily, the first frame original image and the second frame original image may correspond to Figure 3Among the first original image 1 to the sixth original image 6, any two adjacent original images may be raw images. After obtaining the first frame of the original image, the first frame of the original image may be processed into a first frame of the image by the ISP first module and the ISP second module, and the first frame of the image may be displayed on the display screen. The first frame of the image may be a YUV or RGB image. After obtaining the second frame of the original image, the second frame of the original image may be processed into a second frame of the image by the ISP first module and the ISP second module, and the second frame of the image may be displayed on the display screen. The first frame of the image may be a YUV or RGB image.

[0019] It can be understood that the brightness of the first frame of the original image can be the average brightness of each pixel. After the first frame of the original image is collected, the brightness of the first frame of the original image and the first target brightness can be obtained; the electronic device can adjust the average brightness of the original image to converge to the target brightness; so that the exposure value corresponding to the brightness difference between the brightness of the first frame of the original image and the first target brightness acts on the second frame of the original image, thereby affecting the brightness of the second frame of the image. The brightness of the second frame of the image is different from the brightness of the first frame of the image.

[0020] It should be noted that the electronic device can adjust the exposure value through multiple frames of original images, so the brightness of the second frame image is different from that of the first frame image, but the brightness difference between the second frame image and the first frame image will not be too large to affect the user's shooting experience.

[0021] In this way, the electronic device can continuously display the image with the adjusted exposure value on the display screen, so that the display screen presents a high-quality image that is neither overexposed nor underexposed.

[0022] Optionally, when the target scene is a black scene, the brightness of the second frame image is smaller than the brightness of the first frame image; or, when the target scene is a white scene, the brightness of the second frame image is larger than the brightness of the first frame image.

[0023] In this way, in a black scene, the electronic device can achieve the effect of not overexposure by lowering the brightness value of the image; in a white scene, the electronic device can achieve the effect of not underexposure by increasing the brightness value of the image.

[0024] Optionally, the first 3A statistical data includes one or more of the following: a brightness value LV of the first frame original image, an image block brightness block luma of the first frame original image, a white balance gain coefficient AWB gain of the first frame original image, a grayscale histogram of the first frame original image, a luminance index value lux index of the first frame original image, a color temperature of the first frame original image, and a sensitivity ISO of the first frame original image;

[0025] Among them, the brightness of the first subject target is positively correlated with the LV of the first frame original image; the brightness of the first subject target is positively correlated with the block luma of the first frame original image; the brightness of the first subject target is positively correlated with the AWB gain of the first frame original image; the brightness of the first subject target is positively correlated with the grayscale histogram of the first frame original image, the brightness of the first subject target is positively correlated with the lux index of the first frame original image; the brightness of the first subject target is positively correlated with the color temperature of the first frame original image; the brightness of the first subject target is positively correlated with the ISO of the first frame original image.

[0026] In this way, electronic equipment can more accurately obtain the brightness of the main target in the target scene through multiple 3A statistical data, thereby improving the accuracy of EVO target and reducing the problems of underexposure and overexposure in the target scene.

[0027] Optionally, before obtaining the brightness of the first subject target according to the first 3A statistical data, it also includes: detecting the shooting scene of the first frame of the original image; obtaining the brightness of the first subject target according to the first 3A statistical data, including: when the shooting scene meets a preset condition, obtaining the brightness of the first subject target according to the first 3A statistical data; wherein the preset condition includes one or more of the following: the first frame of the original image does not include facial information; in the process of acquiring the first frame of the original image, the distance between the subject object and the camera is less than a distance threshold; the brightness difference between each pixel area in the first frame of the original image is less than a brightness difference threshold; in the first frame of the original image, the brightness of the subject object is greater than the brightness threshold; in the first frame of the original image, the ratio of the maximum color level gain coefficient to the minimum color level gain coefficient is less than a third threshold.

[0028] Among them, the fact that the first frame of the original image does not include facial information may correspond to preset condition 1 and / or preset condition 6. In the process of capturing the first frame of the original image, the distance between the main object and the camera is less than the distance threshold; this may correspond to preset condition 2 and / or preset condition 7. The brightness difference between each pixel area in the first frame of the original image is less than the brightness difference threshold; this may correspond to preset condition 3 and / or preset condition 8. In the first frame of the original image, the brightness of the main object is less than the brightness threshold; this may correspond to preset condition 4 and / or preset condition 9. In the first frame of the original image, the ratio of the maximum color scale gain coefficient to the minimum color scale gain coefficient is less than the third threshold, which may correspond to preset condition 5 and / or preset condition 10.

[0029] In this way, the electronic device can more accurately identify the target scene based on the above-mentioned preset conditions, thereby improving the accuracy of the image processing method in the embodiment of the present application, and avoiding the problem of underexposure or overexposure caused by using the image processing method in the embodiment of the present application in ordinary scenes.

[0030] Optionally, the brightness difference of each pixel area in the first frame of the original image is obtained by the ambient light brightness value alsLV of the multispectral sensor and the ambient light brightness value aeLV of the image sensor, and the brightness difference threshold includes a first brightness difference threshold and a second brightness difference threshold; the brightness difference of each pixel area is less than the brightness difference threshold, including: the difference between alsLV and aeLV is less than the first brightness difference threshold, and / or the difference between alsLV and aeLV is less than or equal to the second brightness difference threshold; the first brightness difference threshold is less than the second brightness difference threshold. In this way, the dual thresholds can be used to reduce scenes with sudden brightness changes, thereby improving the stability of subsequent AI AE model operation and the accuracy of outputting the brightness of the main target.

[0031] Optionally, in the process of acquiring the first frame of the original image, the difference between alsLV and aeLV of the first frame of the original image is less than the first brightness difference threshold; in the process of acquiring the second frame of the original image to the m-th frame of the original image, the difference between alsLV and aeLV of the second frame of the original image to the m-th frame of the original image is less than the second brightness difference threshold; in the process of acquiring the m+1-th frame of the original image, the difference between alsLV and aeLV of the m+1-th frame of the original image is greater than the second brightness difference threshold; wherein, the shooting scenes of the second frame of the original image to the m-th frame of the original image are the target scenes, and the shooting scene of the m+1-th frame of the original image is not the target scene; m is a positive integer greater than 2.

[0032] This process may correspond to the determination process of the first original image 1 to the fourth original image 4 in step S403. When any frame of the multiple frames of original images is less than the first brightness difference threshold, the multiple frames of original images after the frame enter the determination process of the target scene; until a certain frame is greater than the second brightness difference threshold, the original image after the certain frame no longer enters the determination process of the target scene.

[0033] In this way, the dual thresholds can be used to reduce scenes with sudden brightness changes, thereby improving the stability of subsequent AI AE model operations and the accuracy of the output subject target brightness.

[0034] Optionally, obtaining the first subject target brightness according to the first 3A statistical data also includes: inputting the first 3A statistical data into a first neural network model to obtain the first subject target brightness; wherein, when the target scene is a black scene, the first subject target brightness is less than a first preset brightness, and the first preset brightness is a brightness obtained based on a grayscale histogram of the first frame original image; or, when the target scene is a white scene, the first subject target brightness is greater than the first preset brightness.

[0035] The first neural network model can be an AI AE model, and the first preset brightness can be the brightness obtained by the grayscale histogram in the first frame of the original image. It can be understood that in a black scene, the main target brightness output by the AI ​​AE model will generally be less than the first preset brightness, so as to reduce the EV0 target, thereby achieving the effect of not overexposing the black scene. In a white scene, the main target brightness output by the AI ​​AE model will generally be greater than the first preset brightness, so as to increase the EV0 target, thereby achieving the effect of not underexposing the white scene. In this way, the electronic device obtains the main target brightness based on the AI ​​AE model, and then adjusts the EV0 target by the main target brightness, thereby reducing the problem of overexposure of black scenes and underexposure of white scenes.

[0036] Optionally, the first neural network model is trained by the following method: obtaining a training data set, the training data set includes multiple samples, any sample includes the sample subject target brightness and multiple sample 3A statistical data, and the multiple sample 3A statistical data correspond to multiple sample images with different exposure values ​​under the same target scene; for any sample, multiple sample 3A statistical data are input into the model to be trained to obtain the predicted subject target brightness; the training is completed when the value calculated by the loss function converges to obtain the first neural network model. In this way, the electronic device can obtain a trained AI AE model, and the AI ​​AE model can have the ability to output the subject target brightness according to the sample 3A statistical data, thereby improving the accuracy of the subject target brightness.

[0037] Optionally, based on the brightness of the first target, using a camera to capture a second frame of original image includes: adjusting the exposure value of the camera according to the difference between the brightness of the first target and the brightness of the first frame of original image; wherein, when the brightness of the first target is greater than the brightness of the first frame of original image, the adjusted exposure value is greater than the exposure value before adjustment; or, when the brightness of the first target is less than the brightness of the first frame of original image, the adjusted exposure value is less than the exposure value before adjustment; using a camera to capture the second frame of original image; in the process of capturing the second frame of original image, the exposure value of the camera is the adjusted exposure value. In this way, the electronic device can use the 3A statistical data of the previous frame to adjust the exposure value of the next frame, thereby achieving the purpose of converging to the EV0 target.

[0038] Optionally, after obtaining the second target brightness according to the second subject target brightness, the method includes: using a camera to capture the nth frame of original image based on the n-1th target brightness; obtaining the nth 3A statistical data in the nth frame of original image where n is a positive integer greater than 2; in the case where the electronic device shoots the target scene, obtaining the nth subject target brightness according to the nth 3A statistical data; the nth subject target brightness is positively correlated with the nth 3A statistical data; obtaining the nth target brightness according to the nth subject target brightness; the nth target brightness is positively correlated with the nth subject target brightness; the brightness of the nth frame of original image converges to the nth target brightness.

[0039] It is understandable that, in the process of capturing the original image, the electronic device can continuously adjust the exposure value and the target brightness so that the brightness of the original image converges to the target brightness to achieve a good exposure effect. For example, when the electronic device captures the nth frame of the original image, the brightness of the nth frame of the original image is the same as the nth target brightness or within a certain range, and the brightness of the nth frame of the original image converges to the nth target brightness.

[0040] In this way, the electronic device can converge the brightness of the original image to near the EV0 target by adjusting the exposure value, thereby achieving a good exposure effect and ensuring that the black scene is not overexposed and the white scene is not underexposed.

[0041] Optionally, after obtaining the nth target brightness according to the nth subject target brightness, it also includes: receiving an operation for taking pictures; in response to the operation for taking pictures, using a camera to capture and cache the n+1th frame of original image to the n+ith frame of original image; post-processing the n+2th frame of original image to obtain a target image; the n+2th frame of original image is the original image with the smallest difference between the target brightness and the brightness of the original image among the n+1th frame of original image to the n+ith frame of original image; the original image includes a raw image, and the target image includes an RGB image or a YUV image; the clarity of the target image is higher than that of the n+2th frame of image, and the n+2th frame of image is an image obtained by processing the n+2th frame of original image and sent for display; the n+2th frame of image includes an RGB image or a YUV image; and the target image is saved to a gallery application.

[0042] This process may be a photo-taking process, wherein the operation for taking a photo may be the user pressing the shutter button and then lifting it up; after lifting it up, the electronic device may start processing the original image based on the up event, and generate a photo and save it in the gallery application.

[0043] The target image can be, for example, Figure 3 In the picture 5 of the photographing process of the embodiment shown, for example, the electronic device obtains the fourth original image 4-original image 6, wherein the brightness of the fifth original image 5 converges to the vicinity of the EV0 target, and then further processes the original image. The n+2 frame image can be, for example Figure 3 The illustrated embodiment shows an image 5 in the preview process. It is understandable that, in the process of taking a photo, the electronic device performs post-processing on the image 5 through the post-processing algorithm module to further improve the quality of the image, such as clarity.

[0044] In this way, after continuously adjusting the exposure value during the preview process, the electronic device can obtain an image with a better exposure effect during the photo-taking process, thereby improving the quality of the photo and enhancing the user experience.

[0045] In a second aspect, an embodiment of the present application provides an electronic device, which may also be referred to as a terminal device, a terminal, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), etc. The terminal device may be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with a wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, etc.

[0046] The electronic device comprises: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the method as in the first aspect.

[0047] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the method of the first aspect is implemented.

[0048] In a fourth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the method of the first aspect.

[0049] An embodiment of the present application provides a chip system, which includes at least one processor and a communication interface, the communication interface and at least one processor are interconnected through lines, and the at least one processor is used to run computer programs or instructions to execute the method of the first aspect.

[0050] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of a shooting scene in which a black scene is overexposed and a white scene is underexposed in a possible implementation;

[0052] Figure 2 is a schematic diagram of an exposure process in a possible implementation;

[0053] Figure 3 A schematic diagram of an exposure process provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of a process for identifying a black scene provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of another process for identifying a black scene provided in an embodiment of the present application;

[0056] Figure 6 A schematic diagram of a process for identifying a white scene provided in an embodiment of the present application;

[0057] Figure 7 A schematic diagram of a training process of an AI AE model provided in an embodiment of the present application;

[0058] Figure 8 A software structure block diagram of an electronic device 100 provided in an embodiment of the present application;

[0059] Fig. 9 A schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application;

[0060] Fig.10 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0062] 1) 3A algorithm: It can include three algorithms for automatically adjusting camera parameters: auto focus (AF), automatic exposure (AE) and auto white balance (AWB). The 3A algorithm can help the camera automatically adjust parameters according to environmental conditions to obtain better image quality and effects.

[0063] 2) Electronic equipment

[0064] The terminal device of the embodiment of the present application may also be any form of electronic device. For example, the electronic device may include a handheld device with image processing function, a vehicle-mounted device, etc. For example, some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, terminal devices in 5G networks or future evolved public land mobile communication networks (public land mobile The embodiments of the present application do not limit this.

[0065] As an example but not limitation, in the embodiment of the present application, the electronic device may be a wearable device, which may also be referred to as a wearable smart device. Wearable devices may be, for example, smart watches, smart glasses, smart bracelets, and smart jewelry. In the embodiment of the present application, the electronic device may also be a terminal device in an Internet of Things (IoT) system.

[0066] The electronic devices in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0067] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0068] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0069] It should be noted that the "at..." in the embodiment of the present application can be the instant when a certain situation occurs, or a period of time after a certain situation occurs, and the embodiment of the present application does not specifically limit this. In addition, the display interface provided in the embodiment of the present application is only an example, and the display interface can also include more or less content.

[0070] In a possible implementation, the electronic device can take a photo of the photographed object, and the photo scene can include a preview scene and a photo scene. The preview scene can be understood as: the electronic device displays the picture captured by the camera and does not save the image corresponding to the picture; the photo scene can be understood as: the electronic device displays the picture captured by the camera and saves the image corresponding to the picture. In the photo scene, the electronic device can obtain one or more images including the photographed object. The photographed object can be a person, an animal, etc., or a background such as the sky or snow.

[0071] The following is an example of a preview scene. Figure 1 The problem scenario in which the photographed object is too bright in a black scene and too dark in a white scene in a possible implementation is described.

[0072] In a possible implementation, a black scene may correspond to Figure 1 In the scene shown in Figure a of FIG. 1 , the black scene may include a little girl 101, a stage 102, and a black background 103. In the black scene, the electronic device receives an operation for opening a camera application; in response to the operation, the electronic device may display a camera application interface in the preview scene. The camera application interface in the preview scene may be as follows: Figure 1 From the comparison between the scene and the image displayed by the electronic device, it can be seen that when the electronic device is shooting in a black scene, the object being shot in the displayed image is too bright than the object being shot in the real scene. For example, in the black scene, the color brightness of the little girl 101, the stage 102 and the black background 103 are all lower than the color brightness of the little girl 101, the stage 102 and the black background 103 displayed by the electronic device.

[0073] In a possible implementation, the white scene may correspond to Figure 1 In the scene shown in Figure c in FIG. 1 , the white scene may include a little girl 104, snow 105, and a white background 106. In the black scene, the electronic device receives an operation for opening a camera application; in response to the operation, the electronic device may display a camera application interface in the preview scene. The camera application interface in the preview scene may be as follows: Figure 1 The interface shown in c in FIG. 1 is shown. From the comparison between the scene and the image displayed by the electronic device, it can be seen that when the electronic device is shooting in a white scene, the object being photographed in the displayed image is too dark than the object being photographed in the real scene. For example, in a white scene, the color brightness of the little girl 101, the stage 102, and the black background 103 are all higher than the color brightness of the little girl 101, the stage 102, and the black background 103 displayed by the electronic device.

[0074] This is because the electronic device supports the AE function, and the electronic device automatically adjusts the exposure based on the AE function. When shooting a black scene and / or a black object, since the reflectivity of the photographed object is lower than the reflectivity threshold (e.g., 18%), the AE module can increase the exposure, so that the electronic device can clearly shoot the black scene and / or the black object; but when there is a non-black photographed object (e.g., the little girl 101) in the black scene, the exposure acting on the photographed object increases, resulting in an overexposed image, and the photographed object in the image is too bright.

[0075] Similarly, when photographing a white scene and / or a white object, since the reflectivity of the photographed object is lower than the reflectivity threshold (for example, 18%), the AE module can reduce the exposure so that the electronic device can clearly photograph the white scene and / or the white object; but when there is a non-white photographed object (such as the little girl 104) in the white scene, the exposure acting on the photographed object is reduced, resulting in underexposure of the image and the photographed object in the image being too dark.

[0076] Combine the following Figure 2 The preview process and photo-taking process of an electronic device in a possible implementation are described.

[0077] Figure 2 It is a schematic block diagram of a possible camera system 200. Exemplarily, the camera system 200 includes a camera, an image signal processor (ISP) first module, an ISP second module, a decision module, an AE algorithm (AE algo) module and a cache module.

[0078] The camera is used to collect original images, such as raw images. The ISP first module can be used to obtain 3A statistical data in the original image. Specifically, the ISP first module is provided with hardware devices such as multi-spectral sensors and image sensors, and is also provided with software algorithms related to AE, AF and AWB. The ISP first module can realize the function of obtaining 3A statistical data based on software and hardware collaborative processing.

[0079] The second ISP module is used to process the original image into an RGB image or a YUV image; the decision module may include a target brightness (EV0 target, also called AE target) decision module; the EV0 target decision module may be used to obtain the EV0 target of the brightness of the current frame, so that the average value of the output brightness is close to the EV0 target; the AE algorithm module may adjust the exposure value of the next frame according to the relationship between the brightness of the current frame and the EV0 target. The cache module is used to store the data of the original image. The post-processing algorithm module may be used to process the image offline in the camera mode.

[0080] The camera collects multiple frames of original images in sequence (such as Figure 2 Taking the first original image 1-sixth original image 6) in the preview scene as an example, in the preview scene, the preview path can be: the camera captures a frame of the first original image 1, and passes the first original image 1 to the ISP first module; the ISP first module obtains the 3A statistical data in the first original image 1, passes the 3A statistical data to the decision module, and passes the first original image 1 to the ISP second module.

[0081] After receiving the first original image 1, the ISP second module can process the first original image 1 and output the image 1; the electronic device sends the image 1 for display, and the image 1 is an RGB image or a YUV image.

[0082] After the decision module obtains the 3A statistical data, the EV0 target can be calculated. When the brightness of the first original image 1 is close to EV0target, the AE algorithm module may not adjust the exposure value; that is, before the camera captures the second original image 2, the AE algorithm module does not adjust the exposure value, and the camera can capture the second original image 2 with the same exposure value as the first original image 1.

[0083] When the brightness of the first original image 1 is not close to the EV0 target, the AE algorithm module can adjust the exposure value; that is, before the camera captures the second original image 2, the AE algorithm module adjusts the exposure value, and the camera can capture the second original image 2 with the new exposure value; the electronic device can continue to adjust the exposure value according to the 3A statistical data of the second original image 2... and so on, until the brightness of a frame of image converges to the EV0 target.

[0084] When the camera captures the fourth original image 4, the electronic device receives an operation for taking a photo on the camera interface, such as pressing and lifting the shutter button. At this time, the electronic device recognizes the lift-up event, and starts executing the photo-taking process while transmitting the fourth original image 4 to the preview stream for display.

[0085] In the photo shooting scenario, the photo shooting path may be: upon receiving the up event, the fourth original image 4 is transmitted to the cache module. Afterwards, the camera may capture the fifth original image 5 and the sixth original image 6; the electronic device transmits the fifth original image 5 and the sixth original image 6 to the preview stream for display, and transmits the fifth original image 5 and the sixth original image 6 to the cache module.

[0086] One or more original images can be cached in the cache module, and the electronic device can select the original image whose exposure brightness converges to the EV0 target from the multiple original images, and save the original image to the gallery application after processing. For example, among the fourth original image 4-the sixth original image 6, the original image whose exposure brightness converges is the fifth original image 5, then the electronic device transfers the fifth original image 5 to the ISP second module and the post-processing algorithm module via the ISP first module, and processes it; wherein the post-processing algorithm module can be used to further improve the image quality, and the post-processing algorithm module includes, for example: image enhancement, super-resolution reconstruction, multi-frame fusion and other algorithm modules. The electronic device can save image 5 in the gallery application, and image 5 is an RGB image or a YUV image.

[0087] It is understandable that the data flow between the ISP second module and the post-processing algorithm module can be mutual; for example, the data flow can flow to the post-processing algorithm module through the ISP first module and the ISP second module; the data flow can also flow to the ISP second module through the ISP first module and the post-processing algorithm module; the data flow can also flow out of the ISP first module and pass through the ISP second module and the post-processing algorithm module multiple times. The embodiment of the present application does not limit this.

[0088] For the recording process, please refer to the relevant description of the preview process, which will not be repeated here.

[0089] In a possible implementation, EV0 target is a preset value, for example, EV0 target can be an exposure value that can make the average reflectivity of the current scene brightness equal to the reflectivity threshold. For example, if the reflectivity is 18%, for a black scene with a reflectivity lower than 18%, the AE module will increase the exposure value to achieve the purpose of converging the brightness to EV0 target; at this time, the main object in the black scene (for example, Figure 1 For white scenes with a reflectivity higher than 18%, the AE module will reduce the exposure value to converge the brightness to the EV0 target; at this time, the main object in the white scene (for example, Figure 1 The little girl 104 in the interface shown in d in the figure has the problem of underexposure.

[0090] In view of this, an embodiment of the present application provides an image processing method, whereby an electronic device can adaptively adjust the brightness of a subject target according to a scene; for example, EV0 target is positively correlated with the brightness of the subject target, and in a black scene, the brightness of the subject target is appropriately reduced, thereby reducing the EV0 target and exposure value in the black scene, solving the problem of overexposure of the subject object in the black scene; in a white scene, the brightness of the subject target is appropriately increased, thereby increasing the EV0target and exposure value in the white scene, solving the problem of underexposure of the subject object in the white scene; in a normal scene, EV0target is not adjusted. In this way, the electronic device can adjust the exposure value in target scenes such as white scenes and black scenes, so that the exposure value of the subject object in the output image is within the normal range, and there will be no problem of overexposure or underexposure.

[0091] Let's combine Figure 3 The preview process and photo taking process of the electronic device in the embodiment of the present application are described.

[0092] Figure 3 This is a schematic block diagram of a camera system 300 provided in an embodiment of the present application. Exemplarily, the camera system 300 includes a camera, an ISP first module, an ISP second module, a decision module, an AE algorithm module, a cache module, a judgment module, and an AI processing module.

[0093] It should be understood that the ISP first module and the ISP second module involved in the preview stream may be partially or fully multiplexed with the ISP first module and the ISP second module involved in the photo stream, or may be independent of each other, and the embodiments of the present application are not limited to this.

[0094] For example, the preview stream corresponds to a set of ISP first module and ISP second module; the photo stream corresponds to a set of ISP first module and ISP second module. For another example, the preview stream and the photo stream share the same set of ISP first module and ISP second module. It can be understood that the above description of the ISP first module and ISP second module is only an example description, and the embodiments of the present application are not limited to this.

[0095] Exemplarily, the camera is used to capture original images, such as raw images. Specifically, the camera may include a lens. In some embodiments, light can be focused on an image sensor through the lens. The image sensor includes a plurality of photosensitive elements. The photosensitive elements can convert light signals into electrical signals, and then transmit the electrical signals to the ISP for conversion into digital image signals.

[0096] The first ISP module can be used to obtain 3A statistics in the original image; the 3A statistics can be, for example, lighting value (LV), block luma, automatic white balance gain coefficient (AWB gain), grayscale histogram, lux index value (lux index), color temperature (correlated color temperature, CCT) and sensitivity (International Organization for Standardization, ISO), etc. It can be understood that the embodiment of the present application only shows some 3A statistics related to brightness, and the first ISP module of the embodiment of the present application can obtain more or less 3A statistics, and the embodiment of the present application does not limit this.

[0097] The second ISP module is used to process the original image and process the raw image into an RGB image or a YUV image. For example, the second ISP module may include a demosaic module, a color correction module, a global tone mapping module, a gamma module, and a color space transform module. After the above modules are processed, the second ISP module can output an RGB image or a YUV image.

[0098] It should be noted that the embodiment of the present application only exemplarily shows the relationship between the ISP first module and the ISP second module. In actual scenarios, the electronic device may also first process the image through some algorithms in the ISP second module, so that the ISP first module obtains 3A statistical data from the processed image. The embodiment of the present application does not limit the order in which the ISP first module and the ISP second module are operated.

[0099] The judgment module is used to determine the scene of the current frame image. For example, the scene may include a normal scene, a black scene, a white scene, and a face scene, etc. In the embodiment of the present application, the judgment module may perform corresponding output according to the scene recognition result; for example, in normal scenes and face scenes, the judgment module may pass the 3A statistical data to the decision module; in black scenes and / or white scenes, the judgment module may output the 3A statistical data to the AI ​​processing module.

[0100] It should be noted that, in the embodiments of the present application, a black scene can be understood as a black and / or dark scene, and a white scene can be understood as a white and / or light scene. The image processing method provided in the embodiments of the present application is not only applicable to scenes involving pure black and pure white. For example, a black scene includes a shooting scene in which the proportion of black areas is greater than a first threshold, and a white scene includes a shooting scene in which the proportion of white areas is greater than a second threshold; the first threshold may be the same as or different from the second threshold. For example, the first threshold and the second threshold may be, for example, 80%. The embodiments of the present application do not limit the specific values ​​of the first threshold and the second threshold.

[0101] The AI ​​processing module may include an AI processing model (also referred to as an AI AE model). In an embodiment of the present application, a neural network model may be pre-set in the AI ​​processing module. The neural network model may use 3A statistical data as input, perform neural network processing through trained parameters, and output a main target brightness indicating the current scene. It is understandable that the model has the ability to adaptively output the main target brightness based on the 3A statistical data.

[0102] For example, in a black scene, after the AI ​​processing module obtains the 3A statistical data, it can output the main target brightness, which is less than the preset brightness. The preset brightness can be a brightness value obtained based on the grayscale histogram of the original image. Since the EV0 target in the possible implementation is related to the preset brightness, and in the embodiment of the present application, the main target brightness is less than the preset brightness, Figure 2 Compared with the EV0 target in the decision module, Figure 3 The EV0 target obtained by the decision module referring to the subject target brightness is lower than Figure 2 EV0 target in the image; it can reduce the exposure value in black scenes and reduce the image brightness, thereby reducing the problem of overexposure in black scenes and improving image quality.

[0103] In a white scene, the AI ​​processing module obtains 3A statistical data and outputs the main target brightness. The main target brightness is greater than the preset brightness, so Figure 2 Compared with the EV0 target in the decision module, Figure 3 The EV0 target obtained by the decision module referring to the subject target brightness is higher than Figure 2 EV0 target in the image; it can improve the exposure value in white scenes and increase the image brightness, thereby reducing the problem of underexposure in white scenes and improving image quality.

[0104] In the embodiment of the present application, the decision module, the AE algorithm module, the cache module and the post-processing algorithm module can refer to Figure 2The relevant introduction in the embodiment of the present application will not be repeated here.

[0105] In the embodiment of the present application, Figure 3 The process of previewing and taking photos can be referred to Figure 2 The relevant description in , here will not describe in detail the image display process in the preview stream and the image storage process in the photo stream. Figure 3 In the preview process, the electronic device can process the first original image 1 to the sixth original image 6 based on the ISP first module and the ISP second module to obtain and display images 1 to 6. Figure 3 In the shooting process, before capturing the fourth original image 4, upon receiving a photo taking operation, the electronic device may cache the fourth original image 4 to the sixth original image 6, and post-process the original image with the best exposure value convergence to obtain a target image, and save the target image to a gallery application. For example, the target image is image 5, and image 5 is saved to the gallery application.

[0106] Below Figure 3 The adjustment process of the exposure value is explained in detail. Figure 3 As shown:

[0107] Taking the preview scene as an example, when the electronic device receives a trigger operation for starting the camera application, the camera obtains the original image 31 and transmits the original image 31 to the first ISP module; the first ISP module transmits the original image 32 to the second ISP module, and the second ISP module processes the original image 32 to obtain and display the RGB image or YUV image 33. The original image 31 and the original image 32 may be the same image or different images.

[0108] At the same time, the camera system 300 can adjust the exposure value of the next frame of the original image based on the relevant data of the original image of the current frame. It is understandable that within a period of time, the background, main object, etc. between the multiple frames of original images collected by the camera are relatively similar, and the continuous multiple frames of original images in the preview stream are related, so the exposure value of the next frame can be predicted and adjusted based on the original image information of the current frame.

[0109] Specifically, for any frame of the original image in the preview process, the processing process of the original image 31 of the current frame may include:

[0110] S301: The camera transmits the original image 31 to the first ISP module.

[0111] The original image 31 may be a raw image.

[0112] S302 , the first ISP module obtains 3A statistical data in the original image 31 .

[0113] 3A statistics may include AE ​​statistics, AWB statistics, and AF information. In the embodiment of the present application, the electronic device may adjust the exposure parameters, and 3A statistics may be understood as AE statistics related to exposure. 3A statistics may include, for example: LV, block luma, AWB gain, grayscale histogram, lux index, CCT, ISO, etc.

[0114] It should be noted that, in the embodiment of the present application, the 3A statistical data obtained by the ISP first module can also be obtained from the image processed by the ISP second module. For example, before the preview stream enters the ISP second module, the ISP first module obtains part of the 3A statistical data; after the preview stream enters the ISP second module, the ISP first module can also obtain part of the 3A statistical data. For example, the ISP second module includes a color correction module and a color space conversion module. After the ISP second module performs color correction and color space conversion on the original image 31, the ISP first module can also obtain part of the 3A statistical data. This embodiment of the present application will not be described in detail.

[0115] S303: The ISP first module transmits the 3A statistical data to the judgment module.

[0116] S304: The judgment module judges whether the current scene is a target scene based on 3A statistical data.

[0117] The judgment module can be used to judge the current scene; for example, the scenes include: ordinary scene, white scene, black scene and face scene, etc. Among them, the target scene can be a white scene and a black scene.

[0118] The judgment module may be one or more pre-set neural network models. The neural network model may use 3A statistical data as input, perform neural network processing through trained parameters, and output a scene label indicating the current scene and a confidence level corresponding to the current scene.

[0119] The identifier of a normal scene may be a first value, the identifier of a black scene may be a second value, the identifier of a face scene may be a third value, and the identifier of a white scene may be a fourth value. The first value to the fourth value may be four different values ​​in any form such as a number or a letter, for example, the label of a normal scene is 0, the label of a black scene is 1, the label of a face scene is 2, and the label of a white scene is 3.

[0120] In an embodiment of the present application, the judgment module can judge the current scenario based on the 3A statistical data, and instruct the AI ​​processing module to perform subsequent processing in an abnormal scenario; or instruct the decision module to perform subsequent processing in a normal scenario.

[0121] The embodiments of this application may be combined with Figure 4-Figure 6 The illustrated embodiment describes the judgment process of the judgment module in detail, which will not be described here in detail.

[0122] S305: When the current scene is the target scene, the judgment module inputs the 3A statistical data into the AI ​​processing module.

[0123] S306. The AI ​​processing module outputs the predicted subject target brightness to the decision module.

[0124] The AI ​​processing module may be a pre-set AI AE model. The AI ​​AE model may take 3A statistical data as input, perform AI AE model processing through trained parameters, and output predicted subject target brightness. The subject target brightness may be a target brightness value of a subject object, and the subject object may be an area in which a certain brightness range accounts for a large proportion in the shooting scene. For example, the black area in a black scene accounts for the largest proportion, and the black area is the subject object. The white area in a white scene accounts for the largest proportion, and the white area is the subject object. The embodiments of the present application do not limit this. The subject target brightness may be a grayscale value, for example: 0-255, wherein the subject target brightness is positively correlated with the grayscale value.

[0125] In an embodiment of the present application, 3A statistical data may include statistical parameters of multiple different structures. For example, blockluma may be brightness data of a multivariate image block of a×b×c (for example, 64×48×3); univariate data such as LV and grayscale histogram. In an embodiment of the present application, a convolutional neural network (CNN) model may be used to predict the brightness of the subject target. This is because the structure of block luma data is similar to that of an image in RGB format. For example, both are data of three data channels, and the CNN mode is suitable for processing image data (for example, images in RGB format), so in an embodiment of the present application, the AI ​​processing module may include a CNN model.

[0126] It should be noted that in the embodiment of the present application, the target brightness of the main object can be predicted so that when the exposure parameters are subsequently adjusted, the target brightness of the main object can be referred to to reduce the impact of the black background or white background on the brightness of the main object. For example, in a possible implementation, in a black scene, the electronic device will increase the brightness of the black background, causing the main object to be overexposed; in a white scene, the electronic device will reduce the brightness of the white background, causing the main object to be underexposed. In the embodiment of the present application, in the process of adjusting the exposure, the target brightness of the main object can be referred to so that after adjusting the exposure parameters, the brightness of the main object is moderate and there will be no overexposure or underexposure.

[0127] S307, the decision module adjusts EV0 target according to the brightness of the main target; wherein the brightness of the main target is positively correlated with EV0 target.

[0128] EV0 target can be the target brightness, and EV0 target can be understood as a standard value that the average brightness of the pixels in the image can reach after exposure. In an embodiment of the present application, the AI ​​processing module can obtain the subject target brightness, and the decision module can adjust the EV0 target according to the subject target brightness. For example, the subject target brightness is positively correlated with EV0 target. The greater the subject target brightness output by the AI ​​processing module, the greater the value of EV0 target; the smaller the subject target brightness output by the AI ​​processing module, the smaller the value of EV0 target.

[0129] It is understandable that the embodiments of the present application can adjust the EV0 target by the subject target brightness. For example, in a black scene, the AI ​​processing module can output a lower subject target brightness, thereby reducing the EV0 target. Subsequently, the electronic device can converge the overall average brightness of the image to the vicinity of the EV0 target by adjusting frame by frame. The average brightness of the image in the embodiments of the present application is relatively low, so that the problem of overexposure of the subject object in the black scene will not occur. Similarly, in a white scene, the AI ​​processing module can output a higher subject target brightness, thereby increasing the EV0 target. Subsequently, the electronic device can converge the overall average brightness of the image to the vicinity of the EV0 target by adjusting frame by frame. The average brightness of the image in the embodiments of the present application is relatively high, so that the problem of underexposure of the subject object in the white scene will not occur.

[0130] S308, the AE algorithm module adjusts the exposure parameters according to the EV0 target and the brightness of the original image 31.

[0131] The AE algorithm module can obtain the brightness and EV0 target of the original image 31.

[0132] When the absolute value of the difference between the brightness of the original image 31 and the EV0 target is less than or equal to the threshold, it can be considered that the brightness of the original image 31 is roughly equal to the target brightness, and a properly exposed original image can be obtained according to the current exposure parameters. The camera system 300 can end the execution process of the current frame without adjusting the exposure parameters. The camera can use the exposure parameters of the current frame to continue to capture the next frame of the original image (for example, the original image 37, Figure 3 The original image 37 is not shown in the figure, and the relevant processes of steps S301-S309 are executed again.

[0133] When the absolute value of the difference between the brightness of the original image 31 and the EV0 target is greater than the threshold, it can be considered that the brightness of the original image 31 is significantly different from the target brightness, and a properly exposed original image cannot be obtained according to the current exposure parameters. The AE algorithm can adjust the exposure parameters according to the AE table.

[0134] For example, when the brightness of the original image 31 is less than the EV0 target, the AE algorithm may adjust the exposure parameters to increase the exposure; for example, the way to increase the exposure may include extending the exposure time (shutter), increasing the sensor gain factor (sensorgain), and increasing the ISP gain factor (ISP gain), etc. When the brightness of the original image 31 is greater than the EV0 target, the AE algorithm may adjust the exposure parameters to reduce the exposure; for example, the way to reduce the exposure may include shortening the exposure time, reducing the sensor gain factor, and reducing the ISP gain factor, etc.

[0135] S309, the AE algorithm module transmits the exposure parameters to the camera.

[0136] The AE algorithm module may instruct to adjust the exposure parameters, and the adjusted exposure parameters may act on the camera, and the camera may use the adjusted exposure parameters for exposure and obtain the next frame of the original image (for example, the original image 37, Figure 3 not shown).

[0137] It is understandable that the electronic device can poll steps S301-S308 once each time it obtains an original image. When the absolute value of the difference between the brightness of the original image 37 and the EV0 target is still less than or equal to the threshold value during the execution of the relevant process of the original image 37, the AE algorithm module can increase or decrease the exposure accordingly; thereby converging the brightness of the original image frame by frame to near the EV0 target. In the preview scene, the electronic device can obtain an original image with a brightness value close to EV0target according to the adjusted exposure parameters, and the original image will not have the problem of overexposure or underexposure.

[0138] Optionally, after step S304, the following steps may also be included:

[0139] S310: When the current scene is not the target scene, the judgment module inputs the 3A statistical data into the decision module.

[0140] S311, the decision module passes the EV0 target to the AE algorithm module.

[0141] S312, the AE algorithm module adjusts the exposure parameters according to the EV0 target and the brightness of the original image 31.

[0142] It should be noted that, when the current scene is not a black scene or a white scene, the electronic device can use the exposure process of a normal scene to process the image exposure. In this process, the electronic device does not use the AI ​​processing module to predict the brightness of the subject target.

[0143] Taking the photo scene as an example, when the electronic device receives a trigger operation for the shutter key (for example, the shutter button and / or volume button in the camera application interface, etc.), the original image 34 is stored in the cache module; the cache module transmits the original image 34 to the ISP first module; the ISP first module transmits the original image 35 to the ISP second module and the post-processing algorithm module, and after the ISP second module and the post-processing algorithm module process the original image 35, the RGB image or YUV image 36 is saved. Among them, the original image 34 and the original image 35 can be the same image or different images.

[0144] It can be understood that, for any frame, the definition of the RGB image or YUV image 33 in the preview stream is lower than the definition of the RGB image or YUV image 36 in the capture stream.

[0145] It should be noted that, when the current scene is not a black scene or a white scene, the electronic device can use the exposure process of a normal scene to process the image exposure. In this process, the electronic device does not use the AI ​​processing module to predict the brightness of the subject target.

[0146] The above embodiments illustrate the process of adjusting the exposure of any frame of image in the embodiments of the present application. The following describes the process of determining the target scene shown in step S304.

[0147] Taking the detection of a black scene as an example, in a first possible implementation manner, the judgment module may screen out non-black scenes according to a plurality of preset conditions, thereby determining a black scene.

[0148] In a second possible implementation, an AI model may be added based on the first possible implementation, and the AI ​​model is used to verify the result of the first possible implementation.

[0149] In a third possible implementation, a classification model may be provided in the judgment module, wherein the classification model has the ability to output scene identification according to 3A statistical data; when the classification model outputs a label value for identifying a black scene, it can be determined that the current scene is a black scene.

[0150] Let's combine Figure 4 The determination process of the first possible implementation method is described below. Figure 4 As shown:

[0151] The preset conditions may include: preset condition 1, preset condition 2, preset condition 3, preset condition 4 and preset condition 5. Among them, preset condition 1 is used to filter out face scenes, preset condition 2 is used to filter out distant scenes, preset condition 3 can be used to filter out non-uniform scenes, preset condition 4 can be used to filter out scenes with low brightness of the main object; preset condition 5 can be used to filter out pure color scenes with colors.

[0152] Specifically, after the judgment module obtains the 3A statistical data, the flow process of the 3A statistical data in the judgment module may be as shown in the following steps S401-S405:

[0153] S401: Determine whether the current scene is a face scene.

[0154] Preset condition 1 may be that no face is recognized. A face recognition model is set in the electronic device. When the current scene includes a face, the judgment module terminates the judgment process and instructs to adjust the exposure parameters using the exposure process of a normal scene. When the preset condition 1 is met and no face is recognized, the judgment module may continue to execute step S402.

[0155] It is understandable that the electronic device may be provided with an exposure scheme for a face scene, and when a face scene is recognized, the electronic device may use a face scene exposure process to adjust the exposure of the face scene.

[0156] It should be noted that the embodiment of the present application can use the exposure process of the face scene to process the pixels related to the face in the original image; and the embodiment of the present application can use the image processing method to process the pixels other than the face in the original image. For example, in a black scene, the face area can be exposed using the exposure process of the face scene, and the background, clothes and other areas can be exposed using the image processing method provided by the embodiment of the present application, so that the brightness value of the overall image will not be overexposed.

[0157] S402: When the current scene is not a face scene, determine whether the current scene is a distant scene.

[0158] The distant scene may be a scene where the distance between the subject and the electronic device exceeds the distance threshold. It is understandable that the problem of overexposure of black scenes and underexposure of white scenes often occurs in scenes shot at close distances. When the subject is far away from the camera, the probability of underexposure or overexposure using the exposure process of ordinary scenes is low. Therefore, in the distant scene, the judgment module may terminate the judgment process and instruct to use the exposure process of ordinary scenes to adjust the exposure parameters.

[0159] 3A statistical data may include the ambient brightness value (aeLV) and the distance value of the image sensor, wherein the distance value may be a laser distance or an object distance parameter. In some embodiments, a time of flight (TOF) camera may be provided in the electronic device, and the judgment module may measure the laser distance between the subject object and the electronic device based on the TOF camera. In other embodiments, the electronic device may use a monocular, binocular or other camera to collect the object distance parameter to obtain the distance between the subject object and the camera.

[0160] Taking the distance value as laser distance as an example, preset condition 2 may be that the aeLV value is greater than the aeLV threshold, and the laser distance is greater than the distance threshold; the aeLV threshold may be, for example, 60, and the distance threshold may be, for example, 400 mm. When the 3A statistical data of the current scene meets preset condition 2, the judgment module may execute step S403; when the 3A statistical data of the current scene meets preset condition 2, the judgment module may terminate the judgment process.

[0161] S403: When the current scene is not a distant scene, determine whether the current scene is a non-uniform scene.

[0162] A uniform scene may be a scene in which the difference in brightness of each pixel region in the image is within a brightness difference threshold range, and a pixel region may be a pixel point and a pixel block of multiple pixel points. A non-uniform scene may be a scene in which the difference in brightness of each pixel region in the image exceeds a brightness difference threshold range. For example, a uniform scene may be a uniform night scene and a uniform indoor scene.

[0163] It can be understood that in the embodiment of the present application, a black scene can be a scene in which the proportion of black areas in the current shooting scene is greater than a first threshold, and a white scene can be a scene in which the proportion of white areas in the current shooting scene is greater than a second threshold. The first threshold can be the same as the second threshold, or different. For example, the first threshold and the second threshold are 80%. It can be seen that the brightness of the pixels in the black scene and the white scene is relatively uniform, so in step S403, the non-uniform scenes that are not black scenes and white scenes can be screened out by the preset condition 3, further improving the accuracy of scene recognition.

[0164] Specifically, the 3A statistical data may also include the ambient brightness value (alsLV) of the multispectral sensor. The electronic device may use the brightness difference obtained by the image sensor and the multispectral sensor to determine whether it is a non-uniform scene. The positions of the light received by the image sensor and the multispectral sensor are different. For example, the image sensor and the multispectral sensor overlap partially or completely, and the area of ​​the image sensor is smaller than the area of ​​the multispectral sensor; in some embodiments, the brightness value received by the image sensor and the brightness value received by the multispectral sensor in the non-overlapping area may be calculated to determine whether it is a non-uniform scene. Among them, the smaller the brightness difference, the higher the probability that the current scene is a black scene; the larger the brightness difference, the lower the probability that the current scene is a black scene.

[0165] In some embodiments, the judgment module may judge the non-uniform scene by means of a dual threshold. For example, the brightness difference threshold may include a first brightness difference threshold and a second brightness difference threshold; the first brightness difference threshold is less than the second brightness difference threshold. When the brightness difference between the alsLV value and the aeLV value is less than the first brightness difference threshold, the judgment module enters the judgment process of the next scene, and the judgment module may continue to execute step S404; when the brightness difference between the alsLV value and the aeLV value is greater than the second brightness difference threshold, the judgment module exits the judgment process of the next scene, and the judgment module may not execute step S404.

[0166] The dual threshold determination process can be understood as follows: when the previous frame of the original image has not entered the determination process of the next scene, the preset condition 3 of the current frame of the original image can be that the brightness difference between the alsLV value and the aeLV value is less than the first brightness difference threshold; when the previous frame of the original image enters the determination process of the next scene, the preset condition 3 of the current frame of the original image can be that the brightness difference between the alsLV value and the aeLV value is less than or equal to the second brightness difference threshold. When the preset condition 3 is met, the current scene can be considered to be a uniform scene; when the preset condition 3 is not met, the current scene can be considered to be a non-uniform scene.

[0167] For example, the first brightness difference threshold is 5, and the second brightness difference threshold is 10. The preset condition 3 is described below by taking the first original image 1 to the fourth original image 4 as an example.

[0168] 1) For the original image 1: the brightness difference between the alsLV value and the aeLV value is greater than or equal to 5, for example, the brightness difference is 6, and the judgment module does not execute step S404; it is determined that the original image 1 does not enter the judgment process of the next scene. At this time, the preset condition 3 for the next frame of the original image to enter step S404 is that the brightness difference between the alsLV value and the aeLV value is less than 5.

[0169] 2) For the second original image 2: the brightness difference between the alsLV value and the aeLV value is less than 5, for example, the brightness difference is 4, which satisfies the preset condition 3 in step 1), and the judgment module can execute step S404 to determine whether the second original image 2 enters the next scene judgment process. At this time, the next frame of the original image can enter the preset condition 3 of step S404, which is changed to the brightness difference between the alsLV value and the aeLV value is less than 10.

[0170] 3) For the third original image 3: the brightness difference between the alsLV value and the aeLV value is less than or equal to 10, for example, the brightness difference is any value between 0-10, and the preset condition 3 in step 2) is satisfied, and the judgment module can execute step S404 to determine whether the original image 3 enters the next scene judgment process. At this time, the preset condition 3 for the next frame of the original image to enter step S404 is still that the brightness difference between the alsLV value and the aeLV value is less than or equal to 10.

[0171] 4) For the fourth original image 4: the brightness difference between the alsLV value and the aeLV value is greater than 10, for example, the brightness difference is 11, which does not meet the preset condition 3 in step 3), and the judgment module may not execute step S404, and determine that the fourth original image 4 does not enter the judgment process of the next scene. At this time, the preset condition 3 of the next frame of the original image can enter step S404 and is changed to the brightness difference between the alsLV value and the aeLV value is less than 5.

[0172] And so on.

[0173] It should be noted that the above embodiment only illustrates the dual-threshold determination method by way of example, and the brightness difference between the first original image 1 and the fourth original image 4 may also be other values, which is not limited in the embodiment of the present application.

[0174] It can be understood that in the embodiment of the present application, the AI ​​processing module involves an AI AE model. During the frame-by-frame convergence of the model, if the brightness value suddenly changes, it may affect the stability of the output result of the AI ​​AE model. Therefore, in order to improve the stability of the brightness value, the embodiment of the present application adopts a dual threshold method to judge non-uniform scenes.

[0175] S404: When the current scene is not a non-uniform scene, determine whether the current scene is a scene with low brightness of the main object.

[0176] The scene with low brightness of the main object may be a scene in which the brightness of the main object is less than the brightness threshold. It is understandable that when the brightness of the main object is lower than the brightness threshold, the shooting scene may be a night scene. The electronic device may be provided with an exposure scheme for night scene scenes. In the night scene scene, the electronic device may use the exposure process of the night scene scene for ordinary scenes to adjust the exposure of the night scene scene. Therefore, in the scene with low brightness of the main object, the image processing method provided in the embodiment of the present application may not be used.

[0177] In the embodiment of the present application, when the preset condition 4 is met, the judgment module may execute step S405, and when the preset condition 4 is not met, the judgment module may terminate the judgment process and instruct to adjust the exposure parameters using the exposure process of the normal scene. The preset condition 4 may be that the image block brightness is greater than the image block brightness threshold; for example, the image block brightness threshold may be 20. When the block luma is greater than 20, the current scene is not a scene with low brightness of the main object; when the block luma is less than 20, the current scene is a scene with low brightness of the main object.

[0178] S405: When the current scene is not a scene with low brightness of the main object, determine whether the current scene is a pure color scene with color.

[0179] It is understandable that after the scene screening in steps S401-S404, the electronic device may obtain pure color scenes, such as red scenes, green scenes, black scenes, and white scenes. Pure color scenes with colors can be other scenes other than black scenes and white scenes in pure color scenes. For pure color scenes with colors, the electronic device can output appropriate exposure parameters, and there will be no overexposure problem similar to that in black scenes. Therefore, it is necessary to filter the pure color scenes with colors here.

[0180] In an embodiment of the present application, when the preset condition 5 is met, the judgment module may execute step S406. When the preset condition 5 is not met, the judgment module may terminate the judgment process and instruct to adjust the exposure parameters using the exposure process of the normal scene. Among them, the 3A statistical data may include RGB values, and the preset condition 5 may be that the ratio of the maximum RGB gain coefficient to the minimum RGB gain coefficient is less than the ratio threshold. Specifically, the ratio threshold may be 2, for example, and the preset condition 5 may be max(Rgain, Ggain, Bgain) / mix(Rgain, Ggain, Bgain)<2. Among them, Rgain=R / G; Ggain=G / G; Bgain=B / G.

[0181] It should be noted that the embodiment of the present application uses the order of steps S401-S405 to illustrate the judgment process, but the embodiment of the present application does not limit the execution order of the steps. For example, the judgment module can first judge the 3A statistical data based on any preset condition in preset conditions 1-5, and when the judgment result is no, continue to judge other preset conditions in preset conditions 1-5 until the judgment result is yes, or all preset conditions are involved in the judgment process. The embodiment of the present application does not limit the order of using the preset conditions.

[0182] S406: When the current scene is not a pure color scene, the judgment module transmits the 3A statistical data to the AI ​​processing module, and the AI ​​processing module outputs the main target brightness based on the 3A statistical data.

[0183] After the process of steps S401-S405, the judgment module filters the non-black scenes to determine that the current scene is a black scene. In a black scene, the electronic device can use the AI ​​processing module to predict the brightness of the main target, thereby reducing the problem of overexposure in a black scene.

[0184] Figure 4 The embodiment shown illustrates the first possible implementation. The second possible implementation provided by the embodiment of the present application is described below.

[0185] The second possible implementation is used to verify the accuracy of the determination result in the first possible implementation. The process can be referred to Figure 4 , the embodiments of the present application will not explain this in detail.

[0186] Exemplarily, the judgment module may further include an auxiliary verification model, which may be used to assist in scene detection and verify the judgment result. :

[0187] S501. Determine whether it is a face scene based on preset condition 1.

[0188] For step S501, reference may be made to the relevant description in step S401, which will not be repeated here.

[0189] S502: When the result is determined to be a face scene, the 3A statistical data is input into the auxiliary verification model to obtain result A; result A may include an identifier of the current scene and a confidence level corresponding to the current scene.

[0190] The auxiliary verification model can be used to verify the face scene. The auxiliary verification model can output the identification label = 2 of the face scene and the confidence of the face scene. If the confidence is greater than the confidence threshold, it is determined that the judgment result based on the preset condition 1 is accurate, and the current scene is a face scene. In this way, the auxiliary verification model can be used to verify the result of the judgment based on the preset conditions to improve the accuracy of detecting the target scene.

[0191] Alternatively, S503: when the current scene is not a face scene, determine whether it is a distant scene based on preset condition 2.

[0192] For step S503, reference may be made to the relevant description in step S402, which will not be repeated here.

[0193] S504: When the result of the determination is a distant scene, the 3A statistical data is input into the auxiliary verification model to obtain result B; result B may include an identifier of the current scene and a confidence level corresponding to the current scene.

[0194] The auxiliary verification model can be used to verify the distant scene. The ordinary scene may include the distant scene. The auxiliary verification model can output the label of the ordinary scene = 0 and the confidence of the ordinary scene. If the confidence is greater than the confidence threshold, it is determined that the determination result obtained based on the preset condition 2 is accurate, and the current scene is a distant scene.

[0195] Alternatively, S505 , when the current scene is not a distant scene, determine whether it is a non-uniform scene based on preset condition 3.

[0196] For step S505, reference may be made to the relevant description in step S403, which will not be repeated here.

[0197] S506: When the result of the determination is a non-uniform scene, the 3A statistical data is input into the auxiliary verification model to obtain a result C; the result C may include an identifier of the current scene and a confidence level corresponding to the current scene.

[0198] The auxiliary verification model can be used to verify the non-uniform scene. The normal scene may include the non-uniform scene. The auxiliary verification model can output the label of the normal scene = 0 and the confidence of the normal scene. If the confidence is greater than the confidence threshold, it is determined that the determination result obtained based on the preset condition 3 is accurate, and the current scene is a non-uniform scene.

[0199] Alternatively, S507 , when the current scene is not a non-uniform scene, determine whether it is a scene with low brightness of the main object based on preset condition 4.

[0200] For step S507, reference may be made to the relevant description in step S404, which will not be repeated here.

[0201] S508. When the judgment result is a scene with low brightness of the main object, the 3A statistical data is input into the auxiliary verification model to obtain a result D; the result D may include an identifier of the current scene and a confidence level corresponding to the current scene.

[0202] The auxiliary verification model can be used to verify the scene with low brightness of the main object. The ordinary scene may include the scene with low brightness of the main object. The auxiliary verification model can output the label of the ordinary scene = 0, and the confidence of the ordinary scene. If the confidence is greater than the confidence threshold, it is determined that the determination result obtained based on the preset condition 4 is accurate, and the current scene is a scene with low brightness of the main object.

[0203] Alternatively, S509 , when the current scene is not a scene with low brightness of the main object, determine whether it is a pure color scene based on preset condition 5.

[0204] For step S509, reference may be made to the relevant description in step S405, which will not be repeated here.

[0205] S510: When the judgment result is a pure color scene with color, the 3A statistical data is input into the auxiliary verification model to obtain a result E; the result E may include an identifier of the current scene and a confidence level corresponding to the current scene.

[0206] The auxiliary verification model can be used to verify the pure color scene with color. The ordinary scene may include the pure color scene with color. The auxiliary verification model can output the label of the ordinary scene = 0 and the confidence of the ordinary scene. If the confidence is greater than the confidence threshold, it is determined that the determination result obtained based on the preset condition 5 is accurate, and the current scene is a pure color scene with color.

[0207] In an embodiment of the present application, the auxiliary verification model can be a single model or multiple sub-models trained jointly or non-jointly, and the embodiment of the present application does not impose any restrictions on this.

[0208] In this way, based on the output results of the auxiliary verification model, the problem of false detection of preset conditions can be reduced and the accuracy of the target scene can be improved.

[0209] In a third possible implementation, the judgment module may include a classification model, which may be a model trained using 3A statistical data of multiple scenarios, and the classification model has the ability to output multiple scenario identifiers and confidence levels. Figure 5 The process shown:

[0210] S601, the judgment module inputs the 3A statistical data into the classification model to obtain the scene identification and confidence.

[0211] The classification model can output identifications of multiple scenes based on 3A statistical data. The scene identification is not limited to label=0-3 shown in the embodiment of the present application; the classification scene can have the ability to identify more scenes. For example, the scene identification may include label=0-N, where N is a positive integer.

[0212] S602: When the scene is identified as a black scene, the judgment module transmits the 3A statistical data to the AI ​​processing module.

[0213] For example, the classification model may output label=1 and the confidence that the current scene is a black scene; if the confidence is greater than or equal to the confidence threshold, step S603 is executed; if the confidence is less than the confidence threshold, the judgment process ends.

[0214] S603: The AI ​​processing module outputs the subject target brightness.

[0215] For this step, please refer to the relevant description in step S306, which will not be repeated here.

[0216] Alternatively, after step S601, the method further includes:

[0217] S604: When the scene identification is not a black scene, the judgment module transmits the 3A statistical data to the decision module.

[0218] When the output result of the classification model is not label=1, the current scene is not a black scene, and the judgment module may instruct to use the exposure process of a normal scene for exposure processing.

[0219] It should be noted that the above embodiment takes the black scene as an example and introduces three methods for determining the target scene. The embodiment of the present application can use any one method alone to perform scene detection, or can combine the above methods to improve the accuracy of scene detection. The embodiment of the present application does not limit this.

[0220] The above embodiments take a black scene as an example to illustrate the method for an electronic device to detect a black scene. The following describes a method for determining a white scene.

[0221] It should be noted that the determination method of the white scene is similar to the determination method of the black scene. The present embodiment only describes the determination process that is different between the white scene and the black scene. For other parts, please refer to Figure 4 The embodiments shown to Figure 5 The relevant descriptions in the illustrated embodiments are not repeated here. For example, taking the detection of a white scene as an example, in a first possible implementation, the judgment module may screen out non-white scenes according to a plurality of preset conditions, thereby determining a white scene.

[0222] In a second possible implementation, an AI model may be added based on the first possible implementation, and the AI ​​model is used to verify the result of the first possible implementation.

[0223] In a third possible implementation, a classification model may be provided in the judgment module, wherein the classification model has the ability to output scene identification according to 3A statistical data; when the classification model outputs a label value for identifying a white scene, it can be determined that the current scene is a white scene.

[0224] The following first describes the first possible implementation method in a white scene.

[0225] The preset conditions for identifying white scenes may include: preset condition 6, preset condition 7, preset condition 8, preset condition 9 and preset condition 10. Among them, preset condition 6 is used to screen out face scenes, preset condition 7 is used to screen out distant scenes, preset condition 8 can be used to screen out non-uniform scenes, preset condition 9 can be used to screen out scenes with low brightness of the main object; preset condition 10 can be used to screen out pure color scenes with colors.

[0226] Specifically, after the judgment module obtains the 3A statistical data, the flow process of the 3A statistical data in the judgment module may be as shown in the following steps S701-S705:

[0227] S701: Determine whether the current scene is a face scene.

[0228] Preset condition 6 may be that a face is not recognized. When preset condition 1 is met and a face is not recognized, the judgment module may continue to execute step S702.

[0229] S702: When the current scene is not a face scene, determine whether the current scene is a distant scene.

[0230] The preset condition 7 may be: the aeLV value is greater than the aeLV threshold, and / or the laser distance is greater than the distance threshold.

[0231] It is understandable that the aeLV threshold in the white scene may be higher than the aeLV threshold in the black scene; the distance threshold in the white scene may be higher than the distance threshold in the black scene, and the distance threshold may be the same or different. In some embodiments, the preset condition 7 may be, for example, that the aeLV value is greater than the aeLV threshold, and the laser distance is greater than the distance threshold; in other embodiments, the preset condition 7 may be, for example, that the aeLV value is greater than the aeLV threshold.

[0232] In the embodiment of the present application, there are many white scenes, such as snow scenes and white close scenes. For snow scenes, the judgment module can use preset condition 7: the aeLV value is greater than the aeLV threshold for restriction; for white close scenes, the judgment module can use preset condition 7: the aeLV value is greater than the aeLV threshold, and the laser distance is greater than the distance threshold for restriction. The embodiment of the present application does not limit this.

[0233] When the preset condition 7 is met, the determination module may continue to execute step S703.

[0234] S703: When the current scene is not a distant scene, determine whether the current scene is a non-uniform scene.

[0235] Preset condition 8 may be: when the brightness difference between the alsLV value and the aeLV value is less than the third brightness difference threshold, the judgment module enters the judgment process of the next scene, and the judgment module may continue to execute step S704; when the brightness difference between the alsLV value and the aeLV value is greater than the fourth brightness difference threshold, the judgment module exits the judgment process of the next scene, and the judgment module may not execute step S704. The third brightness difference threshold is less than the fourth brightness difference threshold; wherein the third brightness difference threshold may be the same as or different from the first brightness difference threshold; the fourth brightness difference threshold may be the same as or different from the second brightness difference threshold. The embodiment of the present application does not limit this.

[0236] When the preset condition 8 is met, the determination module may continue to execute step S704.

[0237] S704: When the current scene is not a non-uniform scene, determine whether the current scene is a scene with low brightness of the main object.

[0238] Preset condition 9 may be: the image block brightness is greater than the image block brightness threshold. In the embodiment of the present application, the image block brightness threshold in the black scene is less than the image block brightness threshold in the white scene. For example, in the black scene, the image block brightness threshold may be 20; in the white scene, the image block brightness threshold may be 70. Preset condition 9 may be, for example: block luma>70.

[0239] When the preset condition 9 is met, the determination module may continue to execute step S705.

[0240] S705: When the current scene is not a scene with low brightness of the main object, determine whether the current scene is a pure color scene with color.

[0241] The preset condition 10 may be: the ratio of the maximum RGB gain coefficient to the minimum RGB gain coefficient is less than the ratio threshold. The ratio threshold in the white scene may be the same as or different from the ratio threshold in the black scene, and this embodiment of the application does not limit this.

[0242] When the preset condition 10 is met, the determination module may continue to execute step S706 .

[0243] S706: When the current scene is not a pure color scene, the judgment module transmits the 3A statistical data to the AI ​​processing module, and the AI ​​processing module outputs the main target brightness based on the 3A statistical data.

[0244] After the process of steps S701-S705, the judgment module filters the non-white scenes to determine that the current scene is a white scene. In the white scene, the electronic device can use the AI ​​processing module to predict the brightness of the main target, thereby reducing the problem of underexposure in the white scene.

[0245] It should be noted that steps S701-S706 can refer to Figure 4 The related descriptions in steps S401-S406 are not repeated here.

[0246] For the second possible implementation in the white scene, the judgment module may also adopt a process similar to that shown in the second possible implementation in the black scene to verify the detection result of the white scene. The process may refer to the relevant description in steps S501-S510, which will not be repeated here.

[0247] For the third possible implementation in the white scene, the judgment module may include a classification model, and the classification model can also be used to identify the white scene, such as Figure 6 As shown:

[0248] Exemplarily, S801, the judgment module inputs 3A statistical data into the classification model to obtain the scene identification and confidence.

[0249] S802: When the scene is identified as a white scene, the judgment module transmits the 3A statistical data to the AI ​​processing module.

[0250] For example, the classification model may output label=3 and the confidence that the current scene is a white scene; if the confidence is greater than or equal to the confidence threshold, step S803 is executed; if the confidence is less than the confidence threshold, the judgment process ends.

[0251] S803: The AI ​​processing module outputs the subject target brightness.

[0252] Alternatively, after step S801, the method further includes:

[0253] S804: When the scene identification is not a white scene, the judgment module transmits the 3A statistical data to the decision module.

[0254] When the output result of the classification model is not label=3, the current scene is not a white scene, and the judgment module may instruct to use the exposure process of a normal scene for exposure processing.

[0255] Part of the description in steps S801-S804 can refer to steps S601-S604, and this embodiment of the application will not be repeated here.

[0256] The above embodiments illustrate the process of determining the target scene in the image processing method in the embodiment of the present application. Figure 7 The training process of the AI ​​AE model in the embodiment of the present application is explained.

[0257] S901. Obtain a training data set.

[0258] The training data set may include multiple training data, and any training data includes sample subject target brightness and multiple sample 3A statistical data.

[0259] The process of obtaining the training dataset can be as follows:

[0260] a) collecting multiple groups of sample images under the target scene, wherein any group of sample images includes multiple sample images of the same target scene with different exposures.

[0261] In the embodiment of the present application, underexposed, normal, and overexposed sample images of the same scene can be obtained by adjusting the exposure time and exposure gain. The sample images of the same scene can be as follows: Figure 7 Sample images 1-10 are shown.

[0262] Exemplarily, sample image 1-sample image 10 can be sample images of different exposure levels collected in a black scene. For example, sample image 1-sample image 2 are underexposed sample images in a black scene; sample image 3 is a sample image with appropriate exposure in a black scene; sample image 4-sample image 10 are overexposed sample images in a black scene.

[0263] And / or, sample image 1-sample image 10 may also be sample images of different exposure levels collected in a white scene, for example, sample image 1-sample image 8 are underexposed sample images in a white scene; sample image 9 is a sample image with appropriate exposure in a white scene; and sample image 10 is an overexposed sample image in a white scene.

[0264] It should be noted that the embodiments of the present application only illustrate sample images of the target scene, but the above sample images do not limit the embodiments of the present application. For example, the electronic device may also use different sample images to respectively train the model to be trained to output the ability of the main target brightness in the black scene and the ability of the main target brightness in the white scene. For example, sample images 1-sample images 10 of the white scene are different from sample images 1-sample images 10 of the black scene. The model to be trained can be trained using sample images of different exposure levels in the white scene to obtain an AIAE model suitable for the white scene; or, the model to be trained can be trained using sample images of different exposure levels in the black scene to obtain an AI AE model suitable for the black scene.

[0265] b) For any sample image in any group of sample images, the exchangeable image file format (Exif) information of the sample image is parsed; the Exif information includes sample 3A statistical data of the sample image.

[0266] For example, the Exif information of image 1 to image 10 is parsed to obtain Exif information 1 to Exif information 10. 3A statistical data may include: LV, block luma, AWB gain, grayscale histogram, lux index, color temperature and ISO, etc.

[0267] c) For any group of sample images, mark the target sample image, and obtain the Exif information of the target sample image, wherein the Exif information of the target sample image includes the main target brightness.

[0268] The target sample image may be a sample image with moderate exposure in the current scene.

[0269] Among them, the rules for selecting the target sample image can be as follows: the target sample image is not overexposed or underexposed; the brightness of the target sample image is not overexposed, and the dark details are relatively intact; the brightness values ​​of the grayscale histogram are accumulated in the middle, and there is no cutoff or overflow on the left and right sides.

[0270] Any training data includes sample subject target brightness and multiple sample 3A statistics. For example, for the training process of black scenes, the training data includes the subject target brightness of sample image 3 and the 3A statistics in Exif information 1-Exif information 10. For the training process of white scenes, the training data includes the subject target brightness of sample image 9 and the 3A statistics in Exif information 1-Exif information 10.

[0271] S902. Train the model to be trained based on the training data set to obtain an AI AE model.

[0272] During training, the training data set can be input into the model to be trained, and the loss calculation is performed between the main target brightness output by the model to be trained and the main target brightness marked in the sample data. Then, the parameters of the model to be trained are updated based on the result of the loss calculation. This process is repeated until the predetermined maximum number of iterations is reached, or the loss converges, or the loss value is less than a certain value. The model training can be considered to be completed and the AI ​​AE model is obtained.

[0273] Exemplarily, the model to be trained can learn the mapping relationship between 3A statistical data and the brightness of the main target, so that the trained AI AE model has the ability to adaptively output the brightness of the main target according to the 3A statistical data. Specifically, the mapping relationship between the 3A statistical data and the brightness of the main target can be, for example: the LV value is positively correlated with the brightness of the main target, the block luma value is positively correlated with the brightness of the main target, the AWB gain is positively correlated with the brightness of the main target, the Y value of the grayscale histogram is positively correlated with the brightness of the main target, the lux index is positively correlated with the brightness of the main target, the color temperature is positively correlated with the brightness of the main target, and the ISO is positively correlated with the brightness of the main target.

[0274] In this way, in black scenes, the AI ​​AE model can adaptively output the subject target brightness according to the 3A statistical data, thereby reducing scenes in which the subject object brightness values ​​are too high in black scenes; and in white scenes, the AI ​​AE model can adaptively output the subject target brightness according to the 3A statistical data, thereby reducing scenes in which the subject object brightness values ​​are too low in white scenes.

[0275] It should be noted that in the embodiments of the present application, the electronic device may be provided with preset conditions 1 to 5, auxiliary detection models and classification models to identify black scenes. The AI ​​AE model may be a model trained based on data of black abnormal scenes. The AI ​​AE model may output the main target brightness in black scenes more accurately.

[0276] Alternatively, the electronic device may also be provided with preset conditions 6-preset conditions 10, auxiliary detection models and classification models for identifying white scenes. The AI ​​AE model may be a model trained based on data of white abnormal scenes. The AI ​​AE model may output the main target brightness in white scenes more accurately.

[0277] Alternatively, the electronic device may also be provided with detection rules for black scenes and white scenes, as well as an AI AE model trained based on black abnormal scenes and an AI AE model trained based on white abnormal scenes; when the electronic device recognizes a black scene, the 3A statistical data is input into the AI ​​AE model corresponding to the black scene to predict the brightness of the main object in the black scene; when the electronic device recognizes a white scene, the 3A statistical data is input into the AI ​​AE model corresponding to the white scene to predict the brightness of the main object in the white scene. The embodiments of the present application do not limit the combination implementation of the above embodiments.

[0278] The above embodiments illustrate the image processing method provided by the embodiments of the present application. Figure 8 and Fig. 9 The software structure and hardware architecture of the electronic device in the embodiments of the present application are described respectively.

[0279] Figure 8 It is a schematic diagram of the architecture (including software system and some hardware) of the embodiment of the present application. Figure 8 As shown, the application architecture is divided into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the application architecture can be divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the hardware abstraction layer HAL, the kernel layer (also called the driver layer, driver layer) and the hardware layer.

[0280] like Figure 8 As shown, the application layer includes the camera and the gallery.

[0281] Understandably, Figure 8 The application layer includes some applications, and in fact, the application layer may also include other applications, which are not limited in this application. For example, the application layer also includes information, alarm clock, weather, stopwatch, compass, timer, flashlight, calendar, Alipay and other applications.

[0282] like Figure 8 As shown, the application framework layer includes a camera access interface. The camera access interface includes camera management and camera devices. The hardware abstraction layer includes a camera hardware abstraction layer and a camera algorithm library. Among them, the camera hardware abstraction layer includes multiple camera devices. The camera algorithm library includes modules related to the exposure adjustment process, such as an AI processing module, a judgment module, a decision module, and an AE algorithm module.

[0283] It should be understood that the decision module can also be placed in other layers. As a possible implementation, the decision module can be placed in the application layer or the application framework layer.

[0284] The kernel layer is used to drive hardware resources. The kernel layer can include multiple driver modules. Figure 8As shown, the kernel layer includes the camera device driver.

[0285] The hardware layer includes sensors and ISP. The sensors include image sensors, TOF cameras, and multi-spectral sensors. The ISP includes the ISP first module and the ISP second module.

[0286] For example, the user can click on the camera application. When the user clicks on the camera to take a photo, the photo taking instruction can be sent to the camera hardware abstraction layer through the camera access interface. The camera hardware abstraction layer calls the camera device driver, and the camera device driver can drive the image sensor to receive light information to obtain the original image and aeLV value; the camera device driver can drive the TOF camera to measure the distance between the main object and the lens; and the camera device driver can drive the multispectral sensor to collect alsLV values. The image sensor passes the original image to the ISP. Among them, after the first module of the ISP obtains the original image, it can extract the 3A statistical information in the original image; the first module of the ISP passes the original image to the second module of the ISP, and the second module of the ISP processes the original raw image into an RGB image or a YUV image. The ISP uploads the RGB image or YUV image to the camera hardware abstraction layer through the camera device driver; the camera access interface reports it to the camera application, and the camera application displays the image.

[0287] In addition, the ISP can also report the 3A statistical information to the camera algorithm library through the camera device driver and the camera hardware abstraction layer. The judgment module determines whether the current scene is the target scene based on the 3A statistical information. When the current scene is the target scene, the judgment module inputs the 3A statistical data into the AI ​​processing module to obtain the predicted subject target brightness; after the decision module obtains the subject target brightness, it outputs EV0 target according to the subject target brightness; the AE algorithm adjusts the exposure parameters of the next frame according to the EV0 target. Alternatively, when the current scene is a normal scene, the judgment module inputs the 3A statistical data into the decision module; the decision module sets the preset EV0 target; the AE algorithm adjusts the exposure parameters of the next frame according to the EV0 target.

[0288] The above describes in detail the software system used in the embodiment of the present application. Fig. 9 The hardware system of the electronic device 100 is described.

[0289] Fig. 9The structure diagram of the terminal device 100 is shown. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, a subscriber identification module (SIM) card interface 195, and an embedded secure element (eSE) 196. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0290] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0291] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0292] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The display screen 194 is used to display images, videos, and the like.

[0293] In the embodiment of the present application, the electronic device 100 displays the RGB image or YUV image after exposure adjustment based on the display screen 194 .

[0294] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0295] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0296] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0297] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0298] In the image processing method provided in the embodiment of the present application, the electronic device 100 can collect 3A statistical data based on the ISP and process the raw image; the electronic device 100 can collect raw images based on the camera 193, and the electronic device can also implement neural network algorithms such as face recognition, subject target brightness prediction, and scene recognition through the computing and processing capabilities provided by the NPU.

[0299] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0300] In the embodiment of the present application, the code for implementing the shooting method described in the embodiment of the present application may be stored in a non-volatile memory. When running the camera application, the electronic device 100 may load the executable code stored in the non-volatile memory into the random access memory. For example, in the photo-taking path, the electronic device 100 may cache the original image through the internal memory 121.

[0301] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focus. In the embodiment of the present application, the distance sensor 180F can be used to detect whether the current scene is a distant scene. For example, the distance sensor 180F can be a TOF camera.

[0302] The image processing method of the embodiment of the present application has been described above, and the device for performing the above image processing method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can perform the steps in the above image processing method.

[0303] like Fig.10 As shown, the image processing apparatus 1000 can be used in a communication device, a circuit, a hardware component or a chip, and the image processing apparatus includes: a display unit 1001 and a processing unit 1002. The display unit 1001 is used to support the display step performed by the image processing apparatus 1000; the processing unit 1002 is used to support the image processing apparatus 1000 to perform the information processing step.

[0304] In a possible implementation, the image processing apparatus 1000 may also include a communication unit 1003. Specifically, the communication unit is used to support the image processing apparatus 1000 to perform the steps of sending and receiving data. The communication unit 1003 may be an input or output interface, a pin or a circuit.

[0305] In a possible embodiment, the image processing device may further include: a storage unit 1004. The processing unit 1002 and the storage unit 1004 are connected via a line. The storage unit 1004 may include one or more memories, and the memory may be a device used to store programs or data in one or more devices or circuits. The storage unit 1004 may exist independently and be connected to the processing unit 1002 of the image processing device via a communication line. The storage unit 1004 may also be integrated with the processing unit 1002.

[0306] The storage unit 1004 can store computer-executable instructions of the method in the terminal device so that the processing unit 1002 executes the method in the above embodiment. The storage unit 1004 can be a register, a cache, or a RAM, etc. The storage unit 1004 can be integrated with the processing unit 1002. The storage unit 1004 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, and the storage unit 1004 can be independent of the processing unit 1002.

[0307] The image processing method provided in the embodiment of the present application can be applied to electronic devices with communication functions. The electronic devices include terminal devices, and the specific device form of the terminal devices can refer to the above related descriptions, which will not be repeated here.

[0308] An embodiment of the present application provides an electronic device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the above method.

[0309] The embodiment of the present application provides a chip system, which includes at least one processor and a communication interface, wherein the communication interface and the at least one processor are interconnected through a line, and the at least one processor is used to run a computer program or instruction to execute the above method. Its implementation principle and technical effect are similar to those of the above-mentioned related embodiments, and will not be repeated here.

[0310] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0311] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0312] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0313] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0314] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: Used in electronic equipment, including: In response to an operation for starting a camera application, using a camera to capture a first frame of raw image; Acquire first 3A statistical data in the first frame of original image; the first 3A statistical data includes two or more different categories of 3A statistical data; When the electronic device photographs a target scene, obtaining a first subject target brightness according to the first 3A statistical data; Obtaining a first target brightness according to the first subject target brightness; Based on the brightness of the first target, using the camera to collect a second frame of original image; Acquire second 3A statistical data in the second frame of original image; the second 3A statistical data includes two or more different categories of 3A statistical data; When the electronic device photographs the target scene, obtaining a second subject target brightness according to the second 3A statistical data; A second target brightness is obtained according to the second subject target brightness.

2. The method according to claim 1, characterized in that The target scene includes a black scene or a white scene, wherein the black scene includes a shooting scene in which a black area accounts for a greater than a first threshold, and the white scene includes a shooting scene in which a white area accounts for a greater than a second threshold.

3. The method according to claim 1 or 2, characterized in that: The first subject target brightness is positively correlated with the first 3A statistical data; the first target brightness is positively correlated with the first subject target brightness; the second subject target brightness is positively correlated with the second 3A statistical data; the second target brightness is positively correlated with the second subject target brightness.

4. The method according to any one of claims 1 to 3, characterized in that: After obtaining the first target brightness according to the first subject target brightness, the method further includes: displaying a first frame of image, where the first frame of image is obtained after the electronic device processes the first frame of original image; wherein the image format of the first frame of original image is different from the image format of the first frame of image; After obtaining the second target brightness according to the second subject target brightness, the method further includes: Display a second frame image; the second frame image is obtained after the electronic device processes the second frame original image; wherein, the image format of the second frame original image is different from the image format of the second frame image; the brightness of the first frame image is different from the brightness of the second frame image; the brightness of the second frame image is related to the brightness of the first frame original image and the brightness of the first target.

5. The method according to claim 4, characterized in that When the target scene is a black scene, the brightness of the second frame image is lower than the brightness of the first frame image; or, when the target scene is a white scene, the brightness of the second frame image is higher than the brightness of the first frame image.

6. The method according to any one of claims 1 to 5, characterized in that: The first 3A statistical data includes at least two of the following: a brightness value LV of the first frame original image, an image block brightness block luma of the first frame original image, a white balance gain coefficient AWB gain of the first frame original image, a grayscale histogram of the first frame original image, a luminance index value lux index of the first frame original image, a color temperature of the first frame original image, and a sensitivity ISO of the first frame original image; Among them, the brightness of the first subject target is positively correlated with the LV of the first frame of original image; the brightness of the first subject target is positively correlated with the block luma of the first frame of original image; the brightness of the first subject target is positively correlated with the AWB gain of the first frame of original image; the brightness of the first subject target is positively correlated with the grayscale histogram of the first frame of original image, and the brightness of the first subject target is positively correlated with the lux index of the first frame of original image; the brightness of the first subject target is positively correlated with the color temperature of the first frame of original image; the brightness of the first subject target is positively correlated with the ISO of the first frame of original image.

7. The method according to any one of claims 1 to 6, characterized in that: Before obtaining the first subject target brightness according to the first 3A statistical data, the method further includes: Detecting the shooting scene of the first frame of original image; The obtaining the first subject target brightness according to the first 3A statistical data includes: When the shooting scene meets a preset condition, obtaining the first subject target brightness according to the first 3A statistical data; Among them, the preset conditions include one or more of the following: the first frame of original image does not include facial information; in the process of acquiring the first frame of original image, the distance between the main object and the camera is less than a distance threshold; the brightness difference between each pixel area in the first frame of original image is less than a brightness difference threshold; in the first frame of original image, the brightness of the main object is greater than a brightness threshold; in the first frame of original image, the ratio of the maximum color level gain coefficient to the minimum color level gain coefficient is less than a third threshold.

8. The method according to claim 7, characterized in that The brightness difference of each pixel area in the first frame of the original image is obtained by the ambient light brightness value alsLV of the multispectral sensor and the ambient light brightness value aeLV of the image sensor, and the brightness difference threshold includes a first brightness difference threshold and a second brightness difference threshold; the brightness difference of each pixel area is less than the brightness difference threshold, including: the difference between the alsLV and the aeLV is less than the first brightness difference threshold, and / or the difference between the alsLV and the aeLV is less than or equal to the second brightness difference threshold; the first brightness difference threshold is less than the second brightness difference threshold.

9. The method according to claim 8, characterized in that During the process of acquiring the first frame of original image, the difference between the alsLV and the aeLV of the first frame of original image is less than the first brightness difference threshold; during the process of acquiring the second frame of original image to the m-th frame of original image, the difference between the alsLV and the aeLV of the second frame of original image to the m-th frame of original image is less than the second brightness difference threshold; during the process of acquiring the m+1-th frame of original image, the difference between the alsLV and the aeLV of the m+1-th frame of original image is greater than the second brightness difference threshold; wherein, the shooting scenes of the second frame of original image to the m-th frame of original image are the target scene, and the shooting scene of the m+1-th frame of original image is not the target scene; and m is a positive integer greater than 2.

10. The method according to any one of claims 1 to 9, characterized in that: The step of obtaining the first subject target brightness according to the first 3A statistical data further includes: Inputting the first 3A statistical data into a first neural network model to obtain the first subject target brightness; Among them, when the target scene is a black scene, the brightness of the first main target is less than a first preset brightness, and the first preset brightness is a brightness obtained based on a grayscale histogram of the first frame original image; or, when the target scene is a white scene, the brightness of the first main target is greater than the first preset brightness.

11. The method according to claim 10, characterized in that The first neural network model is trained by the following method: obtaining a training data set, the training data set including multiple samples, any of the samples including sample main target brightness and multiple sample 3A statistical data, the multiple sample 3A statistical data corresponding to multiple sample images with different exposure values ​​under the same target scene; for any of the samples, inputting the multiple sample 3A statistical data into the model to be trained to obtain the predicted main target brightness; completing the training when the value calculated by the loss function converges, and obtaining the first neural network model.

12. The method according to any one of claims 1 to 11, characterized in that: The method of using the camera to collect a second frame of original image based on the first target brightness includes: Adjusting the exposure value of the camera according to the difference between the brightness of the first target and the brightness of the first frame of the original image; wherein, when the brightness of the first target is greater than the brightness of the first frame of the original image, the exposure value after adjustment is greater than the exposure value before adjustment; or, when the brightness of the first target is less than the brightness of the first frame of the original image, the exposure value after adjustment is less than the exposure value before adjustment; The second frame of original image is acquired using the camera; in the process of acquiring the second frame of original image, the exposure value of the camera is the adjusted exposure value.

13. The method according to any one of claims 1 to 12, characterized in that: After obtaining the second target brightness according to the second subject target brightness, the method includes: Based on the brightness of the n-1th target, the camera is used to collect the nth frame of the original image; n is a positive integer greater than 2 to obtain the nth 3A statistical data in the nth frame of the original image; When the electronic device photographs the target scene, an nth subject target brightness is obtained according to the nth 3A statistical data; the nth subject target brightness is positively correlated with the nth 3A statistical data; The nth target brightness is obtained according to the nth main target brightness; the nth target brightness is positively correlated with the nth main target brightness; and the brightness of the nth frame original image converges to the nth target brightness.

14. The method according to claim 13, characterized in that After obtaining the nth target brightness according to the nth subject target brightness, the method further includes: An operation for taking a photo is received; In response to the operation for taking a picture, using the camera to collect and cache the original image from the n+1th frame to the n+ith frame; The n+2th frame original image is post-processed to obtain a target image; the n+2th frame original image is an original image with the smallest difference between the target brightness and the brightness of the original image among the n+1th frame original image to the n+ith frame original image; the original image includes a raw image, and the target image includes an RGB image or a YUV image; the clarity of the target image is higher than that of the n+2th frame image, and the n+2th frame image is an image displayed after processing the n+2th frame original image; the n+2th frame image includes an RGB image or a YUV image; The target image is saved to a gallery application.

15. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 14.

16. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

17. A chip system, characterized in that: The method comprises at least one processor and a communication interface, wherein the communication interface and the at least one processor are interconnected via a line, and the at least one processor is used to run a computer program or instruction to execute the method according to any one of claims 1 to 14.

18. A computer program product, characterized in that The method comprises a computer program, which, when being executed, enables a computer to execute the method according to any one of claims 1 to 14.

Images