Image processing method and related device

Abstract

Embodiments of the present application provide an image processing method and related equipment, which relate to the technical field of terminals. The subject target brightness is adjusted to reduce the problem of overexposure in a black scene and underexposure in a white scene. Specifically, in response to an operation for starting a camera application, a first frame of original image is collected; first 3A statistical data is obtained, the first 3A statistical data including two or more different categories of 3A statistical data; in the case that the electronic device photographs a target scene, a first subject target brightness is obtained according to the first 3A statistical data; a first target brightness is obtained according to the first subject target brightness; and a second frame of original image is collected using the camera based on the first target brightness. In this way, in the target scene, the electronic device can obtain a target brightness suitable for the target scene through the subject target brightness; and the target brightness of the previous frame is used to continuously adjust the exposure value of the next frame, so that a high-quality image that is not overexposed and underexposed is obtained, and the user's photographing experience is improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image processing method and related equipment. Background Technology

[0002] Electronic devices can provide a variety of services. For example, electronic devices can provide users with services such as taking photos and recording videos based on camera applications.

[0003] In some implementations, when using electronic devices to capture images of black, dark, or white scenes, there may be significant color differences between the photographed object in the image and the actual object; for example, the subject may be too bright in a black scene or too dark in a white scene. This results in poor image quality from the electronic device, degrading the user's photography experience. Summary of the Invention

[0004] This application provides an image processing method and related equipment, which are applied in the field of terminal technology. In shooting scenarios, it can reduce the problems of overexposure in black scenes and underexposure in white scenes.

[0005] In a first aspect, embodiments of this application propose an image processing method; the method includes: in response to an operation for launching a camera application, acquiring a first frame of raw image using a camera; obtaining first 3A statistical data in the first frame of raw image; when an electronic device is capturing a target scene, obtaining a first subject target brightness based on the first 3A statistical data, wherein the first 3A statistical data includes two or more different categories of 3A statistical data; obtaining a first target brightness based on the first subject target brightness; acquiring a second frame of raw image using the camera based on the first target brightness; acquiring second 3A statistical data in the second frame of raw image; when an electronic device is capturing a target scene, obtaining a second subject target brightness based on the second 3A statistical data; and obtaining a second target brightness based on the second subject target brightness.

[0006] After the electronic device launches the camera application, the camera can capture raw images in real time and display the processed images. The shooting scene can be a target scene, which can be a black scene or a white scene as described in this embodiment. In the target scene, the electronic device can execute the image processing method provided in this embodiment.

[0007] The first subject target brightness can be predicted by the AI ​​AE model in this embodiment based on the first 3A statistical data, and the target brightness can correspond to the EV0 target (also called the AE target); the first target brightness can be the EV0 target obtained with reference to the first subject target brightness. The second subject target brightness can be predicted by the AI ​​AE model in this embodiment based on the second 3A statistical data, and the second target brightness can be the EV0 target obtained with reference to the second subject target brightness.

[0008] In this embodiment, the electronic device can use the first target brightness to adjust the exposure value during the second frame acquisition process, thereby achieving the effect of adjusting the exposure brightness of the next frame original image using the 3A statistical data of the previous frame original image.

[0009] Specifically, in a possible implementation, the EV0 target is a preset value obtained from the ISO sensitivity in the 3A statistics. This preset value is applicable to normal scenes, but in black scenes, electronic devices will still use the preset value. When the reflectivity of black areas is low, converging the brightness of the image to the preset value will result in overexposure. Similarly, in white scenes, converging the brightness of the image to the preset value will result in underexposure.

[0010] In this embodiment, the electronic device uses various 3A statistical data to predict the brightness of a subject target, and the EV0 target can be adjusted based on the subject target brightness. For example, in a black scene, if the AI ​​AE model outputs a low subject target brightness based on the 3A statistical data, the EV0 target is reduced, making the EV0 target in this embodiment less than a preset value, thereby reducing overexposure in black scenes. In a white scene, if the AI ​​AE model outputs a high subject target brightness based on the 3A statistical data, the EV0 target is increased, making the EV0 target in this embodiment greater than a preset value, thereby reducing underexposure in white scenes.

[0011] In this way, when the shooting scene is the target scene, the electronic device can obtain the target brightness that is adapted to the target scene through the brightness of the main target; and continuously adjust the exposure value of the next frame using the target brightness of the previous frame, so as to obtain a high-quality image that is neither overexposed nor underexposed, thus improving the user's shooting experience.

[0012] Optionally, the target scene includes a black scene or a white scene, wherein a black scene includes a shooting scene in which the proportion of black area is greater than a first threshold, and a white scene includes a shooting scene in which the proportion of white area is greater than a second threshold.

[0013] The first threshold can be the same as or different from the second threshold. In this way, the electronic device can use the image processing method provided in the embodiments of this application in either black or white scenes to obtain high-quality images that are not overexposed in black scenes and not underexposed in white scenes.

[0014] Optionally, the brightness of the first main target is positively correlated with the first 3A statistical data; the brightness of the first target is positively correlated with the brightness of the first main target; the brightness of the second main target is positively correlated with the second 3A statistical data; and the brightness of the second target is positively correlated with the brightness of the second main target.

[0015] It is understood that in the embodiments of this application, the subject target brightness is positively correlated with 3A statistical data; the target brightness is positively correlated with the subject target brightness; enabling the electronic device to obtain the EV0 target through various 3A statistical data. For example, in a black scene, a lower subject target brightness can be obtained, thereby reducing the EV0 target. In a white scene, a higher subject target brightness can be obtained, thereby increasing the EV0 target. In this way, by using the mapping relationship between 3A statistical data and subject target brightness, and the mapping relationship between subject target brightness and EV0 target, the electronic device can be controlled to obtain an image that is not overexposed in a black scene, and an image that is not underexposed in a white scene.

[0016] Optionally, after obtaining the first target brightness based on the first subject target brightness, the method further includes: displaying a first frame image, wherein the first frame image is obtained by processing the first frame original image by an electronic device; wherein the image format of the first frame original image is different from the image format of the first frame image; after obtaining the second target brightness based on the second subject target brightness, the method further includes: displaying a second frame image; wherein the second frame image is obtained by processing the second frame original image by an electronic device; wherein the image format of the second frame original image is different from the image format of the second frame image; the brightness of the first frame image is different from the brightness of the second frame image; and the brightness of the second frame image is related to the brightness of the first frame original image and the first target brightness.

[0017] This process can correspond to Figure 3 In the preview process, electronic devices can use 3A statistics to adjust the exposure of the next frame, or process the original image and display the preview image on the screen.

[0018] For example, the first original image and the second original image can correspond to Figure 3In the first to sixth original images (6), any two adjacent original images can be raw images. After obtaining the first original image, it can be processed by the first and second ISP modules to obtain the second original image, which is then displayed on the screen. The first original image can be a YUV or RGB image. After obtaining the second original image, it can be processed by the first and second ISP modules to obtain the second original image, which is then displayed on the screen. The second original image can be a YUV or RGB image.

[0019] Understandably, the brightness of the first frame of the original image can be the average brightness of each pixel. After acquiring the first frame, the brightness of the first frame and the first target brightness can be obtained. The electronic device can adjust the average brightness of the original image to converge towards the target brightness. This causes the exposure value corresponding to the brightness difference between the first frame and the first target brightness to be applied to the second frame, thus affecting the brightness of the second frame. The brightness of the second frame is different from the brightness of the first frame.

[0020] It should be noted that electronic devices can adjust the exposure value through multiple frames of raw images, so the brightness of the second frame image is different from that of the first frame image. However, there will be no phenomenon where the brightness difference between the second frame image and the first frame image is too large, which will affect the user's shooting experience.

[0021] In this way, the electronic device can continuously display the image with the adjusted exposure value on the screen, so that the screen presents a high-quality image that is neither overexposed nor underexposed.

[0022] Optionally, if the target scene is a black scene, the brightness of the second frame image is less than the brightness of the first frame image; or, if the target scene is a white scene, the brightness of the second frame image is greater than the brightness of the first frame image.

[0023] In this way, in a black scene, electronic devices can reduce the brightness value of the image to avoid overexposure; in a white scene, electronic devices can increase the brightness value of the image to avoid underexposure.

[0024] Optionally, the first 3A statistical data includes one or more of the following: the luminance value LV of the first frame original image, the block luma of the first frame original image, the white balance gain coefficient AWB of the first frame original image, the grayscale histogram of the first frame original image, the lux index of the first frame original image, the color temperature of the first frame original image, and the ISO of the first frame original image.

[0025] Specifically, the brightness of the first subject target is positively correlated with the LV of the first frame original image; the brightness of the first subject target is positively correlated with the block luma of the first frame original image; the brightness of the first subject target is positively correlated with the AWB gain of the first frame original image; the brightness of the first subject target is positively correlated with the grayscale histogram of the first frame original image; the brightness of the first subject target is positively correlated with the lux index of the first frame original image; the brightness of the first subject target is positively correlated with the color temperature of the first frame original image; and the brightness of the first subject target is positively correlated with the ISO of the first frame original image.

[0026] In this way, electronic devices can obtain the brightness of the main target in the target scene more accurately through multiple 3A statistical data, thereby improving the accuracy of the EVO target and reducing the problems of underexposure and overexposure in the target scene.

[0027] Optionally, before obtaining the brightness of the first subject target based on the first 3A statistical data, the method further includes: detecting the shooting scene of the first frame original image; obtaining the brightness of the first subject target based on the first 3A statistical data includes: obtaining the brightness of the first subject target based on the first 3A statistical data when the shooting scene meets preset conditions; wherein, the preset conditions include one or more of the following: the first frame original image does not contain facial information; during the acquisition of the first frame original image, the distance between the subject object and the camera is less than a distance threshold; the brightness difference of each pixel region in the first frame original image is less than a brightness difference threshold; the brightness of the subject object in the first frame original image is greater than a brightness threshold; the ratio of the maximum color gain coefficient to the minimum color gain coefficient in the first frame original image is less than a third threshold.

[0028] The following conditions are considered in the original image: The first frame does not contain facial information, which corresponds to preset condition 1 and / or preset condition 6. During the acquisition of the first frame, the distance between the subject and the camera is less than a distance threshold, which corresponds to preset condition 2 and / or preset condition 7. The brightness difference between pixel regions in the first frame is less than a brightness difference threshold, which corresponds to preset condition 3 and / or preset condition 8. The brightness of the subject in the first frame is less than a brightness threshold, which corresponds to preset condition 4 and / or preset condition 9. The ratio of the maximum to the minimum color gain coefficient in the first frame is less than a third threshold, which corresponds to preset condition 5 and / or preset condition 10.

[0029] In this way, the electronic device can more accurately identify the target scene based on the above-mentioned preset conditions, improve the accuracy of the image processing method in this application embodiment, and avoid the problems of underexposure or overexposure caused by using the image processing method in this application embodiment in ordinary scenes.

[0030] Optionally, the brightness difference of each pixel region in the first frame of the original image is obtained using the ambient light brightness value (alsLV) from the multispectral sensor and the ambient light brightness value (aeLV) from the image sensor. The brightness difference thresholds include a first brightness difference threshold and a second brightness difference threshold. A brightness difference of less than the brightness difference threshold for each pixel region includes: the difference between alsLV and aeLV being less than the first brightness difference threshold, and / or the difference between alsLV and aeLV being less than or equal to the second brightness difference threshold; and the first brightness difference threshold being less than the second brightness difference threshold. This dual thresholding reduces scenarios with abrupt brightness changes, thereby improving the stability of subsequent AI AE model operation and the accuracy of outputting the brightness of the main target.

[0031] Optionally, during the acquisition of the first original image, the difference between alsLV and aeLV of the first original image is less than the first brightness difference threshold; during the acquisition of the second original image to the m-th original image, the difference between alsLV and aeLV of the second original image to the m-th original image is less than the second brightness difference threshold; during the acquisition of the (m+1)-th original image, the difference between alsLV and aeLV of the (m+1)-th original image is greater than the second brightness difference threshold; wherein, the shooting scene of the second original image to the m-th original image is the target scene, and the shooting scene of the (m+1)-th original image is not the target scene; m is a positive integer greater than 2.

[0032] This process corresponds to the determination process in step S403, from the first original image 1 to the fourth original image 4. If any frame of the multiple original images is less than the first brightness difference threshold, then the multiple original images after that frame enter the determination process of the target scene; until a certain frame is greater than the second brightness difference threshold, then the original images after that frame no longer enter the determination process of the target scene.

[0033] In this way, the use of dual thresholds can reduce scenes with sudden changes in brightness, thereby improving the stability of subsequent AI AE model operation and the accuracy of outputting the brightness of the main target.

[0034] Optionally, obtaining the first subject target brightness based on the first 3A statistical data further includes: inputting the first 3A statistical data into a first neural network model to obtain the first subject target brightness; wherein, when the target scene is a black scene, the first subject target brightness is less than a first preset brightness, and the first preset brightness is the brightness obtained based on the grayscale histogram of the original image of the first frame; or, when the target scene is a white scene, the first subject target brightness is greater than the first preset brightness.

[0035] The first neural network model can be an AI AE model, and the first preset brightness can be the brightness obtained from the grayscale histogram of the first frame of the original image. Understandably, in a black scene, the brightness of the main subject output by the AI ​​AE model is generally lower than the first preset brightness to reduce the EV0 target, thus preventing overexposure in black scenes. In a white scene, the brightness of the main subject output by the AI ​​AE model is generally higher than the first preset brightness to increase the EV0 target, thus preventing underexposure in white scenes. In this way, the electronic device obtains the main subject brightness based on the AI ​​AE model and then adjusts the EV0 target accordingly, reducing overexposure in black scenes and underexposure in white scenes.

[0036] Optionally, the first neural network model is trained using the following method: A training dataset is acquired, comprising multiple samples. Each sample includes the brightness of the main target and multiple sample 3A statistical data points, which correspond to multiple sample images with different exposure values ​​in the same target scene. For any sample, the multiple sample 3A statistical data points are input into the model to be trained to obtain the predicted brightness of the main target. Training is complete when the values ​​calculated using the loss function converge, resulting in the first neural network model. In this way, the electronic device can obtain a trained AI AE model, which has the ability to output the brightness of the main target based on the sample 3A statistical data, thus improving the accuracy of the main target brightness.

[0037] Optionally, acquiring a second frame of the original image using a camera based on the first target brightness includes: adjusting the camera's exposure value according to the difference between the first target brightness and the brightness of the first frame of the original image; wherein, if the first target brightness is greater than the brightness of the first frame of the original image, the adjusted exposure value is greater than the original exposure value; or, if the first target brightness is less than the brightness of the first frame of the original image, the adjusted exposure value is less than the original exposure value; acquiring the second frame of the original image using the camera; and during the acquisition of the second frame of the original image, the camera's exposure value is the adjusted exposure value. In this way, the electronic device can use the 3A statistical data of the previous frame to adjust the exposure value of the next frame, thereby achieving convergence to the EV0 target.

[0038] Optionally, after obtaining the second target brightness based on the second main target brightness, the process includes: capturing the nth frame of the original image using a camera based on the (n-1)th target brightness; obtaining the nth 3A statistical data in the nth frame of the original image where n is a positive integer greater than 2; obtaining the nth main target brightness based on the nth 3A statistical data when the electronic device is capturing the target scene; the nth main target brightness is positively correlated with the nth 3A statistical data; obtaining the nth target brightness based on the nth main target brightness; the nth target brightness is positively correlated with the nth main target brightness; and the brightness of the nth frame of the original image converges to the nth target brightness.

[0039] Understandably, during the acquisition of the original image, the electronic device can continuously adjust the exposure value and target brightness to bring the brightness of the original image closer to the target brightness, achieving a good exposure effect. For example, when the electronic device acquires the nth frame of the original image, the brightness of the nth frame of the original image is the same as or within a certain range of the nth target brightness, and the brightness of the nth frame of the original image converges to the nth target brightness.

[0040] In this way, electronic devices can adjust the exposure value to bring the brightness of the original image closer to the EV0 target, achieving a good exposure effect and ensuring that black scenes are not overexposed and white scenes are not underexposed.

[0041] Optionally, after obtaining the nth target brightness based on the nth subject target brightness, the method further includes: receiving an operation for taking a picture; in response to the operation for taking a picture, using a camera to capture and cache the (n+1)th to (n+1)th original images; post-processing the (n+2)th original image to obtain the target image; the (n+2)th original image is the original image with the smallest difference between the target brightness and the brightness of the original image among the (n+1)th to (n+1)th original images; the original image includes a raw image, and the target image includes an RGB image or a YUV image; the clarity of the target image is higher than the clarity of the (n+2)th frame image, and the (n+2)th frame image is the image obtained by processing the (n+2)th original image and then displayed; the (n+2)th frame image includes an RGB image or a YUV image; and saving the target image to a gallery application.

[0042] This process can be a photo-taking process, where the action of taking a photo can be the user pressing and releasing the shutter button; after releasing, the electronic device can start processing the original image based on the up event and generate a photo to be saved in the gallery application.

[0043] The target image can be, for example Figure 3 In the illustrated embodiment's image capture process, image 5, for example, is obtained by the electronic device from the fourth raw image 4 to the raw image 6. If the brightness of the fifth raw image 5 converges to near the EV0 target, further processing is performed on the raw image. The (n+2)th frame image can, for example... Figure 3 Image 5 is shown in the preview process of the illustrated embodiment. It is understood that during the image capture process, the electronic device performs post-processing on image 5 using a post-processing algorithm module to further improve image quality, such as sharpness.

[0044] In this way, by continuously adjusting the exposure value during the preview process, the electronic device can obtain an image with better exposure during the shooting process, thereby improving the quality of the photo and enhancing the user experience.

[0045] Secondly, embodiments of this application provide an electronic device, which may also be referred to as a terminal device, terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. Terminal devices can be mobile phones, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on.

[0046] The electronic device includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the electronic device to perform the method as described in the first aspect.

[0047] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method as described in the first aspect.

[0048] Fourthly, embodiments of this application provide a computer program product, which includes a computer program that, when run, causes a computer to perform the method as described in the first aspect.

[0049] This application provides a chip system including at least one processor and a communication interface. The communication interface and the at least one processor are interconnected via a line. The at least one processor is used to run computer programs or instructions to perform the method as described in the first aspect.

[0050] It should be understood that the second to fifth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0051] Figure 1 This is a schematic diagram illustrating possible shooting scenarios where a black scene is overexposed and a white scene is underexposed.

[0052] Figure 2 This is a schematic diagram of the exposure process in a possible implementation;

[0053] Figure 3 A schematic diagram of an exposure process provided in an embodiment of this application;

[0054] Figure 4 A flowchart illustrating a process for recognizing a black scene is provided in an embodiment of this application;

[0055] Figure 5 A schematic diagram illustrating another process for recognizing a dark scene provided in an embodiment of this application;

[0056] Figure 6 A schematic diagram of a process for recognizing a white scene provided in an embodiment of this application;

[0057] Figure 7 A schematic diagram of the training process of an AI AE model provided in an embodiment of this application;

[0058] Figure 8 A software structure block diagram of an electronic device 100 provided in this application embodiment;

[0059] Figure 9 This is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of this application;

[0060] Figure 10 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Detailed Implementation

[0061] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0062] 1) 3A Algorithm: This includes three algorithms for automatically adjusting camera parameters: auto focus (AF), automatic exposure (AE), and automatic white balance (AWB). The 3A algorithm helps the camera automatically adjust parameters according to environmental conditions, thereby achieving better image quality and results.

[0063] 2) Electronic equipment

[0064] The terminal device in this application embodiment can also be any form of electronic device. For example, electronic devices may include handheld devices with image processing functions, vehicle-mounted devices, etc. For example, some electronic devices include: mobile phones, tablets, PDAs, laptops, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving vehicles, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, in-vehicle devices, wearable devices, terminal devices in 5G networks, or future evolution of public land mobile communication networks. Terminal devices in a network (PLMN), etc., are not limited to this in the embodiments of this application.

[0065] By way of example and not limitation, in this embodiment, the electronic device can be a wearable device, which may also be called a wearable smart device. Wearable devices can be, for example, smartwatches, smart glasses, smart bracelets, and smart jewelry. In this embodiment, the electronic device can also be a terminal device in an Internet of Things (IoT) system.

[0066] The electronic devices in the embodiments of this application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device, etc.

[0067] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "exemplary" or "for example" are used in the embodiments of this application to indicate that they are examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0068] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0069] It should be noted that the phrase "at...time" in the embodiments of this application can refer to the instant at which a certain situation occurs, or to a period of time after the occurrence of a certain situation; the embodiments of this application do not specifically limit this. Furthermore, the display interface provided in the embodiments of this application is merely an example, and the display interface may include more or less content.

[0070] In a possible implementation, the electronic device can take a picture of the object being photographed. The photographing scene can include a preview scene and a photographing scene. The preview scene can be understood as: the electronic device displays the image captured by the camera, and does not save the corresponding image. The photographing scene can be understood as: the electronic device displays the image captured by the camera and saves the corresponding image. In the photographing scene, the electronic device can obtain one or more images including the object being photographed. The object being photographed can be a person, an animal, or a background such as the sky or snow.

[0071] The following example uses a preview scene, combined with... Figure 1 This document describes potential issues such as objects being too bright in black scenes and objects being too dark in white scenes.

[0072] In a possible implementation, a black scene could correspond to Figure 1 The scene shown in Figure a includes a black scene that may include a little girl 101, a stage 102, and a black background 103. In this black scene, an electronic device receives an operation to open a camera application; in response to this operation, the electronic device can display a preview of the camera application interface within the scene. The preview of the camera application interface can be as follows: Figure 1 The interface shown in b is shown in the image. A comparison between the scene and the image displayed by the electronic device reveals that when the electronic device takes a picture in a black scene, the object in the displayed image is brighter than the object in the real scene. For example, in a black scene, the brightness of the little girl 101, the stage 102, and the black background 103 is lower than the brightness of the little girl 101, the stage 102, and the black background 103 displayed by the electronic device.

[0073] In a possible implementation, the white scene could correspond to Figure 1 The scene shown in Figure c includes a white scene that may include a little girl 104, snow 105, and a white background 106. In the black scene, the electronic device receives an operation to open the camera application; in response to this operation, the electronic device can display the camera application interface in the preview scene. The camera application interface in the preview scene can be as follows: Figure 1 The interface shown in 'c' illustrates this. A comparison of the scene and the image displayed by the electronic device reveals that when the electronic device takes a picture in a white scene, the objects in the displayed image are darker than those in the actual scene. For example, in the white scene, the brightness of the little girl 101, the stage 102, and the black background 103 are all higher than the brightness of the little girl 101, the stage 102, and the black background 103 displayed by the electronic device.

[0074] This is because electronic devices support AE (After Effects) functionality, which automatically adjusts the exposure. Specifically, when shooting a black scene and / or a black object, if the reflectivity of the object is below a reflectivity threshold (e.g., 18%), the AE module can increase the exposure, allowing the electronic device to clearly capture the black scene and / or object. However, if a non-black object (e.g., a little girl) is present in a black scene, the exposure applied to that object increases, resulting in overexposure and an overly bright image.

[0075] Similarly, when shooting a white scene and / or a white object, because the reflectivity of the object being photographed is below the reflectivity threshold (e.g., 18%), the AE module can reduce the exposure so that the electronic device can clearly capture the white scene and / or the white object; however, when there is a non-white object being photographed in the white scene (e.g., a little girl 104), the exposure applied to the object being photographed is reduced, resulting in underexposure of the image and the object being photographed in the image being too dark.

[0076] The following is combined with Figure 2 The preview and photo-taking processes for electronic devices in possible implementations are described.

[0077] Figure 2 This is a schematic block diagram of a possible implementation of a camera system 200. For example, the camera system 200 includes a camera, a first image signal processor (ISP) module, a second ISP module, a decision module, an AE algorithm module, and a cache module.

[0078] The camera is used to capture raw images, such as RAW images. The first ISP module can be used to obtain 3A statistical data from the raw images. Specifically, the first ISP module is equipped with hardware devices such as multispectral sensors and image sensors, as well as software algorithms related to AE, AF, and AWB. The first ISP module can achieve the function of obtaining 3A statistical data through software and hardware collaborative processing.

[0079] The second ISP module processes the raw image, converting it into an RGB or YUV image. The decision module may include a target brightness (EV0 target, also known as an AE target) decision module. The EV0 target decision module determines the EV0 target of the current frame's brightness, ensuring the average brightness of the output image is close to the EV0 target. The AE algorithm module adjusts the exposure value of the next frame based on the relationship between the current frame's brightness and the EV0 target. A caching module stores the raw image data. A post-processing algorithm module can be used for offline image processing in shooting mode.

[0080] The camera sequentially captures multiple frames of raw images (such as...) Figure 2 Taking the first original image 1 to the sixth original image 6 as an example, in the preview scene, the preview path can be: the camera captures a frame of the first original image 1 and transmits the first original image 1 to the first module of the ISP; the first module of the ISP obtains the 3A statistical data in the first original image 1, transmits the 3A statistical data to the decision module, and transmits the first original image 1 to the second module of the ISP.

[0081] After receiving the first raw image 1, the second module of the ISP can process the first raw image 1 and output image 1; the electronic device displays image 1, which is an RGB image or a YUV image.

[0082] After obtaining the 3A statistical data, the decision module can calculate the EV0 target. If the brightness of the first original image 1 is close to the EV0 target, the AE algorithm module does not need to adjust the exposure value; that is, before the camera captures the second original image 2, the AE algorithm module does not adjust the exposure value, and the camera can capture the second original image 2 with the same exposure value as the first original image 1.

[0083] When the brightness of the first original image 1 is not close to the EV0 target, the AE algorithm module can adjust the exposure value; that is, before the camera captures the second original image 2, the AE algorithm module adjusts the exposure value, and the camera can use the new exposure value to capture the second original image 2; the electronic device can continue to adjust the exposure value according to the 3A statistics of the second original image 2... and so on, until the brightness of a certain frame converges to the EV0 target.

[0084] When the camera captures the fourth raw image 4, the electronic device receives an operation for taking a picture on the camera interface, such as pressing and releasing the shutter button. At this time, the electronic device recognizes the release (up) event and, while transmitting the fourth raw image 4 to the preview stream for display, begins the picture taking process.

[0085] In the photo-taking scenario, the photo-taking path can be as follows: upon receiving an up event, the fourth raw image 4 is passed to the cache module. Subsequently, the camera can capture the fifth raw image 5 and the sixth raw image 6; the electronic device transmits the fifth raw image 5 and the sixth raw image 6 to the preview stream for display, and also transmits the fifth raw image 5 and the sixth raw image 6 to the cache module.

[0086] The caching module can cache one or more original images. The electronic device can select the original image whose exposure brightness converges to the EV0 target from among the multiple original images, process this original image, and save it to the image library application. For example, among the fourth original image 4 to the sixth original image 6, the original image with converged exposure brightness is the fifth original image 5. The electronic device then transmits the fifth original image 5 through the first ISP module to the second ISP module and the post-processing algorithm module for processing. The post-processing algorithm module can be used to further improve image quality and includes, for example, image enhancement, super-resolution reconstruction, and multi-frame fusion algorithms. The electronic device can save image 5 in the image library application; image 5 can be an RGB image or a YUV image.

[0087] It is understood that the data flow between the ISP second module and the post-processing algorithm module can be reciprocal; for example, the data flow can pass through the ISP first module and the ISP second module to the post-processing algorithm module; the data flow can also pass through the ISP first module and the post-processing algorithm module to the ISP second module; the data flow can also pass through the ISP second module and the post-processing algorithm module multiple times after exiting the ISP first module. This application embodiment does not impose any limitations on this.

[0088] The recording process can be referred to in the relevant descriptions during the preview process, and will not be repeated here.

[0089] In a possible implementation, the EV0 target is a preset value. For example, the EV0 target can be an exposure value that makes the average reflectance of the current scene equal to a reflectance threshold. For example, with a reflectance of 18%, for a black scene with a reflectance below 18%, the AE module will increase the exposure value to converge the brightness towards the EV0 target; at this time, the main object in the black scene (e.g., Figure 1 The little girl (101) in the interface shown in b) is overexposed. For white scenes with a reflectivity higher than 18%, the AE module will reduce the exposure value to converge the brightness towards the EV0 target; at this time, the main object in the white scene (e.g., Figure 1 The little girl (104) in the interface shown in d has an underexposure problem.

[0090] In view of this, embodiments of this application provide an image processing method in which an electronic device can adaptively adjust the brightness of the subject target according to the scene. For example, the EV0 target is positively correlated with the brightness of the subject target. In a black scene, the brightness of the subject target is appropriately reduced, thereby reducing the EV0 target and exposure value in the black scene, solving the problem of overexposure of the subject object in a black scene. In a white scene, the brightness of the subject target is appropriately increased, thereby increasing the EV0 target and exposure value in the white scene, solving the problem of underexposure of the subject object in a white scene. In a normal scene, the EV0 target is not adjusted. In this way, the electronic device can adjust the exposure value in target scenes such as white and black scenes, so that the exposure value of the subject object in the output image is within the normal range, and there will be no overexposure or underexposure problems.

[0091] Let's combine the following... Figure 3 The preview and photo-taking processes of the electronic devices in the embodiments of this application are described.

[0092] Figure 3 This is a schematic block diagram of a photography system 300 provided in an embodiment of this application. Exemplarily, the photography system 300 includes a camera, a first ISP module, a second ISP module, a decision module, an AE algorithm module, a cache module, a judgment module, and an AI processing module.

[0093] It should be understood that the ISP first module and ISP second module involved in the preview stream may be partially or fully reused with the ISP first module and ISP second module involved in the photo stream, or they may be independent of each other. This application embodiment does not limit this.

[0094] For example, a preview stream corresponds to one set of ISP first module and ISP second module; a photo stream corresponds to one set of ISP first module and ISP second module. Alternatively, the preview stream and photo stream may share the same set of ISP first module and ISP second module. It is understood that the above descriptions of the ISP first module and ISP second module are merely illustrative, and the embodiments of this application are not limited thereto.

[0095] For example, a camera is used to capture raw images, such as raw images. Specifically, the camera may include a lens. In some embodiments, light is converged through the lens to an image sensor, which includes multiple photosensitive elements that convert light signals into electrical signals, which are then transmitted to an ISP for conversion into digital image signals.

[0096] The first module of the ISP can be used to acquire 3A statistical data from the original image. These 3A statistical data may include, for example, lighting value (LV), block luma, automatic white balance gain (AWB gain), grayscale histogram, lux index, color temperature (CCT), and ISO sensitivity. It is understood that this application embodiment only shows some of the 3A statistical data related to brightness; the first module of the ISP in this application embodiment may acquire more or less 3A statistical data, and this application embodiment does not impose any limitations on this.

[0097] The second module of the ISP is used to process the raw image, converting it into an RGB or YUV image. For example, the second module may include a demosaicing module, a color correction module, a global tone mapping module, a gamma module, and a color space transform module. After processing by these modules, the second module of the ISP can output an RGB or YUV image.

[0098] It should be noted that the embodiments of this application only exemplarily illustrate the relationship between the first ISP module and the second ISP module. In actual scenarios, electronic devices may first process the image through some algorithms in the second ISP module so that the first ISP module can obtain 3A statistical data from the processed image. The embodiments of this application do not restrict the order in which the first ISP module and the second ISP module run.

[0099] The judgment module is used to determine the scene of the current frame image. For example, the scene may include a normal scene, a black scene, a white scene, and a face scene. In this embodiment, the judgment module can output corresponding results based on the scene recognition results; for example, in normal scenes and face scenes, the judgment module can transmit 3A statistical data to the decision module; in black scenes and / or white scenes, the judgment module can output 3A statistical data to the AI ​​processing module.

[0100] It should be noted that, in the embodiments of this application, a black scene can be understood as a black and / or dark scene, and a white scene can be understood as a white and / or light scene. The image processing method provided in the embodiments of this application is not only applicable to scenes involving pure black and pure white. For example, a black scene includes shooting scenes where the proportion of black area is greater than a first threshold, and a white scene includes shooting scenes where the proportion of white area is greater than a second threshold; the first threshold may be the same as or different from the second threshold. For example, the first threshold and the second threshold may be, for example, 80%. The embodiments of this application do not limit the specific values ​​of the first threshold and the second threshold.

[0101] The AI ​​processing module may include an AI processing model (also referred to as an AI AE model). In this embodiment, the AI ​​processing module may have a pre-set neural network model. This neural network model can take 3A statistical data as input, process it through pre-trained parameters, and output an indicator of the brightness of the main target in the current scene. It is understood that this model has the ability to adaptively output the brightness of the main target based on the 3A statistical data.

[0102] For example, in a black scene, after obtaining the 3A statistical data, the AI ​​processing module can output the brightness of the main target. The brightness of the main target is less than a preset brightness, which can be a brightness value obtained based on the grayscale histogram of the original image. Since the EV0 target in possible implementations is related to the preset brightness, and in this embodiment, the brightness of the main target is less than the preset brightness, therefore... Figure 2 Compared to the EV0 target in the decision-making module, Figure 3 The EV0 target obtained by the middle decision module based on the brightness of the main target is lower than Figure 2 The EV0 target in the image can reduce the exposure value and image brightness in black scenes, thereby reducing overexposure problems in black scenes and improving image quality.

[0103] In a white scene, after obtaining 3A statistical data, the AI ​​processing module can output the brightness of the main target. Since the brightness of the main target is greater than the preset brightness, it is consistent with... Figure 2 Compared to the EV0 target in the decision-making module, Figure 3 The EV0 target obtained by the middle decision module based on the brightness of the main target is higher than Figure 2 The EV0 target in the image can increase the exposure value and image brightness in white scenes, thereby reducing underexposure issues in white scenes and improving image quality.

[0104] In this embodiment, the decision module, AE algorithm module, caching module, and post-processing algorithm module can be referred to... Figure 2The relevant descriptions in the previous embodiments are not repeated here.

[0105] In the embodiments of this application, Figure 3 The process of previewing and capturing images can be found in the following references. Figure 2 The relevant descriptions in the previous sections will not be elaborated upon here, including the process of displaying images in the preview stream and saving images in the capture stream. For example, in Figure 3 In the preview process, the electronic device can process the first original image 1 to the sixth original image 6 based on the first ISP module and the second ISP module to obtain and display images 1 to 6. Figure 3 In the shooting process, before acquiring the fourth original image 4, the electronic device receives a photo-taking operation. It can cache the fourth original image 4 to the sixth original image 6, and perform post-processing on the original image with the best exposure value convergence to obtain the target image. The target image is then saved to the gallery application. For example, if the target image is image 5, image 5 is saved to the gallery application.

[0106] The following is about Figure 3 The process of adjusting the exposure value will be explained in detail. For example... Figure 3 As shown:

[0107] Taking a preview scene as an example, when the electronic device receives a trigger operation to launch the camera application, the camera obtains the original image 31 and then transmits the original image 31 to the first ISP module. The first ISP module transmits the original image 32 to the second ISP module. The second ISP module processes the original image 32 and obtains and displays an RGB image or a YUV image 33. The original image 31 and the original image 32 can be the same image or different images.

[0108] Meanwhile, the camera system 300 can adjust the exposure value of the next frame's original image based on relevant data from the current frame's original image. It's understandable that over a period of time, the backgrounds and main objects in multiple frames of original images captured by the camera are highly similar, and consecutive frames in the preview stream are correlated. Therefore, the exposure value of the next frame can be predicted and adjusted based on the information from the current frame's original image.

[0109] Specifically, for any frame of the original image in the preview process, the processing of the original image 31 of the current frame may include:

[0110] S301, The camera transmits the raw image 31 to the first module of the ISP.

[0111] The original image 31 can be a raw image.

[0112] S302, the first ISP module obtains the 3A statistical data in the original image 31.

[0113] 3A statistics may include AE ​​statistics, AWB statistics, and AF information. In this embodiment, the electronic device can adjust exposure parameters, and 3A statistics can be understood as AE statistics related to exposure. 3A statistics may include, for example, LV, block luma, AWB gain, grayscale histogram, lux index, CCT, and ISO.

[0114] It should be noted that, in this embodiment, the 3A statistical data obtained by the first ISP module can also be obtained from the image processed by the second ISP module. For example, the first ISP module obtains some 3A statistical data before the preview stream enters the second ISP module; the first ISP module also obtains some 3A statistical data after the preview stream enters the second ISP module. For example, the second ISP module includes a color correction module and a color space conversion module. After the second ISP module performs color correction and color space conversion on the original image 31, the first ISP module can also obtain some 3A statistical data. This embodiment will not elaborate further on this.

[0115] S303, ISP first module transmits 3A statistical data to the judgment module.

[0116] S304. The judgment module determines whether the current scene is the target scene based on 3A statistical data.

[0117] The judgment module can be used to determine the current scene; for example, scenes include: normal scene, white scene, black scene, and face scene, etc. Among them, the target scene can be a white scene or a black scene.

[0118] The judgment module can be one or more pre-set neural network models. The neural network model can take 3A statistical data as input, process it through the pre-trained parameters, and output a label indicating that the current scene is a scene and the corresponding confidence level.

[0119] In this context, the label for a normal scene can be the first value, the label for a black scene can be the second value, the label for a face scene can be the third value, and the label for a white scene can be the fourth value. The first to fourth values ​​can be four different values ​​in any form, such as numbers or letters. For example, the label for a normal scene is 0, the label for a black scene is 1, the label for a face scene is 2, and the label for a white scene is 3.

[0120] In this embodiment of the application, the judgment module can determine the current scene based on 3A statistical data, and in case of an abnormal scene, instruct the AI ​​processing module to perform subsequent processing; or in case of a normal scene, instruct the decision module to perform subsequent processing.

[0121] The embodiments of this application may be combined with the following. Figures 4-6 The embodiment shown provides a detailed explanation of the judgment process of the judgment module, which will not be elaborated here.

[0122] S305. When the current scenario is the target scenario, the judgment module inputs the 3A statistical data into the AI ​​processing module.

[0123] S306 The AI ​​processing module outputs the predicted brightness of the main target to the decision module.

[0124] The AI ​​processing module can be a pre-set AI AE model. The AI ​​AE model can take 3A statistical data as input, process the pre-trained parameters, and output the predicted brightness of the subject target. The subject target brightness can be the target brightness value of the subject object. The subject object can be an area with a large proportion of a certain brightness range in the shooting scene. For example, in a black scene, the black area has the largest proportion, and the black area is the subject object; in a white scene, the white area has the largest proportion, and the white area is the subject object. This application embodiment does not impose such limitations. The subject target brightness can be a grayscale value, for example, 0-255, where the subject target brightness is positively correlated with the grayscale value.

[0125] In this embodiment, the 3A statistical data may include multiple statistical parameters with different structures. For example, block luma may be the brightness data of a×b×c (e.g., 64×48×3) multivariate image blocks; or univariate data such as LV and grayscale histogram. In this embodiment, a convolutional neural network (CNN) model can be used to predict the brightness of the main target. This is because the block luma data has a similar structure to RGB format images, for example, both are three-channel data, and CNN mode is suitable for processing image-type data (e.g., RGB format images). Therefore, in this embodiment, the AI ​​processing module may include a CNN model.

[0126] It should be noted that, in this embodiment, the target brightness of the subject can be predicted, allowing for subsequent adjustments to exposure parameters based on this target brightness, thereby reducing the impact of a black or white background on the subject's brightness. For example, in a possible implementation, in a black scene, the electronic device might increase the brightness of the black background, causing the subject to be overexposed; in a white scene, the electronic device might decrease the brightness of the white background, causing the subject to be underexposed. However, in this embodiment, during exposure adjustment, the target brightness of the subject can be referenced, ensuring that after adjusting the exposure parameters, the subject's brightness is appropriate, avoiding overexposure or underexposure.

[0127] S307. The decision module adjusts the EV0 target based on the brightness of the main target; whereby the brightness of the main target is positively correlated with the EV0 target.

[0128] EV0 target can be considered as the target brightness, which can be understood as a standard value that the average brightness of pixels in an image can reach after exposure. In this embodiment, the AI ​​processing module can obtain the subject target brightness, and the decision module can adjust the EV0 target based on the subject target brightness. For example, the subject target brightness and EV0 target are positively correlated; the greater the subject target brightness output by the AI ​​processing module, the greater the EV0 target value; the smaller the subject target brightness output by the AI ​​processing module, the smaller the EV0 target value.

[0129] It is understood that the EV0 target can be adjusted by controlling the brightness of the main subject in this embodiment. For example, in a black scene, the AI ​​processing module can output a lower brightness for the main subject, thereby reducing the EV0 target. Subsequently, the electronic device can adjust the overall average brightness of the image frame by frame to converge it to near the EV0 target. In this embodiment, the average brightness of the image is low, thus avoiding the problem of overexposure of the main subject in a black scene. Similarly, in a white scene, the AI ​​processing module can output a higher brightness for the main subject, thereby increasing the EV0 target. Subsequently, the electronic device can adjust the overall average brightness of the image frame by frame to converge it to near the EV0 target. In this embodiment, the average brightness of the image is high, thus avoiding the problem of underexposure of the main subject in a white scene.

[0130] The S308 and AE algorithm modules adjust the exposure parameters based on the brightness of the EV0 target and the original image 31.

[0131] The AE algorithm module can obtain the brightness and EV0 target of the original image 31.

[0132] If the absolute value of the difference between the brightness of the original image 31 and the EV0 target is less than or equal to the threshold, it can be considered that the brightness of the original image 31 is roughly equal to the target brightness, and an appropriately exposed original image can be obtained based on the current exposure parameters. The imaging system 300 can terminate the execution flow of the current frame without adjusting the exposure parameters. The camera can use the exposure parameters of the current frame to continue acquiring the next frame of the original image (e.g., original image 37). Figure 3 (The original image 37 is not shown in the image), and the relevant processes of steps S301-S309 are executed again.

[0133] If the absolute value of the difference between the brightness of the original image 31 and the EV0 target is greater than the threshold, it can be considered that the brightness of the original image 31 differs significantly from the target brightness, and an properly exposed original image cannot be obtained based on the current exposure parameters. The AE algorithm can adjust the exposure parameters according to the AE table.

[0134] For example, when the brightness of the original image 31 is less than the EV0 target, the AE algorithm can adjust the exposure parameters to increase the exposure. For example, increasing the exposure can be achieved by extending the shutter time, increasing the sensor gain, or increasing the ISP gain. When the brightness of the original image 31 is greater than the EV0 target, the AE algorithm can adjust the exposure parameters to decrease the exposure. For example, decreasing the exposure can be achieved by shortening the exposure time, decreasing the sensor gain, or decreasing the ISP gain.

[0135] The S309 and AE algorithm modules transmit exposure parameters to the camera.

[0136] The AE algorithm module can instruct adjustments to exposure parameters. These adjusted parameters are then applied to the camera, which uses them to expose the image and obtain the next frame of the original image (e.g., original image 37). Figure 3 (Not shown in the image).

[0137] Understandably, the electronic device can cycle through steps S301-S308 once for each original image it acquires. If, during the execution of the process related to the original image 37, the absolute value of the difference between the brightness of the original image 37 and the EV0 target is still less than or equal to a threshold, the AE algorithm module can correspondingly increase or decrease the exposure; thereby converging the brightness of the original image frame by frame to near the EV0 target. In the preview scene, the electronic device can obtain an original image with a brightness value close to the EV0 target based on the adjusted exposure parameters, and the original image will not suffer from overexposure or underexposure.

[0138] Optionally, after step S304, the following may also be included:

[0139] S310. If the current scene is not the target scene, the judgment module will input the 3A statistical data into the decision module.

[0140] S311, The decision module passes the EV0 target to the AE algorithm module.

[0141] The S312 AE algorithm module adjusts the exposure parameters based on the brightness of the EV0 target and the original image 31.

[0142] It should be noted that when the current scene is neither a black and white scene nor a white and black scene, the electronic device can use the exposure process for a normal scene to process the image exposure. In this process, the electronic device does not use the AI ​​processing module to predict the brightness of the subject.

[0143] Taking a photography scenario as an example, when the electronic device receives a trigger operation on the shutter button (e.g., the shutter button and / or volume buttons in the camera application interface), it stores the original image 34 in the cache module; the cache module then transmits the original image 34 to the first ISP module; the first ISP module transmits the original image 35 to the second ISP module and the post-processing algorithm module. After the second ISP module and the post-processing algorithm module process the original image 35, they save the RGB image or YUV image 36. The original image 34 and the original image 35 can be the same image or different images.

[0144] Understandably, for any given frame, the RGB or YUV image 33 in the preview stream is less sharp than the RGB or YUV image 36 in the shooting stream.

[0145] It should be noted that when the current scene is neither a black and white scene nor a white and black scene, the electronic device can use the exposure process for a normal scene to process the image exposure. In this process, the electronic device does not use the AI ​​processing module to predict the brightness of the subject.

[0146] The above embodiments illustrate the process of adjusting the exposure of any frame of image in this application. The following describes the process of determining the target scene as shown in step S304.

[0147] Taking the detection of black scenes as an example, in the first possible implementation, the judgment module can filter out non-black scenes based on multiple preset conditions, thereby determining the black scene.

[0148] In the second possible implementation, an AI model can be added to the first possible implementation. The AI ​​model is used to test the results of the first possible implementation.

[0149] In the third possible implementation, the judgment module can be equipped with a classification model, which has the ability to output scene identifiers based on 3A statistical data; when the classification model outputs a label value to identify a black scene, it can be determined that the current scene is black.

[0150] Let's combine the following... Figure 4 The decision-making process for the first possible implementation is explained. For example... Figure 4 As shown:

[0151] The preset conditions may include: preset condition 1, preset condition 2, preset condition 3, preset condition 4, and preset condition 5. Among them, preset condition 1 is used to filter out scenes with faces, preset condition 2 is used to filter out distant scenes, preset condition 3 can be used to filter out non-uniform scenes, preset condition 4 can be used to filter out scenes with low brightness of the main object, and preset condition 5 can be used to filter out solid color scenes with color.

[0152] Specifically, after the judgment module obtains the 3A statistical data, the flow of the 3A statistical data in the judgment module can be shown in the following steps S401-S405:

[0153] S401. Determine whether the current scene is a face scene.

[0154] Preset condition 1 allows for the failure to recognize a face. The electronic device is equipped with a face recognition model. If a face is present in the current scene, the judgment module terminates the judgment process and instructs the user to adjust the exposure parameters using the normal scene exposure procedure. If preset condition 1 is met and no face is recognized, the judgment module can continue to execute step S402.

[0155] Understandably, electronic devices can be configured with exposure schemes for face scenes. When a face scene is detected, the electronic device can use the face scene exposure process to adjust the exposure of the face scene.

[0156] It should be noted that the embodiments of this application can use the exposure process for face scenes to process pixels related to the face in the original image; and the embodiments of this application can use image processing methods to process pixels other than the face in the original image. For example, in a black scene, the face area can be exposed using the exposure process for face scenes, while the background, clothing, and other areas can be exposed using the image processing methods provided in the embodiments of this application, so that the overall image brightness value is not overexposed.

[0157] S402. If the current scene is not a face scene, determine whether the current scene is a distant scene.

[0158] Distant scenes can be those where the distance between the subject and the electronic device exceeds a distance threshold. Understandably, overexposure in black scenes and underexposure in white scenes often occur in close-up shots. When the subject is far from the camera, the probability of underexposure or overexposure using the normal scene exposure procedure is lower. Therefore, in distant scenes, the judgment module can terminate the judgment process and instruct the use of the normal scene exposure procedure to adjust the exposure parameters.

[0159] 3A statistical data may include ambient light value (aeLV) and distance value from the image sensor. The distance value can be laser distance or object distance parameter. In some embodiments, the electronic device may be equipped with a time-of-flight (TOF) camera, and the judgment module can measure the laser distance between the subject and the electronic device based on the TOF camera. In other embodiments, the electronic device may use monocular, binocular, or other cameras to collect object distance parameters and obtain the distance between the subject and the camera.

[0160] Taking the distance value as laser distance as an example, the preset condition 2 can be that the aeLV value is greater than the aeLV threshold and the laser distance is greater than the distance threshold; the aeLV threshold can be, for example, 60, and the distance threshold can be, for example, 400mm. When the 3A statistical data of the current scene meets the preset condition 2, the judgment module can execute step S403; when the 3A statistical data of the current scene meets the preset condition 2, the judgment module can terminate the judgment process.

[0161] S403. If the current scene is not a distant scene, determine whether the current scene is a non-uniform scene.

[0162] A uniform scene is one where the brightness difference between pixel regions in an image is within a brightness difference threshold range. A pixel region can be a single pixel or a block of pixels. A non-uniform scene is one where the brightness difference between pixel regions in an image exceeds the brightness difference threshold range. For example, a uniform scene can be a uniform night scene or a uniform indoor scene.

[0163] It is understood that in this embodiment, a black scene can be defined as a scene where the proportion of black areas in the current shooting scene is greater than a first threshold, and a white scene can be defined as a scene where the proportion of white areas in the current shooting scene is greater than a second threshold. The first threshold can be the same as or different from the second threshold; for example, the first threshold and the second threshold can both be 80%. It can be seen that the brightness of pixels in black and white scenes is relatively uniform. Therefore, in step S403, non-uniform scenes that are not black or white scenes can be filtered out using preset condition 3, further improving the accuracy of scene recognition.

[0164] Specifically, 3A statistical data may also include ambient luminance values ​​(alsLV) from multispectral sensors. Electronic devices can use the luminance difference obtained from image sensors and multispectral sensors to determine whether a scene is non-uniform. The positions of the light received by the image sensor and the multispectral sensor are different. For example, the image sensor and the multispectral sensor may partially or completely overlap, and the area of ​​the image sensor may be smaller than that of the multispectral sensor. In some embodiments, the luminance value received by the image sensor and the luminance value received by the multispectral sensor in the non-overlapping area can be calculated to determine whether a scene is non-uniform. The smaller the luminance difference, the higher the probability that the current scene is a black scene; the larger the luminance difference, the lower the probability that the current scene is a black scene.

[0165] In some embodiments, the determination module can determine non-uniform scenes using a dual-threshold approach. For example, the brightness difference threshold may include a first brightness difference threshold and a second brightness difference threshold; the first brightness difference threshold is less than the second brightness difference threshold. If the brightness difference between the alsLV and aeLV values ​​is less than the first brightness difference threshold, the determination module proceeds to the next scene determination process, and the determination module may continue to execute step S404; if the brightness difference between the alsLV and aeLV values ​​is greater than the second brightness difference threshold, the determination module exits the next scene determination process, and the determination module may not execute step S404.

[0166] The dual-threshold determination process can be understood as follows: when the previous frame's original image has not entered the determination process for the next scene, the preset condition 3 for the current frame's original image can be that the brightness difference between the alsLV and aeLV values ​​is less than the first brightness difference threshold; when the previous frame's original image enters the determination process for the next scene, the preset condition 3 for the current frame's original image can be that the brightness difference between the alsLV and aeLV values ​​is less than or equal to the second brightness difference threshold. When preset condition 3 is met, the current scene can be considered a uniform scene; when preset condition 3 is not met, the current scene can be considered a non-uniform scene.

[0167] For example, the first brightness difference threshold is 5, and the second brightness difference threshold is 10. The following uses the first original image 1 to the fourth original image 4 as examples to explain the preset condition 3.

[0168] 1) For original image 1: the brightness difference between the alsLV and aeLV values ​​is greater than or equal to 5, for example, the brightness difference is 6, the judgment module does not execute step S404; it determines that original image 1 has not entered the judgment process of the next scene. At this time, the preset condition 3 for the next frame of original image to enter step S404 is that the brightness difference between the alsLV and aeLV values ​​is less than 5.

[0169] 2) For the second original image 2: the brightness difference between the alsLV value and the aeLV value is less than 5, for example, the brightness difference is 4, which satisfies the preset condition 3 in step 1). The judgment module can execute step S404 to determine whether the second original image 2 can enter the next scene. At this time, the preset condition 3 for the next frame original image to enter step S404 changes to the brightness difference between the alsLV value and the aeLV value being less than 10.

[0170] 3) For the third original image 3: the brightness difference between the alsLV and aeLV values ​​is less than or equal to 10, for example, the brightness difference is any value between 0 and 10, satisfying the preset condition 3 in step 2). The judgment module can execute step S404 to determine whether the original image 3 can enter the next scene. At this time, the preset condition 3 for the next frame original image to enter step S404 is still that the brightness difference between the alsLV and aeLV values ​​is less than or equal to 10.

[0171] 4) For the fourth original image 4: the brightness difference between the alsLV and aeLV values ​​is greater than 10, for example, the brightness difference is 11, which does not meet the preset condition 3 in step 3). The judgment module may not execute step S404 and determine that the fourth original image 4 has not entered the judgment process of the next scene. At this time, the preset condition 3 for the next frame original image to enter step S404 is changed to the brightness difference between the alsLV and aeLV values ​​being less than 5.

[0172] And so on.

[0173] It should be noted that the above embodiments are merely illustrative of the determination method of dual thresholds. The brightness difference between the first original image 1 and the fourth original image 4 can also be other values, and this application embodiment does not limit this.

[0174] It is understood that in the embodiments of this application, the AI ​​processing module involves an AI AE model. If the brightness value changes abruptly during the model's frame-by-frame convergence, it may affect the stability of the AI ​​AE model's output results. Therefore, in order to improve the stability of the brightness value, the embodiments of this application use a dual threshold method to judge non-uniform scenes.

[0175] S404. If the current scene is not a non-uniform scene, determine whether the current scene is a scene where the main object is in low brightness.

[0176] A scene where the subject is in low brightness can be one where the brightness of the subject is below a brightness threshold. It is understood that when the brightness of the subject is below the brightness threshold, the shooting scene may be a night scene. Electronic devices can be configured with exposure schemes for night scenes. In night scenes, the electronic device can use the exposure procedures for normal scenes to adjust the exposure of the subject. Therefore, in scenes where the subject is in low brightness, the image processing method provided in this application's embodiments may not be necessary.

[0177] In this embodiment, if preset condition 4 is met, the judgment module can execute step S405; if preset condition 4 is not met, the judgment module can terminate the judgment process and instruct the normal scene exposure process to adjust the exposure parameters. Preset condition 4 can be that the image block brightness is greater than an image block brightness threshold; for example, the image block brightness threshold can be 20. When the block luma is greater than 20, the current scene is not a scene where the main object is low in brightness; when the block luma is less than 20, the current scene is a scene where the main object is low in brightness.

[0178] S405. If the current scene is not a scene where the main object is in low brightness, determine whether the current scene is a solid color scene with color.

[0179] Understandably, after scene filtering in steps S401-S404, the electronic device may obtain solid color scenes, such as red, green, black, and white scenes. Solid color scenes with color can be other scenes that are neither black nor white within the solid color scene category. For solid color scenes with color, the electronic device can output appropriate exposure parameters, avoiding overexposure issues similar to those encountered in black scenes. Therefore, it is necessary to filter solid color scenes with color here.

[0180] In this embodiment, if preset condition 5 is met, the judgment module can execute step S406; if preset condition 5 is not met, the judgment module can terminate the judgment process and instruct the normal scene exposure process to adjust the exposure parameters. The 3A statistical data may include RGB values, and preset condition 5 can be that the ratio of the largest RGB gain coefficient to the smallest RGB gain coefficient is less than a ratio threshold. Specifically, the ratio threshold can be, for example, 2, and preset condition 5 can be max(Rgain,Ggain,Bgain) / mix(Rgain,Ggain,Bgain)<2. Where Rgain=R / G; Ggain=G / G; Bgain=B / G.

[0181] It should be noted that this application embodiment describes the judgment process using steps S401-S405 in sequence, but this application embodiment does not limit the execution order of the steps. For example, the judgment module can first determine the 3A statistical data based on any one of the preset conditions 1-5. If the determination result is negative, it continues to determine other preset conditions 1-5 until the determination result is positive, or the judgment process involves all preset conditions. This application embodiment does not limit the order in which the preset conditions are used.

[0182] S406. If the current scene is not a solid color scene with color, the judgment module will pass the 3A statistical data to the AI ​​processing module, and the AI ​​processing module will output the brightness of the main target based on the 3A statistical data.

[0183] After steps S401-S405, the judgment module filters out non-black scenes, thus determining the current scene to be a black scene. In a black scene, the electronic device can use the AI ​​processing module to predict the brightness of the main target, thereby reducing overexposure issues in black scenes.

[0184] Figure 4 The illustrated embodiment describes a first possible implementation. The second possible implementation provided by the embodiments of this application is described below.

[0185] The second possible implementation is used to verify the accuracy of the judgment result in the first possible implementation. This process can be referred to... Figure 4 The embodiments in this application will not be described in detail.

[0186] For example, the judgment module may also include an auxiliary verification model, which can be used to assist scene detection and verify the judgment result.

[0187] S501. Determine whether it is a face scene based on preset condition 1.

[0188] Step S501 can be referred to the relevant description in step S401, and will not be repeated here.

[0189] S502. When the determination result is a face scene, input the 3A statistical data into the auxiliary verification model to obtain result A; result A may include the identifier of the current scene and the confidence level corresponding to the current scene.

[0190] The auxiliary verification model can be used to verify face scenes. It outputs a label (label=2) for the face scene and a confidence score. If the confidence score is greater than a threshold, the judgment result based on preset condition 1 is considered accurate, and the current scene is a face scene. Thus, the auxiliary verification model can be used to verify the judgment result based on preset conditions, improving the accuracy of target scene detection.

[0191] Alternatively, S503, if the current scene is not a face scene, determine whether it is a distant scene based on preset condition 2.

[0192] Step S503 can be referred to the relevant description in step S402, and will not be repeated here.

[0193] S504. When the determination result is a distant scene, input the 3A statistical data into the auxiliary verification model to obtain result B; result B may include the identifier of the current scene and the confidence level corresponding to the current scene.

[0194] The auxiliary verification model can be used to verify distant scenes. Ordinary scenes can include distant scenes. The auxiliary verification model can output the label of ordinary scenes (label=0) and the confidence level of ordinary scenes. If the confidence level is greater than the confidence level threshold, then the judgment result obtained based on preset condition 2 is determined to be accurate, and the current scene is a distant scene.

[0195] Alternatively, S505, if the current scene is not a distant scene, determine whether it is a non-uniform scene based on preset condition 3.

[0196] Step S505 can be referred to the relevant description in step S403, and will not be repeated here.

[0197] S506. When the judgment result is a non-uniform scene, input the 3A statistical data into the auxiliary verification model to obtain result C; result C may include the identifier of the current scene and the confidence level corresponding to the current scene.

[0198] The auxiliary verification model can be used to verify non-uniform scenes. Normal scenes can include non-uniform scenes. The auxiliary verification model can output a label of 0 for normal scenes and a confidence level for the normal scene. If the confidence level is greater than the confidence level threshold, then the judgment result obtained based on preset condition 3 is determined to be accurate, and the current scene is a non-uniform scene.

[0199] Alternatively, S507, if the current scene is not a non-uniform scene, determine whether it is a scene with low brightness of the main object based on preset condition 4.

[0200] Step S507 can be referred to the relevant description in step S404, and will not be repeated here.

[0201] S508. When the scene is determined to be a scene with low brightness of the main object, the 3A statistical data is input into the auxiliary verification model to obtain the result D. The result D may include the identifier of the current scene and the confidence level corresponding to the current scene.

[0202] The auxiliary verification model can be used to verify scenes where the main object is in low brightness. Normal scenes can include scenes where the main object is in low brightness. The auxiliary verification model can output a label of 0 for normal scenes, as well as the confidence level of the normal scene. If the confidence level is greater than the confidence threshold, then the judgment result obtained based on preset condition 4 is considered accurate, and the current scene is a scene where the main object is in low brightness.

[0203] Alternatively, S509, if the current scene is not a scene where the main object is in low brightness, determine whether it is a solid color scene with color based on preset condition 5.

[0204] Step S509 can be referred to the relevant description in step S405, and will not be repeated here.

[0205] S510. When the judgment result is a solid color scene with color, input the 3A statistical data into the auxiliary verification model to obtain the result E; the result E may include the identifier of the current scene and the confidence level corresponding to the current scene.

[0206] The auxiliary verification model can be used to verify solid-color scenes with color. Ordinary scenes can include solid-color scenes with color. The auxiliary verification model can output a label of 0 for ordinary scenes, as well as the confidence level of the ordinary scene. If the confidence level is greater than the confidence threshold, then the judgment result obtained based on preset condition 5 is determined to be accurate, and the current scene is a solid-color scene with color.

[0207] In this embodiment, the auxiliary verification model can be a single model or multiple sub-models trained jointly or not jointly; this embodiment does not impose any restrictions on this.

[0208] In this way, the output results of the auxiliary verification model can be used to reduce false detections under preset conditions and improve the accuracy of the target scene.

[0209] In the third possible implementation, the judgment module may include a classification model. This model can be trained using 3A statistical data from multiple scenarios and has the ability to output multiple scenario labels and confidence levels. (The third possible implementation can be found in [reference needed]). Figure 5 The process shown is as follows:

[0210] S601. The judgment module inputs the 3A statistical data into the classification model to obtain the scene identifier and confidence level.

[0211] The classification model can output labels for multiple scenarios based on 3A statistical data. The scenario labels are not limited to label = 0-3 as shown in the embodiments of this application. The classification scenario can have the ability to identify more scenarios. For example, the scenario labels can include label = 0-N, where N is a positive integer.

[0212] S602. When the scene is marked as a black scene, the judgment module transmits the 3A statistical data to the AI ​​processing module.

[0213] For example, the classification model can output label=1 and the confidence level that the current scene is a black scene; if the confidence level is greater than or equal to the confidence level threshold, then step S603 is executed; if the confidence level is less than the confidence level threshold, then the judgment process ends.

[0214] The S603 AI processing module outputs the brightness of the main target.

[0215] This step can be referred to in the relevant description in step S306, and will not be repeated here.

[0216] Alternatively, after step S601, the method may further include:

[0217] S604. If the scene identifier is not a black scene, the judgment module will transmit the 3A statistical data to the decision module.

[0218] When the output of the classification model is not label=1, the current scene is not a black scene, and the judgment module can instruct the normal scene exposure process to be used for exposure processing.

[0219] It should be noted that the above embodiments use a black scene as an example to introduce three ways to determine the target scene. This application embodiment can use any one of the methods for scene detection alone, or it can combine the above methods to improve the accuracy of scene detection. This application embodiment does not limit this.

[0220] The above embodiments use a black scene as an example to illustrate the method of electronic device detection of a black scene. The following describes the method for determining a white scene.

[0221] It should be noted that the method for determining a white scene is similar to that for determining a black scene. This application embodiment only describes the determination process where there are differences between white and black scenes; other parts can be referred to. Figure 4 The embodiments shown to Figure 5 The relevant descriptions in the illustrated embodiments will not be repeated here. For example, taking the detection of a white scene as an example, in the first possible implementation, the judgment module can filter out non-white scenes based on multiple preset conditions, thereby determining the white scene.

[0222] In the second possible implementation, an AI model can be added to the first possible implementation. The AI ​​model is used to test the results of the first possible implementation.

[0223] In the third possible implementation, the judgment module can be equipped with a classification model, which has the ability to output scene identifiers based on 3A statistical data; when the classification model outputs a label value to identify a white scene, it can be determined that the current scene is white.

[0224] The first possible implementation method in the white scene will be explained below.

[0225] The preset conditions for identifying white scenes may include: preset condition 6, preset condition 7, preset condition 8, preset condition 9, and preset condition 10. Among them, preset condition 6 is used to filter out scenes with faces, preset condition 7 is used to filter out distant scenes, preset condition 8 can be used to filter out non-uniform scenes, preset condition 9 can be used to filter out scenes with low brightness of the main object, and preset condition 10 can be used to filter out solid color scenes with color.

[0226] Specifically, after the judgment module obtains the 3A statistical data, the flow of the 3A statistical data in the judgment module can be shown in the following steps S701-S705:

[0227] S701. Determine whether the current scene is a face scene.

[0228] Preset condition 6 can be that no face is recognized. If preset condition 1 is met and no face is recognized, the judgment module can continue to execute step S702.

[0229] S702. If the current scene is not a face scene, determine whether the current scene is a distant scene.

[0230] Preset condition 7 can be: the aeLV value is greater than the aeLV threshold, and / or the laser distance is greater than the distance threshold.

[0231] It is understood that the aeLV threshold in a white scene can be higher than the aeLV threshold in a black scene; the distance threshold in a white scene can be higher than the distance threshold in a black scene, and the two thresholds can be the same or different. In some embodiments, the preset condition 7 can be, for example, an aeLV value greater than the aeLV threshold and a laser distance greater than the distance threshold; in other embodiments, the preset condition 7 can be, for example, an aeLV value greater than the aeLV threshold.

[0232] In this embodiment, white scenes are frequently used, such as snow scenes and white close-up scenes. For snow scenes, the judgment module can use preset condition 7: the aELV value is greater than the aELV threshold for limitation; for white close-up scenes, the judgment module can use preset condition 7: the aELV value is greater than the aELV threshold, and the laser distance is greater than the distance threshold for limitation. This embodiment does not impose any limitations on this.

[0233] If preset condition 7 is met, the judgment module can continue to execute step S703.

[0234] S703. If the current scene is not a distant scene, determine whether the current scene is a non-uniform scene.

[0235] Preset condition 8 can be: if the brightness difference between the alsLV and aeLV values ​​is less than the third brightness difference threshold, the judgment module enters the judgment process for the next scene, and the judgment module can continue to execute step S704; if the brightness difference between the alsLV and aeLV values ​​is greater than the fourth brightness difference threshold, the judgment module exits the judgment process for the next scene, and the judgment module can skip step S704. The third brightness difference threshold is less than the fourth brightness difference threshold; wherein, the third brightness difference threshold can be the same as or different from the first brightness difference threshold; the fourth brightness difference threshold can be the same as or different from the second brightness difference threshold. This application embodiment does not impose any restrictions on this.

[0236] If preset condition 8 is met, the judgment module can continue to execute step S704.

[0237] S704. If the current scene is not a non-uniform scene, determine whether the current scene is a scene where the main object is in low brightness.

[0238] Preset condition 9 can be: image block brightness is greater than the image block brightness threshold. In this embodiment, the image block brightness threshold in a black scene is less than the image block brightness threshold in a white scene. For example, in a black scene, the image block brightness threshold can be 20; in a white scene, the image block brightness threshold can be 70. Preset condition 9 can be, for example: block brightness > 70.

[0239] If preset condition 9 is met, the judgment module can continue to execute step S705.

[0240] S705. If the current scene is not a scene where the main object is in low brightness, determine whether the current scene is a solid color scene with color.

[0241] Preset condition 10 can be: the ratio of the largest RGB gain coefficient to the smallest RGB gain coefficient is less than a ratio threshold. The ratio threshold in the white scene can be the same as or different from the ratio threshold in the black scene; this embodiment does not impose any restrictions on this.

[0242] If preset condition 10 is met, the judgment module can continue to execute step S706.

[0243] S706. If the current scene is not a solid color scene with color, the judgment module will pass the 3A statistical data to the AI ​​processing module, and the AI ​​processing module will output the brightness of the main target based on the 3A statistical data.

[0244] After steps S701-S705, the judgment module filters out non-white scenes, thus determining the current scene to be a white scene. In a white scene, the electronic device can use the AI ​​processing module to predict the brightness of the main target, thereby reducing the problem of underexposure in white scenes.

[0245] It should be noted that steps S701-S706 can be referred to Figure 4 The relevant descriptions in steps S401-S406 will not be repeated here.

[0246] For the second possible implementation in the white scene, the judgment module can also adopt a similar process to that shown in the second possible implementation in the black scene to verify the detection results of the white scene. This process can be referred to the relevant descriptions in steps S501-S510, which will not be repeated here.

[0247] For a third possible implementation in white scenes, the judgment module can include a classification model, which can also be used to identify white scenes, such as... Figure 6 As shown:

[0248] For example, in S801, the judgment module inputs the 3A statistical data into the classification model to obtain the scene identifier and confidence level.

[0249] S802. When the scene is marked as a white scene, the judgment module transmits the 3A statistical data to the AI ​​processing module.

[0250] For example, the classification model can output label=3 and the confidence level that the current scene is a white scene; if the confidence level is greater than or equal to the confidence level threshold, then step S803 is executed; if the confidence level is less than the confidence level threshold, then the judgment process ends.

[0251] The S803 AI processing module outputs the brightness of the main target.

[0252] Alternatively, after step S801, the method may further include:

[0253] S804. If the scene identifier is not a white scene, the judgment module will transmit the 3A statistical data to the decision module.

[0254] When the output of the classification model is not label=3, the current scene is not a white scene, and the judgment module can instruct the normal scene exposure process to be used for exposure processing.

[0255] The descriptions of some steps S801-S804 can be found in steps S601-S604, and will not be repeated in this embodiment.

[0256] The above embodiments illustrate the process of determining the target scene in the image processing method of this application. The following will describe this process in conjunction with... Figure 7 The training process of the AI ​​AE model in the embodiments of this application will be described.

[0257] S901. Obtain the training dataset.

[0258] The training dataset may include multiple training datasets, each of which includes the brightness of the main target in the sample and multiple sample 3A statistics.

[0259] The process of obtaining the training dataset can be described as follows:

[0260] a) Collect multiple sets of sample images of the target scene, where each set of sample images includes multiple sample images of the same target scene with different exposures.

[0261] In this embodiment, underexposed, normal, and overexposed sample images of the same scene can be obtained by adjusting the exposure time and exposure gain. The sample images of the same scene can be as follows: Figure 7 Sample images 1-10 are shown.

[0262] For example, sample images 1-10 can be sample images with different exposure levels collected in a black scene. For instance, sample images 1-2 are underexposed sample images in a black scene; sample image 3 is an appropriately exposed sample image in a black scene; and sample images 4-10 are overexposed sample images in a black scene.

[0263] And / or, sample images 1-10 can also be sample images with different exposure levels captured in a white scene. For example, sample images 1-8 are underexposed sample images in a white scene; sample image 9 is an appropriately exposed sample image in a white scene; and sample image 10 is an overexposed sample image in a white scene.

[0264] It should be noted that the embodiments of this application only illustrate sample images of the target scene, but the above sample images do not limit the embodiments of this application. For example, electronic devices can also use different sample images to train the model to output the brightness of the subject in a black scene and the brightness of the subject in a white scene, respectively. For example, sample images 1-10 of the white scene are different from sample images 1-10 of the black scene. The model to be trained can be trained using sample images with different exposure levels in the white scene to obtain an AIAE model suitable for the white scene; or, the model to be trained can be trained using sample images with different exposure levels in the black scene to obtain an AIAE model suitable for the black scene.

[0265] b) For any sample image in any set of sample images, parse the exchangeable image file format (Exif) information of the sample image; the Exif information includes the sample 3A statistics of the sample image.

[0266] For example, parsing the Exif information of images 1-10 yields Exif information 1-Exif information 10. 3A statistical data may include: LV, block luma, AWB gain, grayscale histogram, lux index, color temperature, ISO, etc.

[0267] c) For any set of sample images, label the target sample image and obtain the Exif information of the target sample image. The Exif information of the target sample image includes the brightness of the main target.

[0268] The target sample image can be a sample image with appropriate exposure in the current scene.

[0269] The rules for selecting target sample images are as follows: the target sample image does not have overexposure or underexposure; the target sample image is not overexposed and the details in the dark areas are well preserved; the brightness values ​​of the grayscale histogram are piled up in the middle and there is no problem of being cut off or overflowing on the left and right sides.

[0270] Each training dataset includes the brightness of the main subject in the sample image and 3A statistical data from multiple samples. For example, for training a black scene, the training data includes the brightness of the main subject in sample image 3 and 3A statistical data from Exif information 1 to Exif information 10. For training a white scene, the training data includes the brightness of the main subject in sample image 9 and 3A statistical data from Exif information 1 to Exif information 10.

[0271] S902. Train the model to be trained based on the training dataset to obtain the AI ​​AE model.

[0272] During training, the training dataset can be input into the model to be trained. The brightness of the main target output by the model to be trained is compared with the brightness of the main target labeled in the sample data to calculate the loss. Then, the parameters of the model to be trained are updated based on the result of the loss calculation. This process is repeated until the predetermined maximum number of iterations is reached, or the loss converges, or the loss value is less than a certain value. At this point, the model training can be considered to be completed, and the AI ​​AE model is obtained.

[0273] For example, the model to be trained can learn the mapping relationship between 3A statistical data and the brightness of the subject target, enabling the trained AI AE model to adaptively output the brightness of the subject target based on the 3A statistical data. Specifically, the mapping relationship between 3A statistical data and the brightness of the subject target can be, for example: LV value is positively correlated with the brightness of the subject target; block luma value is positively correlated with the brightness of the subject target; AWB gain is positively correlated with the brightness of the subject target; Y value of grayscale histogram is positively correlated with the brightness of the subject target; lux index is positively correlated with the brightness of the subject target; color temperature is positively correlated with the brightness of the subject target; and ISO is positively correlated with the brightness of the subject target.

[0274] In this way, in a black scene, the AI ​​AE model can adaptively output the brightness of the subject based on 3A statistical data, thereby reducing the number of scenes where the brightness value of the subject is too high in a black scene; and in a white scene, the AI ​​AE model can adaptively output the brightness of the subject based on 3A statistical data, thereby reducing the number of scenes where the brightness value of the subject is too low in a white scene.

[0275] It should be noted that in the embodiments of this application, the electronic device may be set with preset conditions 1-5, an auxiliary detection model and a classification model to identify black scenes. The AI ​​AE model can be a model trained based on data from black abnormal scenes. The AI ​​AE model can output the brightness of the main target in a black scene relatively accurately.

[0276] Alternatively, electronic devices can be set with preset conditions 6-10, auxiliary detection models and classification models to identify white scenes. The AI ​​AE model can be a model trained based on data from white abnormal scenes. The AI ​​AE model can output the brightness of the main target in a white scene more accurately.

[0277] Alternatively, the electronic device may also be configured with detection rules for both black and white scenes, as well as AI AE models trained on black abnormal scenes and AI AE models trained on white abnormal scenes. When the electronic device detects a black scene, it inputs 3A statistical data into the AI ​​AE model corresponding to the black scene to predict the brightness of the main target in the black scene; when the electronic device detects a white scene, it inputs 3A statistical data into the AI ​​AE model corresponding to the white scene to predict the brightness of the main target in the white scene. This application does not limit the combination of the above embodiments in its embodiments.

[0278] The above embodiments illustrate the image processing method provided in the embodiments of this application. The following will describe the method in conjunction with... Figure 8 and Figure 9 The software structure and hardware architecture of the electronic devices in the embodiments of this application are described respectively.

[0279] Figure 8 This is a schematic diagram of the architecture (including software system and some hardware) used in the embodiments of this application. Figure 8 As shown, the application architecture is divided into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the application architecture can be divided into five layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer (HAL), the kernel layer (also called the driver layer), and the hardware layer.

[0280] like Figure 8 As shown, the application layer includes the camera and the gallery.

[0281] Understandable. Figure 8 The examples shown are only a portion of the applications; in fact, the application layer can include other applications as well, and this application does not limit this. For example, the application layer may also include applications such as messaging, alarm clock, weather, stopwatch, compass, timer, flashlight, calendar, and Alipay.

[0282] like Figure 8 As shown, the application framework layer includes a camera access interface. The camera access interface includes camera management and camera devices. The hardware abstraction layer includes a camera hardware abstraction layer and a camera algorithm library. The camera hardware abstraction layer includes multiple camera devices. The camera algorithm library includes modules related to the exposure adjustment process, such as an AI processing module, a judgment module, a decision-making module, and an AE algorithm module.

[0283] It should be understood that the decision-making module can also be placed in other layers. As one possible implementation, the decision-making module can be placed in the application layer or the application framework layer.

[0284] The kernel layer is used to drive hardware resources. The kernel layer can include multiple driver modules. For example... Figure 8As shown, the kernel layer includes the camera device driver.

[0285] The hardware layer includes sensors and an ISP. The sensors include image sensors, TOF cameras, and multispectral sensors. The ISP includes ISP module one and ISP module two.

[0286] For example, a user can tap the camera application. When the user taps the camera to take a picture, the shooting command is sent to the camera hardware abstraction layer (HAL) via the camera access interface. The HAL calls the camera device driver, which in turn drives the image sensor to receive light information, obtaining the raw image and AELV value; the camera device driver can also drive the TOF camera to measure the distance between the subject and the lens; and the camera device driver can drive the multispectral sensor to acquire AELV values. The image sensor then transmits the raw image to the ISP. The first ISP module, after obtaining the raw image, extracts the 3A statistical information from it; the first ISP module then transmits the raw image to the second ISP module, which processes the raw image into an RGB or YUV image. The ISP uploads the RGB or YUV image to the camera hardware abstraction layer via the camera device driver; the camera access interface then reports it to the camera application, which displays the image.

[0287] In addition, the ISP can report 3A statistical information to the camera algorithm library via the camera device driver and camera hardware abstraction layer. The judgment module determines whether the current scene is the target scene based on the 3A statistical information. When the current scene is the target scene, the judgment module inputs the 3A statistical data into the AI ​​processing module to obtain the predicted subject brightness; after obtaining the subject brightness, the decision module outputs the EV0 target based on the subject brightness; the AE algorithm adjusts the exposure parameters of the next frame based on the EV0 target. Alternatively, when the current scene is a normal scene, the judgment module inputs the 3A statistical data into the decision module; the decision module sets a preset EV0 target; the AE algorithm adjusts the exposure parameters of the next frame based on the EV0 target.

[0288] The software system used in the embodiments of this application has been described in detail above. The following section, in conjunction with... Figure 9 Describe the hardware system of electronic device 100.

[0289] Figure 9A schematic diagram of the terminal device 100 is shown. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, a subscriber identification module (SIM) card interface 195, and an embedded secure element (eSE) chip 196, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0290] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0291] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0292] Electronic device 100 implements display functions through a GPU, display screen 194, and application processor. Display screen 194 is used to display images, videos, etc.

[0293] In this embodiment, the electronic device 100 displays an RGB image or a YUV image with adjusted exposure on the display screen 194.

[0294] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0295] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0296] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0297] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0298] In the image processing method provided in this application embodiment, the electronic device 100 can collect 3A statistical data based on the ISP and process raw images; the electronic device 100 can collect raw images based on the camera 193, and the electronic device can also implement neural network algorithms such as face recognition, subject brightness prediction, and scene recognition through the computing processing capabilities provided by the NPU.

[0299] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0300] In this embodiment, the code implementing the shooting method described in this embodiment can be stored in non-volatile memory. When running a camera application, the electronic device 100 can load the executable code stored in the non-volatile memory into random access memory. For example, in the shooting path, the electronic device 100 can cache the original image through internal memory 121.

[0301] A distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, during scene capture, the electronic device 100 can utilize the distance sensor 180F for distance measurement to achieve fast focusing. In this embodiment, the distance sensor 180F can be used to detect whether the current scene is a distant scene; for example, the distance sensor 180F can be a TOF camera.

[0302] The image processing method of the present application embodiments has been described above. The apparatus for performing the above image processing method provided in the present application embodiments will now be described. Those skilled in the art will understand that the methods and apparatus can be combined with and referenced in each other, and the related apparatus provided in the present application embodiments can perform the steps in the above image processing method.

[0303] like Figure 10 As shown, the image processing apparatus 1000 can be used in communication equipment, circuits, hardware components, or chips. The image processing apparatus includes a display unit 1001 and a processing unit 1002. The display unit 1001 supports the display steps performed by the image processing apparatus 1000; the processing unit 1002 supports the information processing steps performed by the image processing apparatus 1000.

[0304] In a possible implementation, the image processing device 1000 may also include a communication unit 1003. Specifically, the communication unit supports the image processing device 1000 in performing data transmission and data reception steps. The communication unit 1003 may be an input or output interface, pins, or circuits, etc.

[0305] In a possible embodiment, the image processing apparatus may further include a storage unit 1004. The processing unit 1002 and the storage unit 1004 are connected via a line. The storage unit 1004 may include one or more memories, which may be devices in one or more devices or circuits used for storing programs or data. The storage unit 1004 may exist independently and be connected to the processing unit 1002 of the image processing apparatus via a communication line. Alternatively, the storage unit 1004 may be integrated with the processing unit 1002.

[0306] Storage unit 1004 may store computer-executable instructions for the methods in the terminal device, so that processing unit 1002 executes the methods in the above embodiments. Storage unit 1004 may be a register, cache, or RAM, etc., and storage unit 1004 may be integrated with processing unit 1002. Storage unit 1004 may be a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, and storage unit 1004 may be independent of processing unit 1002.

[0307] The image processing method provided in this application can be applied to electronic devices with communication functions. The electronic devices include terminal devices, and the specific device form of the terminal devices can be referred to the above-described related descriptions, which will not be repeated here.

[0308] This application provides an electronic device, which includes a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the electronic device to perform the above-described method.

[0309] This application provides a chip system including at least one processor and a communication interface. The communication interface and the at least one processor are interconnected via a circuit. The at least one processor is used to run computer programs or instructions to execute the above-described method. Its implementation principle and technical effects are similar to the related embodiments described above, and will not be repeated here.

[0310] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the methods described above. The methods described in the above embodiments can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted over the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0311] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any other medium targeted to carry or to store the required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, Digital Versatile Discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0312] This application provides a computer program product, which includes a computer program that, when run, causes a computer to perform the above-described method.

[0313] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0314] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that, Applied to electronic devices, including: In response to the action used to launch the camera app, the first frame of the raw image is captured using the camera. The process involves: acquiring first 3A statistical data from the first frame of the original image; the first 3A statistical data including two or more different categories of 3A statistical data; detecting the shooting scene of the first frame of the original image; and, when the shooting scene meets preset conditions, inputting the first 3A statistical data into a first neural network model, which then outputs a first subject target brightness. The preset conditions include that the brightness difference between each pixel region in the first frame of the original image is less than a brightness difference threshold. The brightness difference between each pixel region in the first frame of the original image is obtained through the ambient light brightness value (alsLV) of a multispectral sensor and the ambient light brightness value (aeLV) of an image sensor. The brightness difference threshold includes a first brightness difference threshold and a second brightness difference threshold. The condition that the brightness difference between each pixel region is less than the brightness difference threshold includes: the difference between alsLV and aaeLV is less than the first brightness difference threshold, and / or the difference between alsLV and aaeLV is less than or equal to the second brightness difference threshold. The first brightness difference threshold is less than the second brightness difference threshold. During the acquisition of the first frame original image, the difference between the alsLV and aeLV of the first frame original image is less than the first brightness difference threshold; during the acquisition of the second frame original image to the m-th frame original image, the difference between the alsLV and aeLV of the second frame original image to the m-th frame original image is less than the second brightness difference threshold; during the acquisition of the (m+1)-th frame original image, the difference between the alsLV and aeLV of the (m+1)-th frame original image is greater than the second brightness difference threshold; wherein, the shooting scene of the second frame original image to the m-th frame original image is the target scene, and the shooting scene of the (m+1)-th frame original image is not the target scene; m is a positive integer greater than 2; in any of the multiple frames original images If the brightness difference between pixel regions in a frame is less than the first brightness difference threshold, then the original images of multiple frames after that frame enter the target scene determination process; until the brightness difference between pixel regions in a certain frame is greater than the second brightness difference threshold, then the original images of that frame exit the target scene determination process; when the original image of the previous frame has not entered the target scene determination process, the preset condition for the original image of the current frame is that the brightness difference between the alsLV value and the aelLV value is less than the first brightness difference threshold; when the original image of the previous frame enters the target scene determination process, the preset condition for the original image of the current frame is that the brightness difference between the alsLV value and the aelLV value is less than or equal to the second brightness difference threshold; when the preset conditions are met, the current scene is the target scene; when the preset conditions are not met, the current scene is not the target scene. The first target brightness is obtained based on the brightness of the first main target; the brightness of the first main target is positively correlated with the brightness of the first target. Based on the difference between the brightness of the first target and the brightness of the first frame original image, the exposure parameters are adjusted, and the adjusted exposure parameters are used to control the camera to capture the second frame original image. Obtain the second 3A statistical data from the second frame of the original image; the second 3A statistical data includes two or more different categories of 3A statistical data; When the electronic device captures the target scene, the second 3A statistical data is input into the first neural network model, and the first neural network model outputs the second subject target brightness; the second target brightness is obtained based on the second subject target brightness; the second subject target brightness and the second target brightness are positively correlated. Based on the difference between the second target brightness and the brightness of the second frame original image, the exposure parameters are adjusted, and the above steps are iteratively executed until the brightness of the currently acquired original image converges to the current target brightness.

2. The method according to claim 1, characterized in that, The target scene includes a black scene or a white scene, wherein the black scene includes a shooting scene in which the proportion of black area is greater than a first threshold, and the white scene includes a shooting scene in which the proportion of white area is greater than a second threshold.

3. The method according to claim 1 or 2, characterized in that, The brightness of the first main target is positively correlated with the first 3A statistical data; the brightness of the second main target is positively correlated with the second 3A statistical data.

4. The method according to claim 3, characterized in that, After obtaining the first target brightness based on the first subject target brightness, the method further includes: The first frame image is displayed, which is obtained by the electronic device after processing the original first frame image; wherein the image format of the original first frame image is different from the image format of the first frame image. After obtaining the second target brightness based on the second main target brightness, the method further includes: The second frame image is displayed; the second frame image is obtained by the electronic device after processing the original second frame image; wherein, the image format of the original second frame image is different from the image format of the second frame image; the brightness of the first frame image is different from the brightness of the second frame image; the brightness of the second frame image is related to the brightness of the original first frame image and the brightness of the first target image.

5. The method according to claim 4, characterized in that, When the target scene is a black scene, the brightness of the second frame image is less than the brightness of the first frame image; or, when the target scene is a white scene, the brightness of the second frame image is greater than the brightness of the first frame image.

6. The method according to any one of claims 1-2 and 4-5, characterized in that, The first 3A statistical data includes at least two of the following: the luminance value LV of the first frame original image, the block luminance of the first frame original image, the white balance gain coefficient AWB of the first frame original image, the grayscale histogram of the first frame original image, the lux index of the first frame original image, the color temperature of the first frame original image, and the ISO of the first frame original image. Specifically, the brightness of the first subject target is positively correlated with the LV of the first frame original image; the brightness of the first subject target is positively correlated with the block luma of the first frame original image; the brightness of the first subject target is positively correlated with the AWB gain of the first frame original image; the brightness of the first subject target is positively correlated with the grayscale histogram of the first frame original image; the brightness of the first subject target is positively correlated with the lux index of the first frame original image; the brightness of the first subject target is positively correlated with the color temperature of the first frame original image; and the brightness of the first subject target is positively correlated with the ISO of the first frame original image.

7. The method according to any one of claims 1-2 and 4-5, characterized in that, The preset conditions also include one or more of the following: the first frame of the original image does not contain facial information; during the acquisition of the first frame of the original image, the distance between the subject and the camera is less than a distance threshold; in the first frame of the original image, the brightness of the subject is greater than a brightness threshold; in the first frame of the original image, the ratio of the maximum color gain coefficient to the minimum color gain coefficient is less than a third threshold.

8. The method according to any one of claims 1-2 and 4-5, characterized in that, The step of obtaining the brightness of the first main target based on the first 3A statistical data further includes: The first 3A statistical data is input into the first neural network model to obtain the brightness of the first main target. Wherein, when the target scene is a black scene, the brightness of the first subject target is less than the first preset brightness, and the first preset brightness is the brightness obtained based on the grayscale histogram of the original image of the first frame; or, when the target scene is a white scene, the brightness of the first subject target is greater than the first preset brightness.

9. The method according to claim 8, characterized in that, The first neural network model is trained by the following method: obtaining a training dataset, which includes multiple samples, each of which includes the brightness of the subject target and multiple sample 3A statistical data, the multiple sample 3A statistical data corresponding to multiple sample images with different exposure values ​​under the same target scene; for any sample, inputting the multiple sample 3A statistical data into the model to be trained to obtain the predicted brightness of the subject target; training is completed when the value calculated by the loss function converges, and the first neural network model is obtained.

10. The method according to any one of claims 1-2, 4-5, and 9, characterized in that, The step of acquiring a second frame of original image using the camera based on the first target brightness includes: The camera's exposure value is adjusted based on the difference between the first target brightness and the brightness of the first frame original image; wherein, if the first target brightness is greater than the brightness of the first frame original image, the adjusted exposure value is greater than the original exposure value; or, if the first target brightness is less than the brightness of the first frame original image, the adjusted exposure value is less than the original exposure value. The camera is used to capture the second frame of the original image; during the capture of the second frame of the original image, the exposure value of the camera is the adjusted exposure value.

11. The method according to any one of claims 1-2, 4-5, and 9, characterized in that, After obtaining the second target brightness based on the second main target brightness, the process includes: The camera captures the nth frame of the original image based on the (n-1)th target brightness; where n is a positive integer greater than 2. Obtain the nth 3A statistical data from the nth frame of the original image; When the electronic device captures the target scene, the brightness of the nth subject target is obtained based on the nth 3A statistical data; the brightness of the nth subject target is positively correlated with the nth 3A statistical data. The brightness of the nth target is obtained based on the brightness of the nth main target; the brightness of the nth target is positively correlated with the brightness of the nth main target; the brightness of the original image in the nth frame converges to the brightness of the nth target.

12. The method according to claim 11, characterized in that, After obtaining the nth target brightness based on the nth subject target brightness, the method further includes: Received a command to take a picture; In response to the operation for taking a picture, the camera is used to capture and cache the original images from frame (n+1) to frame (n+1). The (n+2)th frame of the original image is post-processed to obtain the target image. The (n+2)th frame of the original image is the original image from the (n+1)th frame to the (n+1)th frame of the original image where the difference in brightness between the target image and the original image is the smallest. The original image includes a raw image, and the target image includes an RGB image or a YUV image. The target image has a higher resolution than the (n+2)th frame of the original image. The (n+2)th frame of the original image is the image displayed after processing. The (n+2)th frame of the original image includes an RGB image or a YUV image. Save the target image to the gallery application.

13. An electronic device, characterized in that, include: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the electronic device to perform the method as described in any one of claims 1-12.

14. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-12.

15. A chip system, characterized in that, It includes at least one processor and a communication interface, the communication interface and the at least one processor being interconnected via a line, the at least one processor being configured to run a computer program or instructions to perform the method as described in any one of claims 1-12.

16. A computer program product, characterized in that, Includes a computer program that, when run, causes a computer to perform the method as described in any one of claims 1-12.

Images