Image processing method and device

Abstract

The embodiment of the invention provides an image processing method and device, and relates to the technical field of terminals. The method comprises the steps that different photographing modes are used according to environment illumination and dynamic range values, the advantages of all the photographing modes are brought into full play, the appropriate photographing modes can be selected for different scenes, and therefore the image quality is improved.

Description

Image processing method and device Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to image processing methods and related devices. Background Art

[0002] Some electronic devices support high dynamic range (HDR) imaging. HDR generates images with a wider dynamic range, allowing for better detail in both bright and dark environments. For example, some applications on electronic devices can use HDR to improve image quality when providing preview, photo, or video capture.

[0003] However, in some shooting scenes using HDR mode, the shooting effect of electronic devices is still not very good.

[0004] Summary of the Invention

[0005] The image processing method and related device provided in the embodiments of the present application can use different shooting modes according to the ambient illumination and dynamic range value, give full play to the advantages of each shooting mode, so that different scenes can choose a more appropriate shooting mode, thereby improving image quality.

[0006] In a first aspect, an image processing method provided by an embodiment of the present application includes:

[0007] In response to a first user operation, the image sensor exposes a first image; in response to the first user operation, a preview interface is displayed, the preview interface including the first image; when the illuminance is less than a first illuminance threshold and the exposure ratio is less than or equal to a first gain threshold, the image sensor uses a first mode to expose a second image; when the illuminance is less than the first illuminance threshold and the exposure ratio is greater than the first gain threshold, the image sensor uses a second mode to expose a second image; when the illuminance is greater than or equal to the first illuminance threshold and less than a second illuminance threshold, and the exposure ratio is greater than the second gain threshold, the image sensor uses a third mode to expose a second image; the preview interface includes the second image; in response to a second user operation, the second image is saved; wherein the first mode, the second mode, and the third mode are different from each other. Since different shooting modes have different image output effects under different illuminances and / or different exposure ratios, in order to achieve better image output effects in various shooting scenes, the embodiments of the present application can determine a more appropriate shooting mode by judging the range of illuminance and / or the range of exposure ratio, thereby fully leveraging the advantages of each shooting mode, so that images with a high dynamic range and good quality can be obtained in different scenes.

[0008] In one possible implementation, the method further includes: when the illuminance is greater than or equal to a first illuminance threshold and less than a second illuminance threshold, and the exposure ratio is less than or equal to a second gain threshold, the image sensor uses the second exposure mode to obtain a second image; and when the illuminance is greater than or equal to the second illuminance threshold, the image sensor uses the third exposure mode to obtain the second image. Since neither the illuminance nor the dynamic range is particularly large, the image sensor is suitable for exposure in the second mode; when the illuminance is high and the light is bright, the image sensor is suitable for exposure in the third mode. Scenes with illuminance greater than or equal to the first illuminance threshold and less than the second illuminance threshold, and an exposure ratio less than or equal to the second gain threshold, can be understood as scenes with neither illuminance nor dynamic range being particularly large, and therefore can be imaged in the second mode; scenes with illuminance greater than or equal to the second illuminance threshold can be understood as scenes with high illuminance and bright light, and therefore can be imaged in the third mode. In this way, the advantages of the second and third modes can be fully utilized, thereby improving image quality.

[0009] In one possible implementation, when the image sensor uses the first exposure mode, the exposure ratio is positively correlated with the configured gain ratio. When the image sensor uses the second exposure mode, the exposure ratio is positively correlated with one or more of the following: the configured gain ratio, the conversion gain ratio of the long and short frames, and the exposure time of the long and short frames. The exposure of the long frame is greater than that of the short frame. Because the exposure ratio is positively correlated with the configured gain ratio in the first mode, when shooting in dark scenes, the configured gain of the long frame is higher. This allows the first mode to achieve a more suitable dynamic range by reducing the configured gain of the short frame, resulting in a larger dynamic range. Therefore, the first mode is suitable for use in darker scenes. Because the exposure ratio is positively correlated with the conversion gain ratio of the long and short frames in the second mode, based on the conversion gain ratio, the second mode can provide a certain degree of dynamic range expansion in brighter scenes. Therefore, the second mode is suitable for scenes with medium or high brightness. Because the exposure ratio in the third mode is positively correlated with the exposure times of the long and short frames, configuring the exposure times of the two frames allows for a wider dynamic range. Therefore, the third mode is suitable for brighter scenes. Thus, based on the different characteristics of each shooting mode, different shooting modes are suitable for different shooting scenarios.

[0010] In one possible implementation, the method further includes: when the image sensor uses the fourth exposure mode to obtain the first image, the first gain threshold is the configured gain of the first image; and when the image sensor uses the first exposure mode to obtain the first image, the first gain threshold is the configured gain of the long frame obtained by the image sensor using the first exposure mode. The first gain threshold is the configured gain of the first image. This scenario corresponds to a switch from the fourth mode to the first mode. Since the exposure amount of the fourth mode is the same as the exposure amount of the long frame in the first mode, the configured gain of the first image can be used as the first gain threshold. The first gain threshold is the configured gain of the long frame obtained by the image sensor using the first exposure mode. This scenario corresponds to a switch from the first mode to the second mode. Since the exposure ratio of the first mode is related to the configured gain, the configured gain of the long frame obtained by the image sensor using the first exposure mode can be used as the first gain threshold. In this way, different first gain thresholds can be calculated depending on the current capture mode, thereby enabling a reasonable prediction of the exposure ratio of the next image based on the relationship between the exposure ratio and the first gain threshold.

[0011] In one possible implementation, the method further includes: when the image sensor is exposed using the fourth mode to obtain the first image, the second gain threshold is positively correlated with the configuration gain of the first image and the conversion gain ratio of the long frame to the short frame obtained by the image sensor using the second mode; and when the image sensor is exposed using the second mode to obtain the first image, the second gain threshold is positively correlated with the configuration gain of the long frame obtained by the image sensor using the second mode to obtain the first image and the conversion gain ratio of the long frame to the short frame obtained by the image sensor using the second mode to obtain the first image. When the image sensor is exposed using the fourth mode to obtain the first image, scenarios for determining the second gain threshold may include switching from the fourth mode to the second mode or the third mode. Since only the configuration gain of the image can be obtained in the fourth mode, the conversion gain ratio may use the conversion gain ratio corresponding to the second mode. When the image sensor is exposed using the second mode to obtain the first image, scenarios for determining the second gain threshold may include switching from the second mode to the third mode. In this case, both the configuration gain and the conversion gain ratio may be derived based on the second mode. In this way, considering that factors affecting the exposure ratio vary in different capture modes, the second gain threshold may have different values.

[0012] In one possible implementation, the first mode includes a dual analog gain (DAG) mode, the second mode includes a DXG mode, the third mode includes a line-interleaved exposure (SHDR) mode, and the fourth mode includes a binning mode. The DXG mode is a combination of the dual conversion gain (DCG) mode and the DAG mode. Since the first mode can achieve long and short frames by configuring different gain settings, when shooting in dark scenes, the gain setting for long frames is higher. The first mode can reduce the gain setting for short frames to achieve a more appropriate dynamic range, allowing the first mode to have a larger dynamic range. Therefore, the first mode is suitable for use in darker scenes. The second mode can extend the dynamic range to a certain extent even in brighter scenes based on the conversion gain ratio, making the second mode suitable for medium and high-brightness scenes. The third mode can achieve a larger dynamic range by configuring a two-frame exposure time, making the third mode suitable for even brighter scenes. The fourth mode occupies less memory space, resulting in lower system power consumption and better performance. Since different shooting modes are suitable for different scenes, the embodiments of the present application can select the most appropriate shooting mode for different scenes, thereby achieving better shooting results.

[0013] In one possible implementation, the image sensor uses a first exposure mode to obtain a second image, including: transmitting a first identifier to the image sensor, the first identifier being used to indicate that the image sensor uses the first exposure mode; the image sensor uses a second exposure mode to obtain a second image, including: transmitting a second identifier to the image sensor, the second identifier being used to indicate that the image sensor uses the second exposure mode; and the image sensor uses a third exposure mode to obtain a second image, including: transmitting a third identifier to the image sensor, the third identifier being used to indicate that the image sensor uses the third exposure mode. In this way, using different identifiers to indicate the image sensor's shooting mode can achieve the purpose of message transmission more clearly and concisely, facilitate business logic maintenance, and improve code execution efficiency.

[0014] In one possible implementation, exposing the image sensor to obtain the first image specifically includes: when the exposure ratio is less than the exposure threshold, the image sensor uses the fourth exposure mode to obtain the first image. In this way, the single-frame photography mode occupies less memory space, resulting in lower system power consumption and better performance.

[0015] In one possible implementation, the method further includes obtaining illumination and exposure ratio; wherein the exposure ratio is the exposure ratio of the first image, and the illumination is the illumination of the current environment. Obtaining the illumination of the current environment allows for timely consideration of the impact of current environmental factors on the image sensor's output quality. Obtaining the exposure ratio of the first image allows for comparison with a gain threshold to determine whether the current capture mode covers the current dynamic range. This allows for more appropriate use of an output method that matches the current scene, thereby better capturing image details in both bright and dark light conditions and improving image quality.

[0016] In one possible implementation, an electronic device includes a first module, an automatic exposure (AE) module, and an ambient light sensor. Prior to obtaining the illuminance and exposure ratio, the electronic device further includes: the AE module calculating the exposure ratio based on the first image; obtaining the illuminance and exposure ratio includes: the first module obtaining the illuminance from the ambient light sensor; and the first module obtaining the exposure ratio from the AE module. In this way, the first module calculates the capture mode for the next image based on the exposure ratio and the current ambient illuminance, promptly accounting for the impact of current environmental factors on the image sensor's output, thereby improving image quality and thereby enhancing the user experience.

[0017] In a second aspect, an embodiment of the present application provides an image processing device, which may be an electronic device or a chip or chip system within an electronic device. The device may include a processing unit and a display unit. The processing unit is configured to implement any processing-related method performed by the electronic device in the first aspect or any possible implementation of the first aspect. The display unit is configured to implement any display-related method performed by the electronic device in the first aspect or any possible implementation of the first aspect. When the device is an electronic device, the processing unit may be a processor. The device may also include a storage unit, which may be a memory. The storage unit is configured to store instructions, and the processing unit executes the instructions stored in the storage unit to cause the electronic device to implement the method described in the first aspect or any possible implementation of the first aspect. When the device is a chip or chip system within an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit to cause the electronic device to implement the method described in the first aspect or any possible implementation of the first aspect. The storage unit may be a storage unit within the chip (e.g., a register, a cache, etc.) or a storage unit within the electronic device located outside the chip (e.g., a read-only memory, a random access memory, etc.).

[0018] Exemplarily, the display unit is configured to display a preview interface. The processing unit is configured to obtain a first image; further configured to, when the illuminance is less than a first illuminance threshold and the exposure ratio is less than or equal to a first gain threshold, use a first exposure mode to obtain a second image; further configured to, when the illuminance is less than the first illuminance threshold and the exposure ratio is greater than the first gain threshold, use a second exposure mode to obtain a second image; further configured to, when the illuminance is greater than or equal to the first illuminance threshold and less than the second illuminance threshold and the exposure ratio is greater than the second gain threshold, use a third exposure mode to obtain a second image; and specifically, further configured to save the second image.

[0019] In one possible implementation, the processing unit is used to obtain the second image by using the second exposure mode when the illuminance is greater than or equal to the first illuminance threshold and less than the second illuminance threshold, and the exposure ratio is less than or equal to the second gain threshold; and is also used to obtain the second image by using the third exposure mode when the illuminance is greater than or equal to the second illuminance threshold.

[0020] In one possible implementation, when the image sensor is exposed in the first mode, the exposure ratio is positively correlated with the configured gain ratio; when the image sensor is exposed in the second mode, the exposure ratio is positively correlated with one or more of the following: the configured gain ratio, the conversion gain ratio of the long frame and the short frame; when the image sensor is exposed in the third mode, the exposure ratio is positively correlated with one or more of the following: the configured gain ratio, the conversion gain ratio of the long frame and the short frame, and the exposure time of the long frame and the short frame.

[0021] In one possible implementation, when the image sensor uses the fourth mode exposure to obtain the first image, the first gain threshold is the configuration gain of the first image; when the image sensor uses the first mode exposure to obtain the first image, the first gain threshold is the configuration gain of the long frame obtained by the image sensor using the first mode exposure.

[0022] In one possible implementation, when the image sensor uses the fourth mode for exposure to obtain the first image, the second gain threshold is positively correlated with the configuration gain of the first image and the ratio of the conversion gains of the long frame and the short frame obtained by the image sensor using the second mode for exposure; when the image sensor uses the second mode for exposure to obtain the first image, the second gain threshold is positively correlated with the configuration gain of the long frame obtained by the image sensor using the second mode for exposure and the ratio of the conversion gains of the long frame and the short frame obtained by the image sensor using the second mode for exposure.

[0023] In one possible implementation, the first mode includes a dual analog gain DAG mode, the second mode includes a DXG mode, the third mode includes a row interleaved exposure SHDR mode, and the fourth mode includes a Binning mode.

[0024] In a possible implementation, the processing unit is configured to transmit a first identifier to the image sensor; further configured to transmit a second identifier to the image sensor; and further configured to transmit a third identifier to the image sensor.

[0025] In a possible implementation, the processing unit is configured to adopt a fourth exposure mode to obtain the first image.

[0026] In a possible implementation, the processing unit is configured to obtain illumination and exposure ratio.

[0027] In a possible implementation, the processing unit is configured to calculate an exposure ratio based on the first image; and is further configured to obtain illumination from an ambient light sensor and the exposure ratio from an AE module.

[0028] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory is used to store code instructions, and the processor is used to run the code instructions to execute the method described in the first aspect or any possible implementation of the first aspect.

[0029] In a fourth aspect, the present application provides a chip or chip system, comprising at least one processor and a communication interface, wherein the communication interface and the at least one processor are interconnected via a line, and the at least one processor is configured to execute a computer program or instruction to perform the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip may be an input / output interface, a pin, or a circuit.

[0030] In one possible implementation, the chip or chip system described above in this application further includes at least one memory, in which instructions are stored. The memory may be a storage unit within the chip, such as a register, a cache, etc., or a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.).

[0031] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0032] In a sixth aspect, an embodiment of the present application provides a computer program product comprising a computer program, which, when the computer program runs on a computer, enables the computer to execute the method described in the first aspect or any possible implementation of the first aspect.

[0033] It should be understood that the second to sixth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is a schematic diagram of a photographing mode applicable to various illumination levels and dynamic ranges provided by an embodiment of the present application;

[0035] FIG2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0036] FIG3 is a schematic diagram of a software structure of an electronic device provided in an embodiment of the present application;

[0037] FIG4 is a schematic diagram of a shooting interface of a camera provided in an embodiment of the present application;

[0038] FIG5 is a timing diagram of module interaction of an image processing method provided in an embodiment of the present application;

[0039] FIG6 is a schematic diagram of a photographing mode selection provided in an embodiment of the present application;

[0040] FIG7 is a schematic diagram of an image processing method provided in an embodiment of the present application;

[0041] FIG8 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] To facilitate a clear description of the technical solutions of the embodiments of the present application, some of the terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0043] 1. Dynamic range (DR): This refers to the range of brightness that a camera can capture, or its ability to simultaneously capture both brighter and darker areas. A camera's dynamic range measures the image sensor's ability to capture exposure details under varying lighting conditions.

[0044] As you can understand, a higher dynamic range means the camera can better handle high-contrast scenes while preserving details and preventing overexposure or underexposure. This means that for scenes shot in sunlight, scenes with strong lighting contrast, indoors, or at night, the camera can preserve more image details.

[0045] 2. Exposure value (EV): This reflects the brightness of an image and can also be understood as the total amount of light received by the image sensor. If the exposure value is too high, the image brightness is too high, which is considered overexposure. If the exposure value is too low, the image brightness is too dark, which is considered underexposure.

[0046] Exposure is determined by both exposure time and gain. Exposure is positively correlated with both time and gain. That is, for a given gain, a longer exposure time results in a higher exposure; and for a given exposure time, a higher gain results in a higher exposure.

[0047] The gain can include analog gain and digital gain, and the gain can be configured by the auto exposure (AE). The AE can configure different analog gain and / or digital gain based on the brightness of the shooting scene. For example, in brighter scenes, the AE can configure a smaller analog gain and / or digital gain; in darker scenes, the AE can configure a larger analog gain and / or digital gain.

[0048] In some implementations, the AE may configure the analog gain first, and when the analog gain reaches a maximum value, may configure the digital gain.

[0049] It can be understood that, since the gain can be configured by AE, for the convenience of description, the gain will be referred to as the configured gain in the following.

[0050] 3. Exposure ratio: used to indicate the ratio of the brightest exposure to the darkest exposure in the same image. It can also be understood as the ratio of light ratio or exposure amount.

[0051] 4. HDR mode: HDR mode is also called dual-frame HDR mode. HDR modes include dual conversion gain (DCG) mode, dual analog gain (DAG) mode, DXG mode, and staggered HDR mode.

[0052] (1.1)DCG model.

[0053] DCG mode, also known as pixel-level dual-gain mode, includes high conversion gain (HCG) and low conversion gain (LCG). HCG has a higher conversion gain, resulting in a better signal-to-noise ratio in dark areas. LCG has a lower conversion gain, ensuring better highlight detail.

[0054] DCG mode automatically switches between HCG and LCG based on lighting conditions, optimizing dynamic range and noise performance. For example, in low-light environments, DCG mode switches to HCG, achieving higher sensitivity and better capture of image details, reducing noise interference in image data, and providing clearer image data. In bright light environments, DCG mode switches to LCG, preventing overexposure and preserving details.

[0055] DCG mode allows you to capture two frames simultaneously in the time it takes to capture one frame. Of the two captured frames, one uses HCG for output, corresponding to a long exposure frame (also called a long frame), while the other uses LCG for output, corresponding to a short exposure frame (also called a short frame). A long frame is considered a frame with a larger exposure, while a short frame is considered a frame with a smaller exposure. The exposure of a long frame is greater than that of a short frame.

[0056] It is understandable that DCG mode allows for simultaneous exposure of both long and short frames, so there is no ghosting issue with DCG mode. Ghosting can be understood as the blurring of the resulting composite image due to inconsistent positions of objects in two adjacent frames caused by camera shake or object movement.

[0057] The DCG mode can also fuse long-exposure frames and short-exposure frames, and the fused image frames are used as the images output by the image sensor.

[0058] In DCG mode, since the exposure time of the two frames is the same, the exposure ratio is determined by the CG ratio of the two frames. The maximum CG ratio of the two frames is determined by the hardware circuit of the image sensor. CG can also be called conversion gain, and the CG ratio can also be called the conversion gain ratio.

[0059] It's understandable that image brightness is determined by both exposure and the conversion gain ratio. Since the maximum conversion gain ratio is determined by the image sensor's hardware circuitry, when the conversion gain ratio is constant, exposure can be adjusted by adjusting the configured gains of the long and short frames using AE, thereby determining image brightness.

[0060] (1.2)DAG mode.

[0061] DAG mode can also capture two frames simultaneously in the same frame time. DAG mode outputs images using a single conversion gain, meaning both frames are output using HCG or LCG. In a possible implementation, DAG mode can output two frames with different exposures by configuring the two amplifier circuits with different gain settings, thereby achieving the effect of long and short frames.

[0062] In DAG mode, the exposure time of the two frames is the same and both frames are output as HCG or LCG. Therefore, the exposure ratio is determined by the configured gain ratio. For example, if the configured gain ratio is 6 to 1, the exposure ratio is 6 times.

[0063] It is understandable that DAG mode can also expose long frames and short frames at the same time, so there is no ghosting problem in DAG mode.

[0064] (1.3)DXG mode.

[0065] DXG mode can be understood as a combination of DCG and DAG modes. DXG mode allows for dual conversion gain and allows for different configuration gains. DXG mode also allows for simultaneous exposure of two frames within a single frame, with one frame outputting using HCG and the other using LCG. Different configuration gains can also be configured for each frame. DXG mode exposes both long and short frames simultaneously, eliminating ghosting.

[0066] In some implementations, in scenes with a relatively low dynamic range, the DCG mode can be used to produce images; in scenes with a relatively high dynamic range, both the DCG mode and the DAG mode can be used to produce images.

[0067] In DXG mode, the exposure ratio of two frames is positively correlated with the conversion gain ratio of the two frames and the configuration gain ratio. For example, if the conversion gain ratio of HCG and LCG is 4 to 1, and the configuration gain ratio of HCG and LCG is 3 to 1, the exposure ratio is 12 times.

[0068] It is understood that in the DXG mode, when the configuration gain ratio of the HCG and LCG is 1:1, it can also be understood as the DCG mode. In other words, the DXG mode includes the DCG mode.

[0069] (1.4)Stagger HDR mode.

[0070] Stagger HDR mode is also called SHDR mode. In SHDR mode, you can configure the long frame exposure time, short frame exposure time, and gain separately.

[0071] In SHDR mode, the exposure ratio of two frames is positively correlated with the conversion gain ratio of the two frames, exposure time, and configuration gain. For example, if the conversion gain ratio of HCG and LCG is 4 to 1, the exposure time ratio of HCG and LCG is 4 to 1, and the configuration gain ratio of HCG and LCG is 2 to 1, the exposure ratio is 32 times.

[0072] 5. Binning Mode: Binning mode, also known as pixel binning or merging mode, is a single-frame capture mode in which the image sensor captures a single frame at a time. It's understandable that binning mode is suitable for scenes with a relatively low dynamic range.

[0073] 6. Terminology

[0074] In the embodiments of this application, terms such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. For example, the terms "first chip" and "second chip" are used solely to distinguish between different chips and do not define their order. Those skilled in the art will understand that terms such as "first" and "second" do not define the quantity or execution order, and do not necessarily define differences.

[0075] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0076] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0077] Some electronic devices support HDR mode. Using HDR mode can produce images with a higher dynamic range, better rendering image details in both bright and dark light conditions. For example, some applications on electronic devices can use HDR mode when providing preview, photo, or video capture functions, thereby improving image quality.

[0078] In some implementations, the electronic device can determine the image sensor's image output mode based on the zoom ratio and dynamic range of the captured scene. For example, when the zoom ratio is within a certain range, the image sensor can output the image in the DCG mode of the HDR mode.

[0079] However, the DCG mode achieves different CGs by configuring two capacitors, namely HCG and LCG. The exposure ratio of the DCG mode can be understood as the ratio of HCG to LCG. For example, if the CG ratio of HCG to LCG is 4 to 1, the exposure ratio is 4 times. Since HCG and LCG are controlled by the size of the capacitor device, and the larger the capacitance, the larger the corresponding circuit area, the exposure ratio of the DCG mode is limited by the hardware conditions of the electronic device and cannot meet the requirements of a higher dynamic range. When shooting scenes in a higher dynamic range, the shooting effect is still not very good.

[0080] In addition, among the various HDR modes mentioned above, the DAG mode is suitable for darker scenes. In darker scenes, the required exposure is larger, and the DAG mode can achieve long and short frames by configuring different configuration gains. For example, when shooting in dark scenes, since the configuration gain value of the long frame is higher, the short frame can obtain a more appropriate dynamic range by reducing the configuration gain, so that the DAG mode can have a larger dynamic range. In other words, the darker the scene being shot, the larger the dynamic range that the DAG mode can cover, so the DAG mode is suitable for use in darker scenes. And considering that HCG has a better signal-to-noise ratio than LCG, in the example of this application, the DAG mode can use HCG to output the image.

[0081] DXG mode is suitable for scenes with medium to high brightness. Because DXG mode uses HCG and LCG for image output, the conversion gain ratio of HCG and LCG enables the image sensor to have a certain dynamic range expansion capability even in relatively bright scenes. Considering the poor signal-to-noise ratio of LCG in low-light scenes, DXG mode is used for scenes with medium to high brightness.

[0082] SHDR mode is suitable for high-dynamic scenes with medium or high brightness, or scenes with high brightness. Since SHDR mode can separately configure the exposure time of long frames and short frames, in brighter scenes, the required exposure is smaller and the configured gain ratio is lower. SHDR mode can obtain a larger dynamic range by configuring the exposure time of two frames. And considering that long and short frames are exposed alternately in SHDR mode, in darker scenes, the long frame needs to be configured with a longer exposure time. However, the exposure interval between long and short frames is longer, which is prone to ghosting problems in motion scenes. Therefore, SHDR mode is suitable for brighter scenes.

[0083] In view of this, the image processing method provided in the embodiment of the present application can use different shooting modes according to the ambient illumination and dynamic range value, give full play to the advantages of each shooting mode, so that different scenes can choose a more appropriate shooting mode, thereby improving image quality.

[0084] FIG1 is a schematic diagram showing photographing modes applicable to various illumination levels and dynamic ranges according to an embodiment of the present application.

[0085] As shown in Figure 1, the dynamic range increases from left to right along the horizontal axis. The illuminance increases from bottom to top along the vertical axis. Illuminance can reflect light intensity and can also be referred to as brightness. An increase in illuminance can also be understood as a transition from dark to bright.

[0086] As shown in Figure 1, for scenes with a lower dynamic range, such as a dynamic range between 0 and x1, within the image sensor's single-frame dynamic range capability, binning mode can be used for image output. This ensures lower system power consumption and better performance.

[0087] As the dynamic range increases, you can switch from single-frame mode to dual-frame mode. For example, a dynamic range between x1 and x4 can be used in dual-frame mode. This allows for better rendering of image details in both bright and dark conditions, improving image quality.

[0088] Among them, the dual-frame shooting mode can include DAG mode, DXG mode, SHDR mode, etc. As shown in Figure 1, within the dynamic range x1-x4, the area between the horizontal axis and the broken line 10 can be applied to the DAG mode, the area between the broken line 10 and the broken line 20 can be applied to the DXG mode, and the area above the broken line 20 can be applied to the SHDR mode.

[0089] In this embodiment of the present application, the broken line 10 may include a first portion parallel to the horizontal axis and a second portion intersecting the horizontal axis. The first portion of the broken line 10 corresponds to an illumination between 0 and y1 and a dynamic range between x1 and x2. Since the DAG mode is suitable for darker scenes, and the illumination between 0 and y1 is relatively dim, the DAG mode may be used.

[0090] It is understandable that, starting from the turning point of the broken line 10, as the dynamic range continues to increase, for example, the dynamic range gradually increases from x2 to x4, the DAG mode cannot continue to cover a larger dynamic range.

[0091] This is because DAG mode achieves long and short frames by configuring different configuration gains. In DAG mode, AE can determine the ratio of the configuration gain of long frames to the configuration gain of short frames based on the dynamic range of the current scene. For example, in brighter scenes, the required exposure is smaller, so when the exposure time remains unchanged, the configuration gain ratio is relatively small; in darker scenes, the required exposure is larger, so when the exposure time remains unchanged, the configuration gain ratio is relatively large. It can also be understood that as the dynamic range increases, parts of the scene in the image become brighter and brighter, and the dynamic range that can be covered by DAG mode becomes smaller and smaller.

[0092] In brighter scenes, increasing the gain of the long frame configuration while keeping the exposure time constant will increase the exposure, potentially causing multiple frames to appear overexposed. Shortening the exposure time, on the other hand, significantly increases the signal-to-noise ratio, degrading the image quality. Therefore, continuing to use DAG mode when the dynamic range continues to increase may result in poorer results.

[0093] In the embodiment of the present application, as the illumination increases and the dynamic range increases, when the DAG mode cannot cover a larger dynamic range, the DXG mode can be used to output the image.

[0094] Polyline 20 may also include a first portion parallel to the horizontal axis and a second portion gradually approaching the horizontal axis. The first portion of polyline 20 corresponds to an illumination between y1 and y2 and a dynamic range between x1 and x3. It is understood that since DXG mode is suitable for medium to high-brightness scenes, DXG mode can be used when the illumination is between y1 and y2 and the dynamic range is between x1 and x3, as the light becomes brighter and / or the dynamic range increases.

[0095] It is understandable that, starting from the turning point of the broken line 20, as the dynamic range continues to increase, for example, the dynamic range gradually increases from x3 to x4, the DAG mode cannot continue to cover a larger dynamic range.

[0096] Assuming the configured gain for long frames in DXG mode is 4 times that of short frames, and the conversion gain ratio between HCG and LCG is 4:1, DXG mode can cover a dynamic range of 16 times, exceeding the larger dynamic range covered by DAG mode. In this case, x3 corresponds to a dynamic range of 16 times. However, as the dynamic range increases further, DXG mode cannot cover the larger dynamic range. Continuing to use DXG mode may result in poorer shooting results.

[0097] Therefore, in the embodiment of the present application, as the illumination continues to increase and the dynamic range continues to increase, when the DXG mode cannot cover a larger dynamic range, the SHDR mode can be used to output the image.

[0098] The area above line 20 corresponds to an illuminance greater than or equal to y2, and a dynamic range between x1 and x4. Because SHDR mode allows for separate configuration of exposure times and gain for long and short frames, it can cover higher brightness levels and a wider dynamic range. Therefore, SHDR mode is suitable for high-dynamic range scenes with medium or high brightness levels, or scenes with high brightness.

[0099] It is understandable that the electronic device of the embodiment of the present application may also be a terminal device in any form. For example, the electronic device may include: a mobile phone, a tablet computer, a PDA, a laptop computer, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a wireless terminal in a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a wireless terminal in a smart home, a wireless terminal in a smart grid, a wireless terminal in a smart city, a wireless terminal in a smart home, a wireless terminal in a smart home, a wireless terminal in a smart home, a wireless terminal in a smart home, a wireless terminal in a smart grid, a wireless terminal in a smart city, a wireless terminal in a smart home, a wireless terminal in a smart home, a wireless terminal in a smart home, a wireless terminal in a smart grid ... home, a wireless terminal in a smart home, a wireless terminal in a smart home, a wireless terminal in a smart home, a wireless terminal in a smart home, a wireless terminal in a smart home, assistant, PDA), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to a wireless modem, vehicle-mounted devices, wearable devices, electronic devices in 5G networks, or electronic devices in future evolved public land mobile communication networks (public land mobile network, PLMN), etc., and the embodiments of the present application are not limited to this.

[0100] As an example and not a limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are fully functional, large in size, and can achieve complete or partial functions without relying on smartphones, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0101] In addition, in the embodiment of the present application, the electronic device can also be an electronic device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network of human-machine interconnection and object-to-object interconnection.

[0102] The electronic devices in the embodiments of the present application may also be referred to as: user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0103] In the embodiments of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory (also known as main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as the Linux operating system, the Unix operating system, the Android operating system, the iOS operating system, or the Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0104] For example, FIG2 shows a schematic structural diagram of an electronic device.

[0105] The electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0106] It is understood that the structures illustrated in the embodiments of the present invention do not constitute specific limitations on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The components shown in the diagram may be implemented in hardware, software, or a combination of software and hardware.

[0107] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors. The controller may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. For example, in an embodiment of the present application, the GPU may be used to process images captured by an image sensor, for example, including color correction, white balance, noise reduction, color enhancement, etc. The ISP may be used to parse and calculate RAW images, for example, including converting RAW images into YUV format images and performing histogram statistics and RGB channel statistics on the images. The DSP may be used to perform digital signal processing on the images, for example, including light compensation, lens distortion correction, exposure control, etc.

[0108] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or is reusing. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system. For example, in an embodiment of the present application, the processor 110 may be used to process the determination of illumination and / or exposure ratio, the selection of a photographing mode, the output of an image, and the like.

[0109] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present invention is only a schematic illustration and does not constitute a structural limitation of the electronic device. In other embodiments of the present application, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0110] The internal memory 121 can be used to store computer executable program code, and the executable program code includes instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function, etc. The data storage area may store data created during the use of the electronic device, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor. For example, in an embodiment of the present application, the internal memory 121 can be used to store a photo-taking instruction, a flag corresponding to a photo-taking mode, relevant codes of an image processing method, etc.

[0111] Camera 193 is used to capture still images or videos. In some embodiments, the electronic device may include one or N cameras 193, where N is a positive integer greater than 1. For example, in the embodiments of the present application, camera 193 can be used to capture images in scenarios such as previewing, taking photos, recording, or watching videos. A camera may also be referred to as an image sensor.

[0112] Figure 3 is a block diagram of the software structure of an electronic device according to an embodiment of the present application. The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system and hardware are layered, from top to bottom, into the application layer, application framework layer, hardware abstraction layer (HAL), driver layer, and hardware layer.

[0113] The application layer, also known as the application layer, can include a series of application packages. As shown in Figure 3, an application package can include applications such as camera, gallery, and video. Applications can include system applications and third-party applications.

[0114] The application framework layer can also be called the application framework layer or the Framework layer. The Framework layer can provide an application programming interface (API) and a programming framework for the application layer applications. The Framework layer can include some predefined functions.

[0115] As shown in Figure 3, the Framework layer can include a camera access interface. The camera access interface includes camera management and camera devices. The camera application in the application layer can call the camera access interface to manage the camera device.

[0116] The application layer and framework layer run in a virtual machine. The virtual machine executes the Java files in the application and framework layers as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection. For example, in the embodiments of the present application, the virtual machine can be used for functions such as determining illumination and / or exposure ratio, selecting a camera mode, and outputting images.

[0117] The hardware abstraction layer (HAL) encapsulates hardware drivers, providing a unified interface for upper-layer applications. The HAL includes the camera HAL and the camera algorithm library. The camera HAL can include multiple camera devices. The camera algorithm library includes the decision module and the automatic image processing (AE) module.

[0118] The decision module can be used to determine the corresponding shooting mode according to the illumination and dynamic range. Optionally, the decision module can also be implemented in other layers, which is not limited in the embodiment of the present application.

[0119] The AE module can be used to automatically adjust the exposure time and configuration gain of the image sensor to adjust the image brightness when the external ambient lighting conditions change, so that it is within a more appropriate brightness range, thereby obtaining a higher quality image.

[0120] The driver layer is used to drive hardware resources. The driver layer may include multiple driver modules. The driver layer may include camera device drivers, DSP drivers, and GPU drivers, among others. For example, in an embodiment of the present application, the camera algorithm library may be used to send digital signals to the DSP driver so that the DSP driver can call the DSP in the hardware layer to perform digital signal processing. The DSP can return the processed digital signals to the camera algorithm library via the DSP driver. The camera algorithm library is also used to send digital signals to the GPU driver so that the GPU driver can call the GPU in the hardware layer to perform digital signal processing. The GPU can return the processed graphics data to the camera algorithm library via the GPU driver.

[0121] The hardware layer includes image sensors, image signal processors (ISPs), digital signal processors (DSPs), and graphics processors (GPUs). The image sensor can generate images based on its camera mode, and the ISP can process the images generated by the image sensor.

[0122] It should be noted that the embodiments of the present application are only illustrated using the Android system as an example. In other operating systems (such as Windows system, IOS system, etc.), as long as the functions implemented by each functional module are similar to those in the embodiments of the present application, the solutions of the present application can also be implemented.

[0123] The following description uses the example of a photo-taking scenario in Figure 4 . As shown in FIG4 a, interface 401 is the preview interface of the camera application, which can also be referred to as the camera application's shooting interface. Interface 401 may include a viewfinder, zoom factor, shooting mode area, album icons, shooting controls, and camera rotation controls.

[0124] The viewfinder can be used to obtain a preview image and display the preview image in real time.

[0125] The zoom ratio may be set to 1X by default, and the user may select a different zoom ratio by operating on the touch screen when using the shooting function.

[0126] The shooting mode area may include aperture, night scene, portrait, photo, video, short video, more 402, etc.

[0127] The album icon displays thumbnails of images. Users tap the album icon to view captured photos or videos. Users tap the capture control to take a photo. The camera rotation control switches cameras.

[0128] It is understandable that users can turn on the HDR mode when shooting. This mode can obtain images with a higher dynamic range, making the shooting effect better and improving the user experience.

[0129] Exemplarily, the user may click on More 402 in the interface 401 . In response to the user-triggered operation of clicking on More 402 , as shown in FIG4 b , the interface 403 is displayed, and the interface 403 may include an HDR mode 404 .

[0130] When the user triggers the HDR mode 404 , in response to the triggering operation, the camera application may turn on the HDR mode.

[0131] It is understandable that the above may be one way to turn on the HDR mode, and interface 401 may also include other ways to turn on the HDR mode.

[0132] Exemplarily, the interface 401 may also include controls corresponding to the HDR mode (not shown in the figure). For example, the controls for the HDR mode may be located in the upper half of the interface 401, or in the shooting mode area, or in any possible position in the interface 401. This is not limited in the embodiments of the present application.

[0133] It is understandable that the electronic device can also automatically enter HDR mode according to the current shooting environment.

[0134] It is understood that the photo-taking scene in FIG4 is merely an example of an application scenario of the embodiment of the present application and does not limit the embodiment of the present application. The embodiment of the present application can also be applied to other scenarios using image sensor devices, such as video recording, video calling, and live video broadcasting.

[0135] FIG5 shows a module interaction timing diagram of the image processing method according to an embodiment of the present application.

[0136] S501: Receive an operation of a user triggering to open a camera application.

[0137] The user can trigger the operation of opening the camera application, where the operation of opening the camera application may include the operation of clicking the camera application icon, the operation of voice input to open the camera application instruction, or the operation of triggering the shortcut key on the electronic device, etc. The specific operation of opening the camera application is not limited in the embodiments of this application.

[0138] After the electronic device receives the user's operation to open the camera application, it can open the camera application of the application layer in response to the operation. The interface of the camera application can be as shown in Figure 4a above. The camera interface 401 can display a preview image. When the camera application displays the preview image, step S502 can be executed.

[0139] S502: The camera application transmits an instruction to capture an image.

[0140] The camera application can call the camera access interface of the application framework layer to pass an instruction, which can be used to request the image sensor to capture an image. It can be understood that the image sensor capturing an image can be called sensor exposure.

[0141] The camera access interface can pass this instruction to the camera hardware abstraction layer, which in turn passes this instruction to the camera device driver in the driver layer. Upon receiving this instruction, the camera device driver can instruct the image sensor in the hardware layer to start operating, perform image exposure, and generate a RAW image. After receiving the instruction from the camera device driver, the image sensor can proceed to step S503.

[0142] S503: The image sensor uses a default configured photographing mode to perform image exposure and generate a RAW image.

[0143] When the image sensor captures the first frame of image, it may use the default configured photo mode to output the image. The default configured photo mode may include a photo mode already configured in the image sensor, such as a Binning mode, or may include a photo mode last used by the image sensor. The specific default configured photo mode is not limited in the embodiments of this application.

[0144] After the image sensor generates the RAW image, step S504 may be further performed.

[0145] S504: The image sensor transmits the RAW image to the ISP.

[0146] S505: ISP processes the RAW image.

[0147] After acquiring the RAW image, the ISP can perform analysis and calculations on it. For example, the ISP can convert the RAW image into a YUV format image. The ISP can also perform histogram statistics and RGB channel statistics on the image to obtain processed image information.

[0148] After the ISP processes the RAW image, step S506 may be executed.

[0149] S506: The ISP returns the processed image information.

[0150] The ISP may pass the processed image information to the camera device driver, and the camera device driver may pass the processed image information to the camera hardware abstraction layer.

[0151] After obtaining the processed image information, the camera hardware abstraction layer can, on the one hand, execute step S507 to return the processed image information to the camera application for display; on the other hand, it can execute step S509 to pass the processed image information to the AE module of the camera algorithm library.

[0152] It is understandable that the execution of step S507 and step S509 does not distinguish between the order of priority. Step S507 can be executed first, or step S509 can be executed first, or step S507 and step S509 can be executed at the same time. This embodiment of the present application does not limit this.

[0153] S507: The camera hardware abstraction layer returns the processed image information to the camera application.

[0154] The camera hardware abstraction layer may pass processed image information to the camera access interface, and the camera access interface may return the processed image information to the camera application.

[0155] S508: The camera application displays the image.

[0156] S509: The camera hardware abstraction layer transmits the processed image information to the AE module.

[0157] S510: The AE module calculates parameter information of the next image.

[0158] The AE module can calculate the exposure of the next image based on the information of the current image and query the exposure time and gain corresponding to the exposure of the next image from the exposure table, thereby adjusting the brightness of the image based on the exposure time and gain. The electronic device can be equipped with an exposure meter, and the exposure meter can be different under different illumination conditions.

[0159] After the AE module calculates the parameter information of the image, on the one hand, it can execute step S511 to transmit the exposure ratio and gain to the decision module; on the other hand, it can execute step S512 to transmit the parameter information such as exposure time, exposure ratio and / or gain of the next image to the image sensor.

[0160] It can be understood that the execution of step S511 and step S512 does not distinguish between the order of priority. Step S511 can be executed first, or step S512 can be executed first, or step S511 and step S512 can be executed at the same time. This embodiment of the present application does not limit this.

[0161] S511. The AE module transmits information such as exposure ratio and gain to the decision module.

[0162] S512: The AE module transmits parameter information such as exposure time, exposure ratio and / or gain to the image sensor.

[0163] The AE module can transmit parameter information such as exposure time, exposure ratio and / or gain to the camera device driver, and the camera device driver can transmit parameter information such as exposure time, exposure ratio and / or gain to the image sensor.

[0164] S513: The decision module obtains illumination information.

[0165] It is understandable that the electronic device may include a sensor for detecting ambient illumination information, such as an ambient light sensor, etc. Taking the ambient light sensor as an example, when the ambient light sensor detects a change in ambient illumination, the ambient light sensor may transmit the illumination information to the decision module.

[0166] It is understandable that the decision module can also receive illumination information transmitted by the AE module.

[0167] It is understood that the execution order of steps S511 and S513 is not specific. In other words, the decision module acquires information such as exposure ratio and gain, and the decision module acquires illumination information in any order. The decision module may acquire information such as exposure ratio and gain first, or may acquire illumination information first, or may acquire information such as exposure ratio and gain simultaneously, and this is not limited in this embodiment of the present application.

[0168] S514: The decision module calculates the photographing mode of the next image based on the illumination information, exposure ratio, gain, etc.

[0169] The decision module can calculate the shooting mode of the next image based on illumination information, exposure ratio, gain, etc.

[0170] It is understood that if the sensor's photographing mode needs to be switched, the decision module can execute step S515 to output a flag corresponding to the photographing mode. The specific method for calculating the photographing mode can refer to the relevant description of the embodiment corresponding to FIG6 below, and will not be repeated here.

[0171] It is understood that the flag bit of the camera mode can be customized, and the data type of the flag bit can include integer, string, etc. For example, the flag bit corresponding to Binning mode is 0, the flag bit corresponding to DAG mode is 1, the flag bit corresponding to DXG mode is 2, and the flag bit corresponding to SHDR mode is 3. The flag bits of different modes can also be set to other values. The specific setting of the flag bit of the camera mode is not limited in the embodiments of the present application.

[0172] S515: The decision module transmits a flag corresponding to the photographing mode to the image sensor.

[0173] The decision module may transmit a flag bit corresponding to the photographing mode to the camera device driver, and the camera device driver may transmit a flag bit corresponding to the photographing mode to the image sensor.

[0174] S516 : The image sensor performs image exposure based on the information transmitted by the AE module and the decision module to generate a RAW image.

[0175] After the image sensor generates a RAW image based on information such as the photographing mode, the above steps S504 to S516 may be repeatedly executed, which will not be described in detail.

[0176] It is understood that during the capture process, multiple frames may be captured, so the interaction between the modules in Figure 5 can occur multiple times. During each interaction, the capture mode, exposure time, and / or gain parameters of the next image can be adjusted. In this way, by continuously adjusting the parameters of the next image, the quality of the captured image can be improved, thereby enhancing the user experience.

[0177] In a possible scenario, in the camera application interface 401, the user can trigger the operation of clicking the shooting control, and in response to this operation, the camera application can take a picture. When the camera application takes a picture, the corresponding data transmission process can also refer to the description of the embodiment corresponding to Figure 5 above, and will not be repeated here.

[0178] When the sensor switches from single-frame binning to dual-frame mode, there's no restriction on whether to use DAG, DXG, or SHDR. The specific mode used can be calculated by the decision module based on environmental factors. For example, when using the camera app, you can switch from single-frame binning to dual-frame DAG mode; the next time you use the camera app, you can switch from single-frame binning to dual-frame DXG mode.

[0179] It is understandable that the changing trends of dynamic range and exposure ratio are consistent. For example, the larger the dynamic range, the larger the exposure ratio. Therefore, the embodiment of the present application can use the exposure ratio for judgment in the photo mode.

[0180] Exemplarily, FIG6 shows a schematic diagram of photographing mode selection implemented by the decision module.

[0181] S601: Determine whether the exposure ratio is less than the exposure threshold.

[0182] For example, the decision module may determine whether to adopt single-frame exposure or double-frame exposure according to the exposure ratio.

[0183] If the exposure ratio is less than the exposure threshold, it means that the current dynamic range is small and single-frame exposure can be used, then step S602 is executed.

[0184] If the exposure ratio is greater than or equal to the exposure threshold, it means that the current dynamic range is large and double-frame exposure can be used, and step 603 can be executed.

[0185] Optionally, the case where the exposure threshold is equal to the exposure threshold can also be judged as a small dynamic range, and single-frame exposure is adopted, which is not limited in the embodiment of the present application.

[0186] The exposure threshold can be customized by the decision module, and the specific exposure threshold is not limited in the embodiment of the present application.

[0187] S602: Select single frame mode.

[0188] Within the image sensor's single-frame dynamic range, binning mode can be used for image output. Because binning mode is a single-frame capture mode, it occupies less memory space than dual-frame capture mode. Therefore, using binning mode for image output ensures lower system power consumption and better performance.

[0189] S603: Determine whether the illumination is less than a first illumination threshold.

[0190] When the exposure ratio is greater than or equal to the exposure threshold, the decision module may determine whether the illuminance is less than a first illuminance threshold.

[0191] If the illuminance is less than the first illuminance threshold, it means that the current illuminance is small and the light is dark. The decision module can further execute step S604 to determine the exposure ratio.

[0192] If the illuminance is greater than or equal to the first illuminance threshold, it means that the current illuminance is relatively high, and the decision module may execute step S606 to make further judgment on the illuminance.

[0193] Optionally, the case where the illumination is equal to the first illumination threshold can also be judged as low illumination, and step S604 is executed, which is not limited in the embodiment of the present application.

[0194] The first illumination threshold can be customized by the decision module. The specific first illumination threshold is not limited in the embodiment of the present application.

[0195] S604: Determine whether the exposure ratio is greater than a first gain threshold.

[0196] It is understandable that the decision module can calculate the first gain threshold according to the current mode to predict the exposure ratio of the next image.

[0197] For example, if the current mode is Binning, the first gain threshold may be the gain of the image sensor when exposing the image in Binning mode. The single-frame exposure in Binning mode may be the same as the long-frame exposure in DAG or DXG mode.

[0198] If the current mode is the DAG mode, the first gain threshold may be a gain of a long frame obtained by the image sensor when exposing in the DAG mode.

[0199] It is understandable that under different illumination conditions, since the calculated gains of the long frames are different, the first gain threshold also takes different values.

[0200] When the illuminance is less than the first illuminance threshold, it can be determined whether the exposure ratio is greater than the first gain threshold.

[0201] If the exposure ratio is less than or equal to the first gain threshold, it indicates that the dynamic range is small and the DAG mode can cover the dynamic range. The decision module can execute step S605 and use the DAG mode to output the image.

[0202] If the exposure ratio is greater than the first gain threshold, it indicates that the dynamic range is large and the DAG mode cannot cover the large dynamic range. To ensure better preview and photo effects, the decision module can execute step S608 and use the DXG mode with a larger dynamic range coverage to produce the image.

[0203] Optionally, the case where the exposure ratio is equal to the first gain threshold can also be determined as a large dynamic range, and step S608 is executed, which is not limited in this embodiment of the present application.

[0204] S605: Select DAG mode.

[0205] The DAG mode is suitable for scenes with low light. The specific DAG mode can be referred to the relevant description in (1.2) DAG mode above, which will not be repeated here.

[0206] S606: Determine whether the illumination is less than a second illumination threshold.

[0207] When the illuminance is greater than or equal to the first illuminance threshold, the decision module may determine whether the illuminance is less than a second illuminance threshold.

[0208] If the illuminance is less than the second illuminance threshold, it means that the current illuminance is not particularly large, and the decision module may further execute step S607 to determine the exposure ratio.

[0209] If the illuminance is greater than or equal to the second illuminance threshold, it means that the current illuminance is large and the light is bright. The decision module can execute step S609 and use the SHDR mode suitable for high-brightness scenes to output the image.

[0210] Optionally, the case where the illumination is equal to the second illumination threshold can also be judged as the illumination is not particularly large, and step S607 is executed, which is not limited in the embodiment of the present application.

[0211] The second illumination threshold can be customized by the decision module. The specific second illumination threshold is not limited in the embodiment of the present application.

[0212] S607: Determine whether the exposure ratio is greater than a second gain threshold.

[0213] Similar to the method of calculating the first gain threshold, the decision module can calculate the second gain threshold according to the current mode to predict the exposure ratio of the next image.

[0214] For example, if the current mode is Binning mode, the second gain threshold may be positively correlated with the gain of an image obtained by the image sensor using Binning mode exposure and the conversion gain ratio of long frames and short frames obtained by the image sensor using DXG mode exposure.

[0215] If the current mode is DXG mode, the second gain threshold may be positively correlated with the gain of the long frame obtained by the image sensor when exposed in DXG mode, and the conversion gain ratio of the long frame to the short frame obtained by the image sensor when exposed in DXG mode.

[0216] The first gain threshold and the second gain threshold may be the same or different.

[0217] When the illuminance is less than the second illuminance threshold, it can be determined whether the exposure ratio is greater than the second gain threshold.

[0218] If the exposure ratio is less than or equal to the second gain threshold, it means that the current dynamic range is not particularly large, and then step S608 can be executed to use DXG to output the image.

[0219] If the exposure ratio is greater than the second gain threshold, it means that the current dynamic range is large and it is a high dynamic scene. Then, step S609 can be executed to use the SHDR mode of the high dynamic scene with medium and high brightness to output the image.

[0220] Optionally, the case where the gain is equal to the second gain threshold can also be determined as a high dynamic scene, and step S609 is executed, which is not limited in this embodiment of the present application.

[0221] S608. Select DXG mode.

[0222] It is understood that DXG mode can be used when the illumination is greater than a certain threshold, and / or in scenes with a high dynamic range, so that the electronic device can achieve better preview and photo effects. The specific description of DXG mode can be referred to in the above (1.3) DXG mode, and will not be repeated here.

[0223] S609. Select SHDR mode.

[0224] The SHDR mode is suitable for high dynamic scenes with medium or high brightness or high brightness scenes. The specific SHDR mode can be referred to the relevant description in the above (1.4) SHDR mode, which will not be repeated here.

[0225] It is understood that the order of determining the illumination and the exposure ratio in the above steps can be determined first, followed by determining the illumination, and this is not limited in the present embodiment. For example, the above steps S603 and S604 can be executed in any order, and the execution order of steps S606 and S607 can also be executed in any order.

[0226] Optionally, when the image sensor captures the first image, the process of the embodiment corresponding to FIG. 6 may not be performed. In some implementations, when the image sensor captures the first image, the image sensor may use a preset default photography mode, or the image sensor may use a previously recorded photography mode to capture the first image. This is not limited in the present embodiment.

[0227] The following describes the method of the embodiment of the present application in detail through specific embodiments. The following embodiments can be combined with each other or implemented independently, and the same or similar concepts or processes may not be repeated in some embodiments.

[0228] FIG7 shows an image processing method according to an embodiment of the present application. The method includes:

[0229] S701 : In response to a first operation of a user, an image sensor is exposed to obtain a first image.

[0230] In the embodiment of the present application, the first operation can be understood as the operation of opening a camera application or other application that uses an image sensor. The first operation can include the operation of clicking an application icon, or the operation of opening an application by voice instruction, etc. The specific implementation method of the first operation is not limited in the embodiment of the present application.

[0231] The first image may be understood as any frame of image captured by the image sensor, or may be understood as an image captured before the photographing mode of the image sensor is switched.

[0232] S702: In response to a first operation of the user, display a preview interface, where the preview interface includes a first image.

[0233] In an embodiment of the present application, the preview interface may include the preview interface of the above-mentioned camera application, and may also include the interface displayed by the application in scenarios such as recording, video calls, and live video broadcasts, which is not limited in the embodiment of the present application.

[0234] S703: When the illumination is less than the first illumination threshold and the exposure ratio is less than or equal to the first gain threshold, the image sensor uses the first exposure mode to obtain a second image.

[0235] In the embodiment of the present application, illuminance can reflect the intensity of light, and illuminance can also be called brightness.

[0236] If the illuminance is less than the first illuminance threshold, it means that the illuminance is small and the light is dim. The first illuminance threshold can be customized by the electronic device, and the specific first illuminance threshold is not limited in the embodiment of the present application.

[0237] If the exposure ratio is less than or equal to the first gain threshold, it indicates a small dynamic range. The first gain threshold can be calculated based on the current camera mode or customized by the electronic device. The first gain threshold can be described with reference to step S604 in the embodiment corresponding to FIG. 6 above and will not be further described. The specific first gain threshold is not limited in this embodiment.

[0238] The first mode can be understood as a photography mode suitable for scenes with low light and a small dynamic range. For example, the first mode may include a DAG mode. The specific DAG mode can be referred to in the relevant description of the DAG mode above (1.2), and will not be repeated here.

[0239] The second image may be understood as an image captured after the photographing mode of the image sensor is switched.

[0240] S704: When the illumination is less than the first illumination threshold and the exposure ratio is greater than the first gain threshold, the image sensor uses a second exposure mode to obtain a second image.

[0241] In the embodiment of the present application, the exposure ratio is greater than the first gain threshold, indicating that the dynamic range is large.

[0242] The second mode can be understood as a photography mode suitable for use in medium and high-brightness scenes with a high dynamic range. For example, the second mode may include DXG mode. The specific description of DXG mode can refer to the relevant description of DXG mode above (1.3) and will not be repeated here.

[0243] S705 : When the illuminance is greater than or equal to the first illuminance threshold and less than the second illuminance threshold, and the exposure ratio is greater than the second gain threshold, the image sensor uses the third exposure mode to obtain a second image.

[0244] In the embodiment of the present application, if the illuminance is greater than or equal to the first illuminance threshold and less than the second illuminance threshold, it means that the illuminance is not particularly high. The second illuminance threshold can be customized by the electronic device, and the specific second illuminance threshold is not limited in the embodiment of the present application.

[0245] If the exposure ratio is greater than the second gain threshold, it indicates a large dynamic range, representing a high dynamic range scene. The second gain threshold can be calculated based on the current camera mode or customized by the electronic device. The second gain threshold can be described with reference to step S607 in the embodiment corresponding to FIG. 6 above and will not be further described. The specific second gain threshold is not limited in this embodiment.

[0246] The third mode can be understood as a photography mode suitable for use in high-dynamic scenes with medium or high brightness. For example, the third mode may include an SHDR mode. The specific SHDR mode can be referred to the relevant description of the SHDR mode above (1.4), and will not be repeated here.

[0247] S706: The preview interface includes the second image.

[0248] S707: In response to a second operation by the user, save the second image.

[0249] In the embodiments of the present application, the second operation may include an operation of triggering a capture control or an operation of triggering saving an image. For example, in the embodiment corresponding to FIG. 4 above, the second operation may be an operation of clicking a capture control in interface 401. The second operation may also include an operation of voice-activated instruction to capture or save an image, etc. The specific implementation method of the second operation is not limited in the embodiments of the present application.

[0250] Since different shooting modes produce different image output effects under different illuminations and / or different exposure ratios, in order to achieve good shooting effects in various shooting scenes, the embodiments of the present application can determine a more appropriate shooting mode by judging the range of illumination and / or the range of exposure ratio, thereby fully leveraging the advantages of each shooting mode and obtaining images with a high dynamic range and good quality in various scenes.

[0251] Optionally, based on the embodiment corresponding to Figure 7, the method may also include: when the illuminance is greater than or equal to the first illuminance threshold and less than the second illuminance threshold, and the exposure ratio is less than or equal to the second gain threshold, the image sensor adopts the second mode exposure to obtain the second image; when the illuminance is greater than or equal to the second illuminance threshold, the image sensor adopts the third mode exposure to obtain the second image.

[0252] In the embodiment of the present application, the illuminance is greater than or equal to the first illuminance threshold and less than the second illuminance threshold, and the exposure ratio is less than or equal to the second gain threshold, indicating that the current illuminance and dynamic range are not particularly large and are suitable for the second mode of image output.

[0253] If the illuminance is greater than or equal to the second illuminance threshold, the current illuminance is high and the light is bright, making the third mode suitable for image output. When the illuminance and dynamic range are both moderate, the image sensor uses the second exposure mode; when the illuminance is high and the light is bright, the image sensor uses the third exposure mode. This allows the advantages of the second and third modes to be fully utilized, thereby improving image quality.

[0254] Optionally, based on the embodiment corresponding to Figure 7, when the image sensor adopts the first mode for exposure, the exposure ratio is positively correlated with the configuration gain ratio; when the image sensor adopts the second mode for exposure, the exposure ratio is positively correlated with one or more of the following: the configuration gain ratio, the conversion gain ratio of the long frame and the short frame; when the image sensor adopts the third mode for exposure, the exposure ratio is positively correlated with one or more of the following: the configuration gain ratio, the conversion gain ratio of the long frame and the short frame, and the exposure time of the long frame and the short frame; wherein, the exposure amount of the long frame is greater than the exposure amount of the short frame.

[0255] In the embodiments of the present application, since the exposure ratio is positively correlated with the configured gain ratio in the first mode, the first mode can achieve long and short frames by configuring different configured gains. When shooting in dark scenes, since the configured gain value of the long frame is higher, the short frame can achieve a more appropriate dynamic range by reducing the configured gain, allowing the first mode to have a larger dynamic range. Therefore, the first mode is suitable for use in darker scenes.

[0256] Since in the second mode, the exposure ratio is positively correlated with the conversion gain ratio of the long frame and the short frame, based on the conversion gain ratio of HCG and LCG, the second mode can have a certain dynamic range expansion capability in higher brightness scenes. Therefore, the second mode is used for medium and high brightness scenes.

[0257] In the third mode, the exposure ratio is positively correlated with the exposure times of the long and short frames, and a larger dynamic range is achieved by configuring the exposure times of the two frames. Therefore, the third mode is suitable for brighter scenes.

[0258] In summary, different shooting modes have different factors that influence exposure ratio. Based on the characteristics of these factors, different shooting modes are suitable for different shooting scenarios. Therefore, different shooting modes can be selected in different scenarios to improve image quality.

[0259] Optionally, based on the embodiment corresponding to Figure 7, the method may further include: when the image sensor adopts the fourth mode exposure to obtain the first image, the first gain threshold is the configuration gain of the first image; when the image sensor adopts the first mode exposure to obtain the first image, the first gain threshold is the configuration gain of the long frame obtained by the image sensor adopting the first mode exposure.

[0260] In the embodiments of the present application, the fourth mode can be understood as a single-frame photography mode. The fourth mode is suitable for scenes with a relatively low dynamic range. For example, the fourth mode can include a binning mode. The specific description of the binning mode can be referred to in the above (1.5) Binning mode, and will not be repeated here.

[0261] When the image sensor uses the fourth exposure mode to obtain a first image, the first gain threshold is the configured gain of the first image. This corresponds to a scenario where the fourth mode switches to the first mode. Since the exposure ratio in the first mode is related to the configured gain, the first gain threshold can be the configured gain of the first image.

[0262] When the image sensor uses the first exposure mode to obtain the first image, the first gain threshold is the configured gain of the long frame obtained by the image sensor using the first exposure mode. This corresponds to the scenario of switching from the first mode to the second mode. Because the first mode does not include a conversion gain ratio, the first gain threshold can be the configured gain of the long frame obtained by the image sensor using the first exposure mode.

[0263] Taking into account the characteristics of different photographing modes, the first gain threshold can have different values. In this way, by calculating the first gain threshold according to the current photographing mode, the exposure ratio of the next image can be more reasonably predicted.

[0264] Optionally, based on the embodiment corresponding to Figure 7, the method may further include: when the image sensor adopts the fourth mode exposure to obtain the first image, the second gain threshold is positively correlated with the configuration gain of the first image and the conversion gain ratio of the long frame and the short frame obtained by the image sensor adopting the second mode exposure; when the image sensor adopts the second mode exposure to obtain the first image, the second gain threshold is positively correlated with the configuration gain of the long frame obtained by the image sensor adopting the second mode exposure and the conversion gain ratio of the long frame and the short frame obtained by the image sensor adopting the second mode exposure.

[0265] In the embodiment of the present application, when the image sensor uses the fourth mode to expose and obtain the first image, the scenario for determining the second gain threshold may include a scenario in which the fourth mode switches to the second mode or the third mode. Since only the configured gain of the image can be obtained in the fourth mode, the conversion gain ratio can use the conversion gain ratio corresponding to the second mode.

[0266] When the image sensor uses the second exposure mode to obtain the first image, the scenario for determining the second gain threshold may include a scenario of switching from the second mode to the third mode. In this case, the configuration gain and the conversion gain ratio may both be obtained based on the second mode.

[0267] Taking into account the characteristics of different shooting modes, the conversion gain ratio of the second gain threshold can be obtained based on the second mode. In this way, the second gain threshold can be calculated according to the current shooting mode, thereby more reasonably predicting the exposure ratio of the next image.

[0268] Optionally, based on the embodiment corresponding to Figure 7, the first mode includes a dual analog gain DAG mode, the second mode includes a DXG mode, the third mode includes a row interleaved exposure SHDR mode, and the fourth mode includes a Binning mode, wherein the DXG mode is a mode that combines the dual conversion gain DCG mode and the DAG mode.

[0269] In the embodiment of the present application, the specific DAG mode can refer to the relevant description in the above (1.2) DAG mode; the DXG mode can refer to the relevant description in the above (1.3) DXG mode; the SHDR mode can refer to the relevant description in the above (1.4) SHDR mode; the Binning mode can refer to the relevant description in the above (1.5) Binning mode, and no further details are given.

[0270] In different shooting environments, the image sensor can adopt different shooting modes, so that the advantages of each shooting mode can be fully utilized to obtain better shooting effects, thereby improving user experience.

[0271] Optionally, based on the embodiment corresponding to Figure 7, the image sensor adopts the first mode exposure to obtain the second image, which may include: transmitting a first identifier to the image sensor, the first identifier is used to indicate that the image sensor adopts the first mode exposure; the image sensor adopts the second mode exposure to obtain the second image, including: transmitting a second identifier to the image sensor, the second identifier is used to indicate that the image sensor adopts the second mode exposure; the image sensor adopts the third mode exposure to obtain the second image, including: transmitting a third identifier to the image sensor, the third identifier is used to indicate that the image sensor adopts the third mode exposure.

[0272] In the embodiment of the present application, the values ​​of the first identifier, the second identifier and the third identifier are all different. The data types of the first identifier, the second identifier and the third identifier may include integer, string and other types.

[0273] For example, as described in the relevant description of the flag of the photographing mode in step S514 of the embodiment corresponding to FIG. 5 , the first flag may take a value of 1, the second flag may take a value of 2, and the third flag may take a value of 3.

[0274] The first identifier, the second identifier, and the third identifier can be customized by the electronic device. The specific values ​​of the first identifier, the second identifier, and the third identifier are not limited in the embodiments of the present application.

[0275] Using different identifiers to indicate the shooting mode adopted by the image sensor can achieve the purpose of message transmission more clearly and concisely, help maintain business logic, and improve code execution efficiency.

[0276] Optionally, based on the embodiment corresponding to FIG. 7 , exposing the image sensor to obtain the first image may specifically include: when the exposure ratio is less than the exposure threshold, the image sensor adopts the fourth exposure mode to obtain the first image.

[0277] In the embodiment of the present application, the exposure threshold can be customized by the decision module, and the specific value of the exposure threshold is not limited in the embodiment of the present application.

[0278] The electronic device can determine whether to use single-frame exposure or double-frame exposure based on the exposure ratio.

[0279] Within the image sensor's single-frame dynamic range, the image sensor can use single-frame exposure mode. Compared to dual-frame exposure mode, single-frame exposure mode occupies less memory space, which can reduce system power consumption and improve performance.

[0280] Optionally, based on the embodiment corresponding to FIG7 , the method may further include: acquiring illumination and exposure ratio; wherein the exposure ratio is the exposure ratio of the first image, and the illumination is the illumination of the current environment.

[0281] In an embodiment of the present application, the shooting mode of the second image is determined based on the illumination of the current environment and the exposure ratio of the first image, so that a more reasonable output method that matches the environment can be used to obtain a second image of better quality.

[0282] Optionally, based on the embodiment corresponding to Figure 7, the electronic device includes a first module, an automatic exposure AE module and an ambient light sensor. Before obtaining the illuminance and exposure ratio, it may also include: the AE module calculates the exposure ratio based on the first image; obtaining the illuminance and exposure ratio may include: the first module obtains the illuminance from the ambient light sensor, and the first module obtains the exposure ratio from the AE module.

[0283] In an embodiment of the present application, the first module may include the decision module in the above embodiment, or may include any module in the electronic device that can obtain illumination and exposure ratio and calculate the shooting mode, and the embodiment of the present application is not limited.

[0284] The exposure ratio calculated by the AE module based on the first image can be obtained by referring to the relevant description of step S510 in the embodiment corresponding to FIG5 , which will not be repeated here.

[0285] It is understandable that the first module can obtain illuminance from the ambient light sensor, or can receive illuminance information transmitted by the AE module. The specific way in which the first module obtains illuminance is not limited in the embodiment of the present application.

[0286] The first module calculates the shooting mode of the next image based on the exposure ratio and the illumination of the current environment. It can timely consider the impact of current environmental factors on the image output effect of the image sensor, improve image quality, and thus enhance user experience.

[0287] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0288] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily appreciate that, in combination with the method steps of each example described in the embodiment disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0289] The embodiment of the present application can divide the functional modules of the device implementing the method according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. In actual implementation, there may be other division methods.

[0290] 8 is a schematic diagram of the structure of a chip provided by an embodiment of the present application. The chip 800 includes one or more (including two) processors 801, a communication line 802, a communication interface 803 and a memory 804.

[0291] In some implementations, the memory 804 stores the following elements: executable modules or data structures, or a subset thereof, or an extended set thereof.

[0292] The method described in the above embodiment of the present application can be applied to the processor 801, or implemented by the processor 801. The processor 801 may be an integrated circuit chip with signal processing capabilities. During the implementation process, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor 801 or an instruction in the form of software. The above-mentioned processor 801 can be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor DSP, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate, transistor logic device or discrete hardware component, and the processor 801 can implement or execute the methods, steps and logic block diagrams related to each processing disclosed in the embodiment of the present application.

[0293] The steps of the method disclosed in the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or can be implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as a random access memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable read-only memory (EEPROM). The storage medium is located in the memory 804, and the processor 801 reads the information in the memory 804 and performs the steps of the above method in conjunction with its hardware.

[0294] The processor 801 , the memory 804 , and the communication interface 803 may communicate with each other via a communication line 802 .

[0295] In the above embodiment, the instructions stored in the memory for execution by the processor may be implemented in the form of a computer program product, wherein the computer program product may be pre-written in the memory or downloaded and installed in the memory in the form of software.

[0296] The embodiment of the present application also provides a computer program product including one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a site, computer, server or data center of a website to a site, computer, server or data center of another website via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. For example, the available medium can include magnetic media (e.g., floppy disk, hard disk or tape), optical media (e.g., digital versatile disc (DVD)), or semiconductor media (e.g., solid state disk (SSD)), etc.

[0297] The present application also provides a computer-readable storage medium. The methods described in the above embodiments can be implemented in whole or in part via software, hardware, firmware, or any combination thereof. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one location to another. The storage medium can be any target medium that can be accessed by a computer.

[0298] As one possible design, computer-readable media may include compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM or other optical disc storage; computer-readable media may include magnetic disk storage or other magnetic disk storage devices. Moreover, any connecting line may also be appropriately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically using lasers.

[0299] The present application embodiment is described with reference to the flow chart and / or block diagram according to the method, device (system) and computer program product of the embodiment of the present application.It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processing unit of general-purpose computer, special-purpose computer, embedded processing machine or other programmable data processing equipment to produce a machine, so that the instruction executed by the processing unit of computer or other programmable data processing equipment produces the device for realizing the function specified in one flow chart flow chart or multiple flow charts and / or one block or multiple blocks of block diagram.

Claims

1. An image processing method, characterized in that, The method is applied to an electronic device, which includes an image sensor. The method comprises: In response to a first operation of a user, the image sensor performs exposure to obtain a first image; In response to the first operation of the user, a preview interface is displayed, and the preview interface includes the first image; When the illuminance is less than a first illuminance threshold and the exposure ratio is less than or equal to a first gain threshold, the image sensor performs exposure in a first mode to obtain a second image; When the illuminance is less than the first illuminance threshold and the exposure ratio is greater than the first gain threshold, the image sensor performs exposure in a second mode to obtain the second image; When the illuminance is greater than or equal to the first illuminance threshold and less than a second illuminance threshold, and the exposure ratio is greater than a second gain threshold, the image sensor performs exposure in a third mode to obtain the second image; The preview interface includes the second image; In response to a second operation of the user, the second image is saved; Wherein, the first mode, the second mode, and the third mode are different from each other.

2. The method according to claim 1, wherein The method further comprises: When the illuminance is greater than or equal to the first illuminance threshold and less than the second illuminance threshold, and the exposure ratio is less than or equal to the second gain threshold, the image sensor performs exposure in the second mode to obtain the second image; When the illuminance is greater than or equal to the second illuminance threshold, the image sensor performs exposure in the third mode to obtain the second image.

3. The method according to claim 1 or 2, characterized in that, When the image sensor performs exposure in the first mode, the exposure ratio is positively correlated with the configured gain ratio; When the image sensor performs exposure in the second mode, the exposure ratio is positively correlated with one or more of the following: the configured gain ratio, the conversion gain ratio between the long frame and the short frame; When the image sensor performs exposure in the third mode, the exposure ratio is positively correlated with one or more of the following: the configured gain ratio, the conversion gain ratio between the long frame and the short frame, the exposure time of the long frame and the short frame; Wherein, the exposure amount of the long frame is greater than the exposure amount of the short frame.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: When the image sensor performs exposure in a fourth mode to obtain the first image, the first gain threshold is the configured gain of the first image; When the image sensor performs exposure in the first mode to obtain the first image, the first gain threshold is the configured gain of the long frame obtained by the image sensor performing exposure in the first mode.

5. The method according to any one of claims 1-4, characterized in that, The method further comprises: When the image sensor performs exposure in a fourth mode to obtain the first image, the second gain threshold is positively correlated with the configured gain of the first image and the conversion gain ratio between the long frame and the short frame obtained by the image sensor performing exposure in the second mode; When the image sensor performs exposure in the second mode to obtain the first image, the second gain threshold is positively correlated with the configured gain of the long frame obtained by the image sensor performing exposure in the second mode and the conversion gain ratio between the long frame and the short frame obtained by the image sensor performing exposure in the second mode.

6. The method according to any one of claims 1-5, characterized in that, The first mode includes a dual analog gain DAG mode, the second mode includes a DXG mode, the third mode includes a line interleaved exposure SHDR mode, and the fourth mode includes a Binning mode. Among them, the DXG mode is a mode combining a dual conversion gain DCG mode and the DAG mode.

7. The method according to any one of claims 1-6, characterized in that, The image sensor obtains a second image by exposing in the first mode, including: Transmitting a first identifier to the image sensor, where the first identifier is used to indicate that the image sensor exposes in the first mode; The image sensor obtains the second image by exposing in the second mode, including: Transmitting a second identifier to the image sensor, where the second identifier is used to indicate that the image sensor exposes in the second mode; The image sensor obtains the second image by exposing in the third mode, including: Transmitting a third identifier to the image sensor, where the third identifier is used to indicate that the image sensor exposes in the third mode.

8. The method according to any one of claims 1 to 7, characterized in that, The image sensor exposes to obtain a first image, specifically including: When the exposure ratio is less than the exposure threshold, the image sensor exposes in the fourth mode to obtain the first image.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: Obtaining the illuminance and the exposure ratio; Among them, the exposure ratio is the exposure ratio of the first image, and the illuminance is the illuminance of the current environment.

10. The method according to claim 9, wherein The electronic device includes a first module, an automatic exposure AE module, and an ambient light sensor. Before obtaining the illuminance and the exposure ratio, it further includes: The AE module calculates the exposure ratio based on the first image; Obtaining the illuminance and the exposure ratio includes: The first module obtains the illuminance from the ambient light sensor, and the first module obtains the exposure ratio from the AE module.

11. An electronic device, characterized in that, The electronic device includes: one or more processors and a memory; The memory is coupled to the one or more processors. The memory is used to store computer program code, and the computer program code includes computer instructions. The one or more processors call the computer instructions to cause the electronic device to execute the method according to any one of claims 1 to 10.

12. A chip system, characterized in that, The chip system is applied to an electronic device. The chip system includes one or more processors, and the one or more processors are used to call computer instructions to cause the electronic device to execute the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions. When the computer instructions run on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 10.

14. A computer program product, characterized in that, The computer program product includes computer program code. When the computer program code runs on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 10.

Images