Method for switching between high dynamic range modes and electronic device storage medium

Abstract

A method for switching between high dynamic range (HDR) modes and an electronic device storage medium, relating to the technical field of terminals. The method is applied to an electronic device supporting an overflow integration capacitor (LOFIC) HDR mode and a dual analog gain (DAG) HDR mode, and comprises: starting a camera operating program; acquiring the illuminance of the environment in which the electronic device is located; and starting the LOFIC HDR mode. According to the scheme, when the illuminance is reduced, the electronic device does not remain in the LOFIC HDR mode, but switches the HDR mode to the DAG HDR mode, so as to utilize a characteristic of the DAG HDR mode wherein same uses the same conversion gain for both long frames and short frames, such that a signal-to-noise ratio drop in a transition area between a bright portion and a dark portion can be mitigated, thereby improving the photographing experience of a user.

Description

A high dynamic range mode switching method and electronic device storage medium Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method for switching a high dynamic range mode and a storage medium for an electronic device. Background Art

[0002] With the development of the terminal industry, many electronic devices support high dynamic range (HDR) photography technology. HDR photography technology can capture and display the huge dynamic range of the real world despite the limited dynamic range available in typical imaging sensors and display devices.

[0003] The lateral overflow integration capacitor (LOFIC) HDR mode expands the dynamic range of complementary metal oxide semiconductor (CMOS) technology by adding lateral overflow capacitors. However, when electronic devices use the LOFIC HDR mode, the signal-to-noise ratio (SNR) may drop from high to low in the transition areas between bright and dark parts of the image during preview or video recording, resulting in signal-to-noise stratification, which degrades the user's shooting experience.

[0004] Summary of the Invention

[0005] In view of this, the present application provides a method for switching a high dynamic range mode and an electronic device storage medium, which can alleviate the signal-to-noise ratio drop in the transition area between bright and dark areas, thereby improving the user's shooting experience.

[0006] In the first aspect, the present application provides a method for switching a high dynamic range mode, which is applied to an electronic device that supports the LOFIC HDR mode and the dual analog gain (DAG) HDR mode. The method includes: starting a camera running program, obtaining the light intensity of the electronic device environment, enabling the LOFIC HDR mode, and when the light intensity drops to less than the first light intensity, switching the HDR mode to the DAG HDR mode.

[0007] When the camera sensor works in LOFIC HDR mode, when the light intensity decreases in the existing solution, the LOFIC HDR mode is continued to be used. At this time, in order to ensure the picture quality under low brightness, the LOFIC HDR mode will gradually increase the exposure gain of the long frame, so that the signal-to-noise ratio drop in the transition area between the bright and dark parts gradually becomes obvious. The solution of the present application does not always remain in LOFIC HDR mode, but switches the HDR mode to DAG HDR mode. This solution utilizes the characteristic that the long frame and the short frame of the DAG HDR mode use the same conversion gain. When the light intensity drops to less than the first light intensity, it indicates that the light intensity is low at this time. At this time, the signal-to-noise ratio drop of the LOFIC HDR mode is more obvious. After adopting the DAG HDR mode, there is no obvious signal-to-noise ratio drop between the long frame and the short frame of the DAG HDR mode. As a result, the signal-to-noise ratio drop in the transition area between the bright and dark parts of the image after the long frame and the short frame are fused is fully alleviated, thereby improving the user's shooting experience.

[0008] In one possible implementation, when the light intensity is greater than or equal to the first light intensity, the method further includes: when the light intensity decreases, controlling the exposure gain of the LOFIC HDR mode to increase as the light intensity decreases, and controlling the exposure ratio of the LOFIC HDR mode to decrease as the light intensity decreases.

[0009] In the existing solution, when the light intensity decreases, in order to ensure the picture quality under low brightness, the exposure gain of the long frame of the LOFIC HDR mode is gradually increased, so that the exposure ratio of the LOFIC HDR mode increases. The solution of the embodiment of the present application increases the exposure gain of the LOFIC HDR mode to ensure the picture quality while controlling the exposure ratio of the LOFIC HDR mode to decrease as the light intensity decreases, so that the exposure ratio of the LOFIC HDR mode will not be too high before switching to the DAG HDR mode. The advantages of doing so are: on the one hand, it avoids the obvious signal-to-noise ratio drop in the picture due to the excessive exposure ratio before switching to the DAG HDR mode; on the other hand, since the pixel circuit of the DAG HDR mode generally does not include a lateral overflow collection capacitor, this causes the maximum exposure ratio of the DAG HDR mode to be smaller than the maximum exposure ratio of the LOFIC HDR mode. By reducing the exposure ratio of the LOFIC HDR mode, the exposure ratio mutation during the HDR mode switching can be reduced or eliminated, making the HDR mode switching more natural.

[0010] In one possible implementation, the exposure gain includes an analog gain and a digital gain, and the exposure gain of the LOFIC HDR mode is controlled to increase as the light intensity decreases, and the exposure ratio of the LOFIC HDR mode is controlled to decrease as the light intensity decreases, specifically including:

[0011] The digital gain of the LOFIC HDR mode is controlled to remain unchanged; the rate of increase of the analog gain of the short frame of the LOFIC HDR mode as the light intensity decreases is greater than the rate of increase of the analog gain of the long frame of the LOFIC HDR mode as the light intensity decreases.

[0012] The exposure ratio is the ratio of the exposure gain of the long frame to the exposure gain of the short frame. When the digital gain remains unchanged, when the analog gain increase rate of the short frame is greater than the analog gain increase rate of the long frame, the exposure ratio will gradually decrease. This implementation method increases the analog gain of the LOFIC HDR mode to ensure picture quality while not exacerbating the signal-to-noise ratio drop. In one possible implementation method, when the light intensity is less than the first light intensity, the HDR mode is switched to the DAG HDR mode, specifically including:

[0013] Determine the analog gain of the short frame of the LOFIC HDR mode and the analog gain of the short frame of the LOFIC HDR mode according to the light intensity; when the analog gain of the long frame is greater than the first preset gain, switch the HDR mode to the DAG HDR mode, and the first preset gain is the analog gain of the short frame of the LOFIC HDR mode when the light intensity is the first light intensity; control the exposure ratio before and after the HDR mode switching to remain unchanged.

[0014] This implementation method controls the exposure ratio before and after switching to HDR mode to remain unchanged, making the switching of HDR mode more natural and the picture consistency higher.

[0015] In one possible implementation, the exposure gain includes an analog gain and a digital gain, and the exposure gain of the LOFIC HDR mode is controlled to increase as the light intensity decreases, and the exposure ratio of the LOFIC HDR mode is controlled to decrease as the light intensity decreases, specifically including:

[0016] The analog gain of the long frame in LOFIC HDR mode remains unchanged;

[0017] The analog gain of the short frames controlling the LOFIC HDR mode increases as the light intensity decreases;

[0018] Controls the digital gain of the LOFIC HDR mode, which increases as light intensity decreases.

[0019] In this implementation, the digital gain is increased to ensure the picture quality in low-light environments. In addition, since the analog gain of the long frame remains unchanged in this solution, the signal-to-noise ratio drop in the transition area between the bright and dark parts of the picture in LOFIC HDR mode is significantly alleviated.

[0020] In a possible implementation, when the illumination intensity is less than the first illumination intensity, the HDR mode is switched to the DAG HDR mode, specifically including:

[0021] The analog gain of the short frame of the LOFIC HDR mode and the digital gain of the LOFIC HDR mode are determined according to the light intensity; when the digital gain of the LOFIC HDR mode is greater than the second preset gain, the HDR mode is switched to the DAG HDR mode, and the second preset gain is the digital gain of the LOFIC HDR mode when the light intensity is the first light intensity; the exposure ratio before and after the HDR mode switching is controlled to remain unchanged.

[0022] In this implementation, as illumination decreases, the digital gain increases. A high digital gain can cause a decrease in the overall image signal-to-noise ratio. When the digital gain exceeds a second preset gain corresponding to the first illumination intensity, HDR mode is switched to ensure a high overall image signal-to-noise ratio. Furthermore, by maintaining the same exposure ratio before and after switching to HDR mode, HDR mode switching becomes more natural and delivers a high degree of image consistency.

[0023] In one possible implementation, the exposure gain includes an analog gain and a digital gain, and the exposure gain of the LOFIC HDR mode is controlled to increase as the light intensity decreases, and the exposure ratio of the LOFIC HDR mode is controlled to decrease as the light intensity decreases, specifically including:

[0024] The digital gain of the LOFIC HDR mode is controlled to remain unchanged, and the rate of increase of the analog gain of the short frame of the LOFIC HDR mode when it increases with decreasing light intensity is greater than the rate of increase of the analog gain of the long frame of the LOFIC HDR mode when it increases with decreasing light intensity;

[0025] When the analog gain of the long frame of the LOFIC HDR mode is greater than the third preset gain, the analog gain of the long frame of the LOFIC HDR mode is controlled to remain unchanged, the analog gain of the short frame of the LOFIC HDR mode increases with the decrease of light intensity, and the digital gain of the LOFIC HDR mode increases with the decrease of light intensity.

[0026] This solution performs two exposure gain adjustment steps as light intensity gradually decreases. In the first exposure gain adjustment step, the digital gain of the LOFIC HDR mode is maintained unchanged, and the analog gain growth rate of the long frame is set to be slower than that of the short frame. In the second exposure gain adjustment step, the digital gain of the LOFIC HDR mode and the analog gain of the short frame are increased, while the analog gain of the long frame is maintained unchanged.

[0027] Through the above two adjustment processes, before switching to HDR mode, the SNR dip in the transition area of ​​the picture can be alleviated, and a large drop in the signal-to-noise ratio in the dark part of the picture can be avoided. After switching to HDR mode, the SNR dip problem is alleviated, and the sudden change in exposure ratio is avoided, making the switching of HDR mode natural. In one possible implementation, when the light intensity is less than the first light intensity, the HDR mode is switched to DAG HDR mode, specifically including:

[0028] Determine the analog gain of the short frame of the LOFIC HDR mode and the digital gain of the LOFIC HDR mode according to the light intensity; when the digital gain of the LOFIC HDR mode is greater than the fourth preset gain, switch the HDR mode to the DAG HDR mode, and the fourth preset gain is the digital gain of the LOFIC HDR mode when the light intensity is the first light intensity; control the exposure ratio before and after the HDR mode is switched to remain unchanged.

[0029] In this implementation, a large digital gain can cause a decrease in the overall image signal-to-noise ratio (SNR). Therefore, when the digital gain exceeds the fourth preset gain corresponding to the first light intensity, HDR mode is switched to ensure a high SNR for the entire image. Furthermore, by maintaining the same exposure ratio before and after the HDR mode switch, HDR mode switching becomes more natural and achieves higher image consistency.

[0030] In one possible implementation, controlling the exposure ratio to remain unchanged before and after switching the HDR mode includes:

[0031] When the maximum exposure ratio corresponding to the current exposure time is less than the exposure ratio before the HDR mode is switched, the exposure time is reduced and the analog gain of the long frame of the DAG HDR mode is increased to control the exposure ratio before and after the HDR mode is switched to remain unchanged. The exposure time is negatively correlated with the maximum exposure ratio corresponding to the exposure time.

[0032] If the maximum exposure ratio under the current exposure time is lower than the exposure ratio before switching to HDR mode, you can adjust the exposure ratio by reducing the exposure time and increasing the analog gain of the long frame. This can adjust the exposure ratio of the DAG HDR mode while reducing the image brightness difference before and after the mode switch.

[0033] In one possible implementation, the method further includes:

[0034] Control the screen brightness to be consistent before and after switching to HDR mode.

[0035] By controlling the brightness of the picture before and after switching to HDR mode, the switching of HDR mode is made more natural and the picture consistency is further improved.

[0036] In one possible implementation, controlling the brightness of the image before and after switching to the HDR mode to be consistent includes:

[0037] Control the product of the exposure time before HDR mode switching and the exposure gain of the long frame of the LOFIC HDR mode to be equal to the product of the exposure time after HDR mode switching and the long frame exposure gain of the DAG HDR mode.

[0038] The product of the long frame's exposure gain and exposure time is the exposure value corresponding to the long frame. By controlling the long frame's exposure value to remain unchanged before and after mode switching, the brightness consistency of the image before and after mode switching is improved.

[0039] In the second aspect, the present application also provides an electronic device that supports LOFIC HDR mode and DAG HDR mode. The electronic device includes a processor and a memory, the memory is used to store a program, and the program is executed by the processor when it is run to execute the high dynamic range mode switching method provided by the first aspect and any one of the implementations of the first aspect. The electronic device can be a mobile phone, a laptop computer, a wearable electronic device (such as a smart watch, smart glasses), a tablet computer, an augmented reality (AR device), a virtual reality (VR) device, etc.

[0040] In a third aspect, the present application further provides a storage medium storing a computer program, which, when executed by an electronic device, implements the method for switching the high dynamic range mode provided by the first aspect and any one of the implementation methods of the first aspect.

[0041] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in this application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution or beneficial effect is included in at least one embodiment. Therefore, the description of a technical feature, technical solution or beneficial effect in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in the present embodiment can also be combined in any appropriate manner. Those skilled in the art will understand that the embodiment can be implemented without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can also be identified in specific embodiments that do not embody all embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] FIG1 is a schematic diagram of a pixel circuit provided in an embodiment of the present application;

[0043] FIG2 is a schematic diagram of a curve showing image signal-to-noise ratio obtained by capacitors with different conversion gains as a function of illumination provided by an embodiment of the present application;

[0044] FIG3 is a schematic diagram of a scenario provided in an embodiment of the present application;

[0045] FIG4 is a flow chart of a method for switching to a high dynamic range mode provided in an embodiment of the present application;

[0046] FIG5 is a flowchart of another method for switching to a high dynamic range mode provided in an embodiment of the present application;

[0047] FIG6 is a flowchart of another method for switching to a high dynamic range mode provided in an embodiment of the present application;

[0048] FIG7 is a schematic diagram of an electronic device provided in an embodiment of the present application;

[0049] FIG8 is an architecture diagram of a software system of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0051] To help those skilled in the art better understand the present invention, the following first describes the inventive concept of the present invention. The electronic device in the embodiments of the present invention may be a mobile phone, tablet computer, personal digital assistant (PDA), or other electronic device. The embodiments of the present invention do not impose any particular limitations on the specific form of the electronic device.

[0052] Dynamic range (DR), also known as exposure range, refers to the range of light intensities captured by a camera. For image signals, it is the ratio of the maximum to minimum values ​​of a variable light signal. Currently, many electronic devices support HDR photography technology. HDR photography can capture and display the vast dynamic range of the real world, despite the limited dynamic range available in typical imaging sensors and displays.

[0053] Currently, HDR modes of electronic devices can include a stagger HDR (SHDR) mode, a LOFIC HDR mode, a dual conversion gain (DCG) mode, and a dual analog gain (DAG) mode.

[0054] The following describes how to implement LOFIC HDR mode in electronic devices.

[0055] Refer to FIG1 , which is a schematic diagram of a pixel circuit provided in this application.

[0056] The pixel circuit used in LOFIC HDR mode adds a large-capacity well capacitor C s , C s The capacity is greater than the ordinary well capacity C FD In practical applications, C s The capacity is generally C FD dozens of times the capacity.

[0057] Before exposure begins, switch TG, switch R, and switch S are turned on to reset C s and C FD .

[0058] At this time, the reset noise N2 remains at C s and C FD .

[0059] During the integration period, the signal charge is integrated at the photodiode (PD) until it reaches saturation, and then the overflow charge is transferred to C through the switch TG. s and C FD , that is, the overflow charge is in C s and CFD The overflow charge from the PD is integrated so that it can be used for the signal.

[0060] After integration, close switch S to turn C s and C FD Isolate and allocate each C FD The charge transferred from PD to FD allows the voltage of FD to be read out as signal N1. Then, by opening switch TG, the signal charge is transferred from PD to FD, and the signal charge transferred from PD to FD is read out as signal S1+N1.

[0061] By opening the switch S, between PD, C s and C FD All signal charges integrated at C s and C FD The mixed signals are then read out as signal S2+N2.

[0062] C s and C FD Reset is performed by opening switches TG and R. Then, switches R and T are closed and the next reset noise N2' is read.

[0063] In summary, each pixel circuit reads out N1, S1+N1, S2+N2 and N2', and can then obtain a high-sensitivity signal S1 and a high-well capacity signal S2. Both S1 and S2 signals have a linear response to the incident light. From the working principle of the above pixel circuit, it can be seen that C FD The output signal-to-noise ratio of the response is higher in low-light scenes, but it is more prone to saturation, corresponding to high voltage conversion gain (HCG). s The characteristic is that the output signal-to-noise ratio is relatively low in low-light scenes, but it is not easy to reach saturation in brighter scenes, and can record more highlight information, corresponding to low voltage conversion gain (LCG). For the sake of convenience, this voltage conversion gain is represented by LOFIC in the following description.

[0064] The pixel circuit of LOFIC HDR mode increases C s Finally, the CMOS has two capacitors with different conversion gains. The two capacitors can realize voltage conversion with different amplification factors. Combining different voltage conversion gains, two signal outputs with different brightness are obtained. The fusion of the two images can expand the dynamic range of the final image, increasing the current dynamic range of CMOS from 60dB to nearly 90dB.

[0065] The information table of LOFIC HDR mode can be found in Table 1 below.

[0066] Table 1: LOFIC HDR mode information table

[0067] Different analog gains AG1 and AG2 are provided by a programmable gain amplifier (PGA).

[0068] See FIG. 2 , which is a schematic diagram of curves showing changes in image signal-to-noise ratio with illumination obtained by capacitors with different conversion gains provided in an embodiment of the present application.

[0069] Curve 1 and Curve 2 are the relationship curves between the signal-to-noise ratio and the light intensity when reading long frames.

[0070] Curve 3 is a curve showing the relationship between the signal-to-noise ratio and the light intensity when reading short frames.

[0071] The high voltage conversion gains corresponding to curves 1 and 2 in the figure are different. The high voltage conversion gain corresponding to curve 2 is 16 times that of curve 1.

[0072] When the light intensity is high, the required high voltage conversion gain is lower, and the gain multiple of curve 1 is used at this time.

[0073] Curve 1 and curve 3 have an intersection A. After starting the LOFIC HDR mode, the signal-to-noise ratio value on the left side of the intersection A is determined by curve 1, and the signal-to-noise ratio value on the right side of the intersection A is determined by curve 3.

[0074] Intersection A represents the transition zone between dark and bright areas. In one possible implementation, short frames are used at point A. Since the signal-to-noise ratio of short frames is significantly lower than that of long frames, signal-to-noise ratio stratification occurs in the transition zone. Specifically, as shown in Figure 2, a drop in signal-to-noise ratio from high to low occurs near or near the transition zone (also known as the fusion location). This phenomenon is known as SNR dip. The extent of the SNR dip depends on the intersection of the two curves. When the SNR dip exceeds a certain value, the fused image exhibits significant noise inconsistency.

[0075] When the ambient light intensity decreases, a higher high-voltage conversion gain is required, and the gain multiple of Curve 2 is used. At this time, Curves 2 and 3 have an intersection point B. After starting the LOFIC HDR mode, the signal-to-noise ratio value on the left side of intersection point B is determined by Curve 2, and on the right side of intersection point B, the signal-to-noise ratio value is determined by Curve 3. As can be seen from Figure 2, when the voltage conversion gain is increased, the saturation point of the HCG curve decreases, causing the intersection point B of Curves 2 and 3 to shift left relative to intersection point A, resulting in a significant increase in SNR drop. The noise inconsistency problem in the transition area is more obvious. When previewing and shooting videos on electronic devices, using this mode in dark scenes will result in a significant SNR dip problem, which is explained below in conjunction with specific scenarios.

[0076] See Figure 3, which is a schematic diagram of the scenario provided in this application.

[0077] When the user shoots at night, there are bright parts 10 and dark parts 20 in the preview screen or the shooting screen. The bright part 10 can be, for example, a luminous street lamp, and the dark part 20 can be a non-luminous object, such as a tree. As described above, using the current LOFIC HDR mode, there will be an obvious SNR dip problem in the transition area between the bright part 10 and the dark part 20, which is visually manifested as an increase in noise in the transition area. In addition, when the ambient light is dark, the noise will be more obvious. For example, the noise in the middle of the night will be more obvious than the noise in the evening, thus reducing the user's shooting experience.

[0078] Based on the above analysis process, it is currently necessary to alleviate the SNR dip problem caused by the LOFIC HDR mode under conditions of low light intensity. The present application provides a method for switching a high dynamic range mode and an electronic device storage medium, which can gradually reduce the exposure ratio of the LOFIC HDR mode when the light intensity decreases, until the LOFIC HDR mode switches to the DAG HDR mode. This solution utilizes the characteristics of the dual analog gain (DAG) HDR mode, which significantly alleviates the SNR dip problem under conditions of low light intensity. Moreover, since the exposure ratio of the LOFIC HDR mode has been gradually reduced before switching, the sudden change of the exposure ratio is alleviated during the switching process, that is, the picture consistency before and after the HDR mode switching is high, which improves the user's shooting experience.

[0079] The following first introduces the DAG HDR mode. The information table of the DAG HDR mode can be seen in Table 2 below.

[0080] Table 2: LOFIC HDR mode information table

[0081] The DAG HDR mode performs one exposure and reads out twice. Both the long and short frames are obtained through the same high conversion gain (HCG), and the long and short frames use different analog gains (analog gain). Among them, one is processed with high analog gain (HAG) and the other is processed with low analog gain (LAG). The camera sensor inputs the two frames of images processed by HAG and LAG into the image processing module of the chip platform at the same time, and generates one frame of image after algorithm processing. The long and short frames generated by the DAG HDR mode do not have obvious SNR dip problems because they use the same conversion gain. The solution of the present application switches the working state of the sensor from LOFIC HDR mode to DAG mode through mode switching to alleviate the SNR dip problem.

[0082] In order to alleviate the SNR dip problem as much as possible, the following aspects need to be met in this embodiment of the application:

[0083] 1. In LOFIC HDR mode, as light intensity decreases, the analog gain of the long frame cannot be too large, otherwise it will cause a significant SNR dip in the transition area. Therefore, the exposure ratio must be properly controlled.

[0084] 2. Since the pixel circuit of DAG HDR mode generally does not include C s , which means that the maximum exposure ratio of DAG HDR mode may be smaller than that of LOFIC HDR mode. When switching from LOFIC HDR mode to DAG HDR mode, the analog gain of the long frame of DAG HDR mode needs to be large enough to avoid a sudden change in the exposure ratio of DAG HDR mode compared to that of LOFIC HDR mode. The minimum analog gain of the short frame of DAG HDR mode is 1, so the maximum exposure ratio depends on the analog gain of the long frame of DAG HDR mode.

[0085] 3. Image brightness does not change suddenly before and after mode switching.

[0086] The technical solutions provided by the embodiments of the present application are described in detail below.

[0087] See FIG4 , which is a flow chart of a method for switching to a high dynamic range mode according to an embodiment of the present application.

[0088] The method comprises the following steps:

[0089] S40: The automatic exposure control module adjusts the exposure ratio of the LOFIC HDR mode according to the light intensity so that the exposure ratio is positively correlated with the light intensity.

[0090] The Automatic Exposure Control (AEC) module determines the dynamic range of the current scene. It performs light intensity measurement, scene analysis, and exposure compensation, each of which is described below.

[0091] Dynamic range, also known as exposure range, refers to the range of light intensities captured by the camera.

[0092] The process of light intensity measurement is to use the exposure information of the image to obtain the lighting information of the current environment.

[0093] Scene analysis is the process of understanding specific lighting conditions, such as backlighting or strong frontal light. Analyzing this information improves sensor usability and significantly enhances image quality, making it a critical technology for automatic exposure.

[0094] After completing light intensity measurement and scene analysis, the AEC module can adjust the corresponding parameters to make the exposure adjustment effective, mainly by setting the exposure time and exposure gain.

[0095] At the same exposure time, the exposure ratio is the ratio of the long frame exposure gain to the short frame exposure gain. Exposure gain includes analog gain and the digital gain of the digital image signal after the image signal processor (ISP) converts the electrical signal into a digital image signal. When the digital gain remains unchanged, the ratio of the long frame analog gain to the short frame analog gain equals the exposure ratio. When the HDR mode is enabled on an electronic device and the ambient light intensity decreases, the sensitivity of the photosensitive element to light is increased by increasing the exposure gain.

[0096] In an embodiment of the present application, when the AEC module adjusts the exposure ratio of the LOFIC HDR mode according to the light intensity, the exposure ratio is positively correlated with the light intensity, that is, when the light intensity shows a downward trend, the exposure ratio also shows a downward trend.

[0097] In a possible implementation, the light intensity is represented by illuminance, and the unit of illuminance is lux.

[0098] In the technical solution of the embodiments of this application, in an HDR environment, when light intensity decreases, to ensure image quality, the AEC module increases the analog gain of both long and short frames simultaneously, with the analog gain of the short frame increasing at a faster rate than that of the long frame, thereby gradually reducing the exposure ratio. Specific examples can be found in Table 3 below.

[0099] Table 3: Gain and exposure ratio changes Table 1

[0100] Each row of data in Table 3 is, from top to bottom, exposure time, simulated exposure gain of short frame, simulated exposure gain of long frame, and exposure ratio.

[0101] The data in Table 3 show, from left to right, the changes in various data as the light intensity gradually decreases in the HDR environment.

[0102] For example, in LOFIC HDR mode, the circuit's well capacitance when reading short frames is 24 times that of long frames. Table 3 shows that the analog gain increases faster for short frames, causing the exposure ratio to decrease as light intensity decreases.

[0103] S41: When the analog gain of the long frame is greater than the first preset gain and / or the light intensity is less than the first preset light intensity, the HDR mode is switched from the LOFIC HDR mode to the DAG HDR mode.

[0104] In an embodiment of the present application, when the SNR dip in the transition area between the bright and dark parts of the picture is more serious, it is necessary to switch the HDR mode. Whether to switch the HDR mode can be determined based on the analog gain size and / or light intensity of the long frame, which is explained in detail below.

[0105] In one possible implementation, whether to switch from LOFIC HDR mode to DAG HDR mode is determined based on the analog gain of the long frame. As the light intensity decreases, the analog gain of the long frame is gradually increased. When the analog gain of the long frame increases to be greater than the first preset gain, the HDR mode is switched from LOFIC HDR mode to DAG HDR mode. The embodiment of the present application does not specifically limit the first preset gain. In actual applications, the first preset gain can be pre-calibrated according to the specific shooting capabilities of the camera of the electronic device. In the above Table 3, the first preset gain is taken as 8 as an example. When the analog gain of the long frame is greater than 8, the HDR mode is switched. The process of gradually increasing the analog gain of the long frame can correspond to the process of gradually converting curve 1 to curve 2 in Figure 2, and the SNR dip phenomenon becomes increasingly serious.

[0106] In another possible implementation, whether to switch from LOFIC HDR mode to DAG HDR mode is determined based on the light intensity. As the light intensity decreases, the analog gain of the long frame is gradually increased. When it is determined that the light intensity at this time is less than the first preset light intensity, the HDR mode is switched from LOFIC HDR mode to DAG HDR mode. The embodiment of the present application does not specifically limit the first preset light intensity. In actual applications, the first preset light intensity can be pre-calibrated according to the specific shooting capability of the camera of the electronic device.

[0107] In another possible implementation, the long frame's analog gain or light intensity can be used to determine whether to switch modes. When either condition indicates a need for mode switching, the mode switch is performed. For example, when the long frame's analog gain is greater than a first preset gain, but the light intensity determination result has not yet been generated, the mode switch is determined to be performed. For another example, when the light intensity is less than the first preset intensity, but the analog gain determination result has not yet been generated, the mode switch is determined to be performed. This implementation utilizes the first condition obtained as the criterion, resulting in a faster response and greater sensitivity to mode switching.

[0108] In another possible implementation, the analog gain size of the long frame and the light intensity can be used to determine the mode switching. When the analog gain is greater than the first preset gain and the light intensity is less than the first preset intensity, the HDR mode is switched from the LOFIC HDR mode to the DAG HDR mode. This implementation method is more accurate.

[0109] The electronic device stores the first preset gain and the corresponding first preset light intensity, which are called when needed.

[0110] S42: The AEC module adjusts the exposure ratio and exposure time of the DAG HDR mode.

[0111] In order to make the switching of HDR mode smooth, it is necessary to maintain the brightness and exposure ratio of the picture before and after the HDR mode switching.

[0112] Specifically, when the exposure ratio of the DAG HDR mode is different from the exposure ratio of the LOFIC HDR mode, the AEC module will use the exposure ratio of the LOFIC HDR mode as the exposure ratio of the DAG HDR mode after the HDR mode is switched. The analog gains corresponding to the long frame and short frame of the DAG HDR mode are determined based on this exposure ratio. In addition, when the maximum exposure ratio at the current exposure time is less than the exposure ratio before the HDR mode is switched, the exposure ratio can be adjusted by reducing the exposure time and increasing the analog gain of the long frame.

[0113] Before mode switching, the analog gain of the short frame is 12 and the analog gain of the long frame is 8. When reading the short frame, the well capacitance of the circuit is 24 times that of the circuit when reading the long frame, so the exposure ratio R1 is 16.

[0114] After switching modes, the analog gain of the short frame can be set to a smaller value to maximize the exposure ratio variation range of the DAG HDR mode and fully utilize the performance of the DAG HDR mode. In this embodiment of the present application, the analog gain of the short frame is set to 1 times the minimum value as an example. In this case, to maintain the exposure ratio unchanged, the analog gain of the long frame in the DAG HDR mode will be configured to 16.

[0115] In order to maintain consistent image brightness before and after mode switching, the exposure time can be shortened, that is, the image brightness can be reduced by shortening the exposure time from 20ms to 10ms.

[0116] The longer the exposure time, the more photons reach the charge coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) phototransistor, and the brighter the captured image will be.

[0117] In the embodiment of the present application, when the digital gain remains unchanged, the value of the product of the exposure time and the analog gain of the long frame is maintained unchanged before and after the mode switching. The data in Table 3 are explained. The value of the product of the exposure time and the analog gain of the long frame before the mode switching is 20*8=160, and the value of the product of the exposure time and the analog gain of the long frame after the mode switching is 10*16=160, thereby ensuring that the screen brightness does not jump before and after the mode switching.

[0118] It is understood that in actual applications, the AEC module can also ensure that the exposure ratio difference before and after the mode switch is within a first preset range, and the image brightness difference is within a second preset range. In other words, a smaller difference in exposure ratio and image brightness before and after the HDR mode switch is allowed to reduce the difficulty of adjustment and improve the adjustment speed.

[0119] In summary, the technical solution provided by the embodiment of the present application increases the analog gain of the long frame and the analog gain of the short frame of the LOFIC HDR mode when the light intensity gradually decreases, and makes the increase rate of the analog gain of the short frame faster, thereby reducing the exposure ratio. When the analog gain of the long frame is greater than the first preset gain and / or the light intensity is less than the first preset light intensity, the HDR mode is switched from the LOFIC HDR mode to the DAG HDR mode. By switching the HDR mode, the SNR dip problem is alleviated. At this time, since the exposure ratio has been lowered in advance, the exposure ratio of the LOFIC HDR mode will not be greater than the maximum exposure ratio of the DAG HDR mode when the mode is switched. Therefore, the DAG HDR mode can adopt the exposure ratio before the mode switching, thereby avoiding the sudden change of the exposure ratio, making the switching of the HDR mode natural. In addition, by adjusting the exposure time, the brightness of the picture before and after switching the HDR mode does not change suddenly, making the switching of the HDR mode more natural. Therefore, this solution can enhance the user's photo-taking experience.

[0120] The above embodiments describe how to adjust the analog gain of two HDR image frames before fusing them. The following describes how to first fuse the two HDR image frames and then adjust the ISP's digital gain to achieve a natural HDR mode switch.

[0121] Refer to FIG5 , which is a flowchart of another high dynamic range mode switching method provided in an embodiment of the present application.

[0122] The method comprises the following steps:

[0123] S50: When the light intensity decreases, the AEC module increases the analog gain of the short frame of the LOFIC HDR mode, maintains the analog gain of the short frame of the LOFIC HDR mode unchanged, and increases the digital gain of the LOFIC HDR mode.

[0124] The digital gain of the LOFIC HDR mode refers to the gain of the digital image signal after the Image Signal Processor (ISP) converts the electrical signal into a digital image signal after the two frames of the LOFIC HDR mode are fused into one frame.

[0125] The automatic exposure control module determines the dynamic range of the current scene. The AEC module can perform light intensity measurement, scene analysis, and exposure compensation.

[0126] The automatic exposure control module adjusts the exposure ratio of the LOFIC HDR mode based on light intensity, ensuring that the exposure ratio is positively correlated with light intensity. That is, the exposure ratio decreases as light intensity decreases. In one possible implementation, light intensity is represented by illuminance, which is measured in lux.

[0127] In the technical solution of the embodiments of this application, in an HDR environment, as light intensity gradually decreases, to ensure image quality, the AEC module gradually increases the analog gain of the short frames in the LOFIC HDR mode while maintaining the analog gain of the short frames in the LOFIC HDR mode unchanged, thereby gradually reducing the exposure ratio. Specific examples can be found in Table 4 below.

[0128] Table 4: Gain and exposure ratio changes Table 2

[0129] The data in each row in Table 4 are, from top to bottom, the exposure time, the product of the analog exposure gain of the short frame and the digital exposure gain of the short frame, the product of the analog exposure gain of the long frame and the digital exposure gain of the long frame, and the exposure ratio.

[0130] The data in Table 4 show, from left to right, how the data changes when the light intensity gradually decreases in the HDR environment.

[0131] For example, in LOFIC HDR mode, the circuit's well capacitance when reading short frames is 24 times that of long frames. Table 4 shows that the analog gain (also known as sensor gain) increases faster for short frames, causing the exposure ratio to decrease as light intensity decreases.

[0132] After the two frames are fused, the analog gain of the two frames cannot be adjusted independently. Therefore, the digital gain of the long and short frames in Table 4 is the same. The digital gain also increases as the light intensity decreases.

[0133] S51: When the digital gain increases to be greater than the second preset gain and / or the light intensity is less than the second preset light intensity, the HDR mode is switched from the LOFIC HDR mode to the DAG HDR mode.

[0134] Since the technical solution of this application first fuses two frames of images and then increases the digital gain, and the digital gain acts on the digital signal, the increase in digital gain will cause the signal-to-noise ratio of the entire image to decrease. Therefore, when the signal-to-noise ratio of the entire image drops to a certain level, it is necessary to switch to HDR mode. The determination of whether to switch to HDR mode can be made based on the size of the digital gain and the light intensity. The details are explained below.

[0135] In one possible implementation, whether to switch from LOFIC HDR mode to DAG HDR mode is determined based on the size of the digital gain. As the light intensity decreases, in order to ensure the quality of the picture, the digital gain gradually increases, resulting in a decrease in the signal-to-noise ratio of the entire image. When the digital gain increases to be greater than the second preset gain, it is considered that the signal-to-noise ratio of the entire image caused by the digital gain is significantly reduced at this time, and it is necessary to switch to HDR mode. The embodiment of the present application does not specifically limit the second preset gain. In actual applications, the second preset gain can be pre-calibrated according to the specific shooting capabilities of the camera of the electronic device. In the above Table 4, the second preset gain of 8 is taken as an example.

[0136] In another possible implementation, whether to switch from LOFIC HDR mode to DAG HDR mode is determined based on the light intensity. At this time, as the light intensity decreases, the digital gain will increase accordingly. When the light intensity drops to less than the second preset light intensity, it is considered that the full-image signal-to-noise ratio caused by the digital gain is greatly reduced, and the HDR mode needs to be switched. The embodiment of the present application does not specifically limit the second preset light intensity. In actual applications, the second preset light intensity can be pre-calibrated according to the specific shooting capability of the camera of the electronic device.

[0137] In another possible implementation, when it is determined that the gain is greater than the second preset gain or the light intensity is less than the second preset light intensity, mode switching is performed, and the first switching condition obtained is used as the criterion, which has a faster response speed and higher sensitivity to mode switching.

[0138] In another possible implementation, when the digital gain increases to be greater than the second preset gain and the light intensity is less than the second preset light intensity, the HDR mode is switched from the LOFIC HDR mode to the DAG HDR mode. This implementation makes the judgment more accurate.

[0139] S52: The AEC module adjusts the exposure time and exposure ratio of the DAG HDR mode.

[0140] In order to make the switching of HDR mode smooth, it is necessary to maintain the brightness and exposure ratio of the picture before and after the HDR mode switching.

[0141] Specifically, when the exposure ratio of the DAG HDR mode is different from the exposure ratio of the LOFIC HDR mode, the AEC module will use the exposure ratio of the LOFIC HDR mode as the exposure ratio of the DAG HDR mode after the HDR mode is switched, and the analog gains corresponding to the long frame and short frame of the DAG HDR mode are determined according to the exposure ratio. In addition, when the maximum exposure ratio under the current exposure time is less than the exposure ratio before the HDR mode is switched, the exposure ratio can be adjusted by reducing the exposure time and increasing the analog gain of the long frame. For example, the maximum analog gain of the long frame corresponding to 20ms is 8, that is, the corresponding maximum exposure ratio is 8, which is less than 16. The exposure time is adjusted to 10ms, so that a larger exposure ratio can be used.

[0142] Before the mode switch, the analog gain of the long frame remains unchanged, and the analog gain of the long frame in Table 4 remains 1. The analog gain of the short frame gradually increases, causing the exposure ratio before the mode switch to decrease.

[0143] After the mode is switched, in order to maximize the exposure ratio variation range of the DAG HDR mode and to give full play to the performance of the DAG HDR mode, the analog gain of the short frame can be selected as a smaller value. In the embodiment of the present application, the analog gain of the short frame is taken as 1 times the minimum value as an example. At this time, in order to maintain the exposure ratio unchanged, the analog gain of the long frame of the DAG HDR mode will be configured to 16. In order to maintain the same image brightness before and after the mode switch, it is necessary to shorten the exposure time, that is, to reduce the image brightness by shortening the exposure time from 20ms to 10ms.

[0144] In an embodiment of the present application, the analog gain of the long frame remains unchanged, and the value of the product of the exposure time and the exposure gain of the LOFIC HDR mode is maintained to be the same as the value of the product of the exposure time of the DAG HDR mode and the exposure gain of the long frame after the mode is switched, thereby ensuring that the brightness of the picture does not jump before and after the mode switch.

[0145] Corresponding to the data in Table 4, the value of the product of the exposure time and the exposure gain before the mode switching is 20*1*8=160.

[0146] After the mode is switched, the digital gain of the DAG HDR mode is configured to 1 by default. At this time, the product of the exposure time and the exposure gain of the long frame is 10*16*1=160.

[0147] It is understood that in actual applications, the AEC module can also ensure that the exposure ratio difference before and after the mode switch is within a first preset range, and the image brightness difference is within a second preset range. In other words, a small difference in exposure ratio and image brightness before and after the HDR mode switch is allowed to reduce the difficulty of adjustment and improve the adjustment speed.

[0148] In summary, the technical solution provided by the embodiment of the present application maintains the analog gain of the long frame of the LOFIC HDR mode unchanged when the light intensity gradually decreases, increases the exposure gain of the short frame of the LOFIC HDR mode, thereby reducing the exposure ratio, and increasing the digital gain of the LOFIC HDR mode. When the digital gain increases to be greater than the second preset gain and / or the light intensity is less than the second preset light intensity, the HDR mode is switched from the LOFIC HDR mode to the DAG HDR mode. By switching the HDR mode, the SNR dip problem is alleviated. At this time, since the exposure ratio has been lowered in advance, the sudden change of the exposure ratio is avoided, making the switching of the HDR mode natural. In addition, by adjusting the exposure time, the brightness of the picture before and after switching the HDR mode does not change suddenly, making the switching of the HDR mode more natural. Therefore, this solution can enhance the user's photo-taking experience.

[0149] The above embodiment describes a method for first fusing two HDR mode images and then adjusting the ISP's digital gain to achieve a natural HDR mode switch. The following describes a method for adjusting the analog gain of the two HDR mode images before fusing them, and then adjusting the ISP's digital gain after fusing them to achieve a natural HDR mode switch.

[0150] See FIG. 6 , which is a flowchart of another method for switching to a high dynamic range mode provided in an embodiment of the present application.

[0151] The method comprises the following steps:

[0152] S60: When the light intensity decreases, the AEC module maintains the digital gain of the LOFIC HDR mode unchanged, increases the analog gain of the long frame and short frame of the LOFIC HDR mode, and reduces the exposure ratio of the LOFIC HDR mode.

[0153] In LOFIC HDR mode, there are two exposure gain adjustment processes. In the first exposure gain adjustment process, the digital gain of LOFIC HDR mode remains unchanged, the analog gain of the long and short frames of LOFIC HDR mode is increased, and the exposure ratio of LOFIC HDR mode is reduced.

[0154] When the AEC module adjusts the exposure ratio of the LOFIC HDR mode according to the light intensity, the exposure ratio is positively correlated with the light intensity. That is, when the light intensity shows a decreasing trend, the exposure ratio also shows a decreasing trend.

[0155] In one possible implementation, the analog gain growth rate of the short frame is greater than the analog gain growth rate of the long frame, thereby gradually reducing the exposure ratio. Specific examples can be seen in Table 5 below.

[0156] Table 5: Gain and exposure ratio changes Table 3

[0157] The data in each row in Table 5 are, from top to bottom, the exposure time, the product of the analog exposure gain of the short frame and the digital exposure gain of the short frame, the product of the analog exposure gain of the long frame and the digital exposure gain of the long frame, and the exposure ratio.

[0158] The data in Table 5 show, from left to right, the changes in various data when the light intensity gradually decreases in the HDR environment.

[0159] For example, in LOFIC HDR mode, the circuit's well capacitance when reading short frames is 24 times that of long frames. Table 5 shows that during the first exposure gain adjustment, the digital gain remains constant at 1, while the analog gain for short frames increases at a faster rate than that for long frames, allowing the exposure ratio to be gradually reduced.

[0160] S61: When the analog gain of the long frame is greater than the third preset gain and / or the light intensity is less than the third preset light intensity, the AEC module increases the digital gain of the LOFIC HDR mode and the analog gain of the short frame, and maintains the analog gain of the long frame unchanged.

[0161] In this embodiment of the present application, when the SNR dip in the transition region between bright and dark parts of the image is large, the first exposure gain adjustment process is terminated and a second exposure gain adjustment process is initiated. Whether to terminate the first exposure gain adjustment process can be determined based on the analog gain of the long frame and / or the light intensity.

[0162] In one possible implementation, whether to terminate the first exposure gain adjustment process is determined based on the analog gain of the long frame. The embodiment of the present application does not specifically limit the third preset gain. In actual applications, the third preset gain can be pre-calibrated according to the specific shooting capabilities of the camera of the electronic device. In the above Table 5, taking the third preset gain of 4 as an example, when the analog gain of the long frame is greater than 4, the first exposure gain adjustment process is terminated, and the analog gain of the long frame remains unchanged. The process of gradually increasing the analog gain of the long frame can correspond to the process of gradually converting curve 1 to curve 2 in Figure 2, and the SNR dip phenomenon becomes increasingly serious.

[0163] In another possible implementation, whether to switch from LOFIC HDR mode to DAG HDR mode is determined based on the light intensity. The embodiment of the present application does not specifically limit the third preset light intensity. In actual applications, the third preset light intensity can be pre-calibrated based on the specific shooting capabilities of the camera of the electronic device. The third preset light intensity in the embodiment of the present application is the light intensity when the AEC module adjusts the analog gain of the long frame to 4.

[0164] In another possible implementation, the analog gain or light intensity of the long frame can be used to determine whether to end the first exposure gain adjustment process. This implementation uses the first acquired condition as the judgment criterion, which has a faster response speed and higher sensitivity.

[0165] In another possible implementation, the analog gain size and light intensity of the long frame can be used for judgment. When the analog gain is greater than the first preset gain and the light intensity is less than the first preset light intensity, the first exposure gain adjustment process is ended. This implementation method makes the judgment more accurate.

[0166] After the first exposure gain adjustment process is completed, the second exposure gain adjustment process begins. In the second exposure gain adjustment process, the AEC module increases the digital gain of the LOFIC HDR mode and the analog gain of the short frame, while maintaining the analog gain of the long frame unchanged.

[0167] S62: When the digital gain increases to be greater than a fourth preset gain and / or the light intensity is less than a fourth preset light intensity, the HDR mode is switched from the LOFIC HDR mode to the DAG HDR mode.

[0168] Digital gain acts on the digital signal. Increasing digital gain will cause the signal-to-noise ratio of the entire image to decrease. Therefore, when the signal-to-noise ratio of the entire image drops to a certain level, it is necessary to switch to HDR mode. The decision to switch to HDR mode can be made based on the size of the digital gain and the light intensity. The details are explained below.

[0169] In one possible implementation, whether to switch from LOFIC HDR mode to DAG HDR mode is determined based on the size of the digital gain. As the light intensity decreases, in order to ensure the quality of the picture, the digital gain gradually increases, resulting in a decrease in the signal-to-noise ratio of the entire image. When the digital gain increases to be greater than the fourth preset gain, it is considered that the signal-to-noise ratio of the entire image caused by the digital gain is significantly reduced at this time, and it is necessary to switch to HDR mode. The embodiment of the present application does not specifically limit the fourth preset gain. In actual applications, the fourth preset gain can be pre-calibrated according to the specific shooting capabilities of the camera of the electronic device. In the above Table 5, the fourth preset gain of 2 is taken as an example.

[0170] In another possible implementation, whether to switch from LOFIC HDR mode to DAG HDR mode is determined based on the light intensity. At this time, as the light intensity decreases, the digital gain will increase accordingly. When the light intensity drops to less than the fourth preset light intensity, it is considered that the full-image signal-to-noise ratio caused by the digital gain is greatly reduced, and the HDR mode needs to be switched. The embodiment of the present application does not specifically limit the fourth preset light intensity. In actual applications, the fourth preset light intensity can be pre-calibrated according to the specific shooting capabilities of the camera of the electronic device.

[0171] In another possible implementation, when it is determined that the gain is greater than the fourth preset gain or the light intensity is less than the fourth preset light intensity, mode switching is performed, and the first switching condition obtained is used as the criterion, which has a faster response speed and higher sensitivity to mode switching.

[0172] In another possible implementation, when the digital gain increases to be greater than the fourth preset gain and the light intensity is less than the fourth preset light intensity, the HDR mode is switched from the LOFIC HDR mode to the DAG HDR mode. This implementation makes the judgment more accurate.

[0173] S63: The AEC module adjusts the exposure time and exposure ratio of the DAG HDR mode.

[0174] In order to make the switching of HDR mode smooth, it is necessary to maintain the brightness and exposure ratio of the picture before and after the HDR mode switching.

[0175] Specifically, when the exposure ratio of the DAG HDR mode is different from the exposure ratio of the LOFIC HDR mode, the AEC module uses the exposure ratio of the LOFIC HDR mode as the exposure ratio of the DAG HDR mode after the HDR mode is switched. The analog gains corresponding to the long frame and short frame of the DAG HDR mode are determined based on the exposure ratio. In addition, when the maximum exposure ratio under the current exposure time is less than the exposure ratio before the HDR mode is switched, the exposure ratio can be adjusted by reducing the exposure time and increasing the analog gain of the long frame.

[0176] In order to maximize the exposure ratio variation range of the DAG HDR mode and give full play to the performance of the DAG HDR mode, the analog gain of the short frame can be selected as a smaller value. In the embodiment of the present application, the analog gain of the short frame is taken as 1 times the minimum value as an example. At this time, in order to maintain the exposure ratio unchanged, the analog gain of the long frame of the DAG HDR mode will be configured to 16. In order to maintain the same image brightness before and after the mode switching, it is necessary to shorten the exposure time, that is, to reduce the image brightness by shortening the exposure time from 20ms to 10ms.

[0177] In an embodiment of the present application, the value of the product of the exposure time and the exposure gain of the LOFIC HDR mode is maintained to be the same as the value of the product of the exposure time and the exposure gain of the DAG HDR mode after the mode is switched, so that the brightness of the picture does not jump before and after the mode switch.

[0178] Corresponding to the data in Table 5, the value of the product of the exposure time and the exposure gain before the mode switching is 20*4*2=160.

[0179] After the mode is switched, the digital gain of the DAG HDR mode is configured to 1 by default. At this time, the product of the exposure time and the exposure gain of the long frame is 10*16*1=160.

[0180] It is understood that in actual applications, the AEC module can also ensure that the exposure ratio difference before and after the mode switch is within a first preset range, and the image brightness difference is within a second preset range. In other words, a small difference in exposure ratio and image brightness before and after the HDR mode switch is allowed to reduce the difficulty of adjustment and improve the adjustment speed.

[0181] In summary, the technical solution provided by the embodiments of the present application sequentially performs two exposure gain adjustment processes as the light intensity gradually decreases. In the first exposure gain adjustment process, the digital gain of the LOFIC HDR mode is maintained unchanged, and the analog gain growth rate of the long frame is set to be lower than that of the short frame. In the second exposure gain adjustment process, the digital gain of the LOFIC HDR mode and the analog gain of the short frame are increased, while the analog gain of the long frame is maintained unchanged. Through these two adjustment processes, the SNR dip in the image transition area can be alleviated before switching to HDR mode, and a significant drop in the signal-to-noise ratio in the dark portion of the image can be avoided. After switching to HDR mode, the SNR dip problem is alleviated. At this time, because the exposure ratio has been lowered in advance, the exposure ratio of the LOFIC HDR mode will not be greater than the maximum exposure ratio of the DAG HDR mode when switching modes. Therefore, the DAG HDR mode can adopt the exposure ratio before the mode switch, avoiding sudden changes in the exposure ratio and making the HDR mode switch more natural. In addition, by adjusting the exposure time, the image brightness before and after switching to HDR mode does not suddenly change, making the HDR mode switch more natural. Therefore, this solution can enhance the user's photography experience.

[0182] Based on the high dynamic range mode switching method provided in the above embodiments, an embodiment of the present application further provides an electronic device, which is described in detail below with reference to the accompanying drawings.

[0183] The electronic device provided in the embodiment of the present application can be a mobile phone, a laptop computer, a wearable electronic device (such as a smart watch, smart glasses), a tablet computer, an AR device or a VR device, etc. The following description will take the electronic device as an example of a mobile phone.

[0184] See FIG. 7 , which is a schematic diagram of an electronic device provided in an embodiment of the present application.

[0185] The electronic device 100 includes a processor 110, a camera 120, a display screen 130, a sensor module 140, and an internal storage area 150. The sensor module 140 may include a color temperature sensor 141, and may also include other types of sensors.

[0186] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU), etc.

[0187] The electronic device 100 can implement a shooting function through an image signal processor (ISP), a camera 120, a video codec, a GPU, a display screen 130, and an application processor.

[0188] The ISP is used to process data fed back by the camera 120. For example, when shooting, the shutter is opened, and light is transmitted through the lens to the camera sensor. The light signal is converted into an electrical signal, which is then transmitted to the ISP for processing and converted into an image visible to the glasses. The ISP can also perform algorithmic optimization on image noise, brightness, and skin color. The ISP can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be installed in the camera 120.

[0189] The camera 120 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include multiple cameras.

[0190] The color temperature sensor 141 is used to sense the ambient color temperature and illumination, thereby enabling the electronic device to perform automatic exposure control based on the detection results.

[0191] The internal memory 150 can be used to store computer executable program code, which includes instructions. The internal memory 150 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100. In addition, the internal memory 150 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 implements the switching method of the high dynamic range mode in the above embodiment by running instructions stored in the internal memory 150, and / or instructions stored in a memory provided in the processor.

[0192] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0193] The software system of the electronic device is described below with reference to the accompanying drawings.

[0194] See Figure 8, which is an architecture diagram of the software system of the electronic device provided in an embodiment of the present application.

[0195] The embodiment of the present application takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device.

[0196] A layered architecture divides software into several layers, which communicate with each other through software interfaces. In some embodiments, the Android system consists of, from top to bottom, the application layer, the application framework layer (Framework), the system runtime layer, the hardware abstraction layer (HAL), and the kernel layer (Linux kernel). For ease of understanding, the hardware layer is also shown in the figure.

[0197] The application layer includes a series of application packages. Both built-in system applications and non-system-level applications belong to the application layer and are responsible for direct interaction with the user. For example, this may include the camera app.

[0198] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0199] The system runtime library layer mainly includes the Android runtime library and program library (Native C / C++ Libraries), which can provide camera services.

[0200] The Hardware Abstraction Layer (HAL) is a routine package of the software layer. It is an interface layer located between the operating system kernel and the hardware circuit. Its purpose is to abstract the hardware and simulate the details of a specific system platform so that programs can directly access the hardware resources.

[0201] The hardware abstraction layer includes the CAMX-CHI architecture, which provides the HAL3 interface for the Camera Provider to call and receive requests from the Camera Provider. The HAL3 interface is implemented internally and the camera driver layer is controlled through the V4L2 standard framework. The request is sent to the driver part, and the result is returned and then reported to the Camera Provider.

[0202] The CAMX-CHI architecture consists of two parts: CAMX and the camera hardware interface (CHI)-camera development kit (CDK).

[0203] CAMX is responsible for implementing the basic service code. It primarily includes the csl module, which implements interaction with the driver; swl, which implements the software node and enables automatic exposure control (AEC); and sensorNode, which generates sensor configuration instructions. csl is the communication module responsible for communication between CAMX and the driver layer.

[0204] CHI-CDK is responsible for realizing the needs of scalability and customization, making it convenient for original equipment manufacturers (OEM) and original design manufacturers (ODM) to add their own extended functions.

[0205] The module module in the OEM module stores configuration files of different sensors and is needed when initializing the sensor.

[0206] The EEPROM module in the OEM module stores the configuration information of the Electrically Erasable Programmable Read Only Memory (EEPROM).

[0207] The sensor module in the OEM module stores register configuration parameters for each sensor mode. For example, it can store configuration parameters related to the LOFIC HDR mode and the DAG HDR mode.

[0208] The Linux kernel layer is the layer between hardware and software. It includes at least basic Linux drivers, such as V2L2. It also includes device drivers, such as sensor drivers. The camera request management (CRM) in the figure manages camera configuration requests.

[0209] When a user opens the camera app for preview or video recording, the electronic device can determine whether to enable HDR mode based on the dynamic range of the current environment. The electronic device can support LOFIC HDR mode and DAG HDR mode. In addition, the electronic device can also support other HDR modes, which are not specifically limited in the embodiments of the present application.

[0210] Since the dynamic range adapted by the LOFIC HDR mode is greater than that of the DAG HDR mode, electronic devices first adopt the LOFIC HDR mode.

[0211] When the ambient light intensity gradually decreases, the solution in the above method embodiment is used to switch the LOFIC HDR mode to the DAG HDR mode, alleviating the SNR dip problem. In addition, by adjusting the exposure time, the brightness of the picture before and after switching the HDR mode does not change suddenly, making the HDR mode switching more natural. Therefore, using this electronic device improves the user's shooting experience.

[0212] An embodiment of the present application further provides a storage medium having a program stored thereon, which, when executed by a processor, implements the method for selecting the high dynamic range mode in the above embodiment.

[0213] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information using any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, parameter random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically-erasable programmable read-only memory (EEPROM), flash memory, or other memory technologies.

[0214] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for switching high dynamic range modes, characterized in that, it is applied to an electronic device supporting an overflow set capacitor (LOFIC) high dynamic range (HDR) mode and a dual analog gain (DAG) HDR mode, and the method includes: Starting the camera running program; Obtaining the light intensity of the environment where the electronic device is located; Enabling the LOFIC HDR mode; When the light intensity drops below a first light intensity, switching the HDR mode to the DAG HDR mode.

2. The method according to claim 1, characterized in that, when the light intensity is greater than or equal to the first light intensity, the method further includes: When the light intensity drops, controlling the exposure gain of the LOFIC HDR mode to increase as the light intensity decreases, and controlling the exposure ratio of the LOFIC HDR mode to decrease as the light intensity decreases.

3. The method according to claim 2, characterized in that, the exposure gain includes an analog gain and a digital gain, and controlling the exposure gain of the LOFIC HDR mode to increase as the light intensity decreases and controlling the exposure ratio of the LOFIC HDR mode to decrease as the light intensity decreases specifically includes: Controlling the digital gain of the LOFIC HDR mode to remain unchanged; Controlling the increasing rate of the analog gain of the short frame of the LOFIC HDR mode to increase as the light intensity decreases to be greater than the increasing rate of the analog gain of the long frame of the LOFIC HDR mode to increase as the light intensity decreases.

4. The method according to claim 3, characterized in that, when the light intensity is less than the first light intensity, switching the HDR mode to the DAG HDR mode specifically includes: Determining the analog gain of the short frame of the LOFIC HDR mode and the analog gain of the short frame of the LOFIC HDR mode according to the light intensity; When the analog gain of the long frame is greater than a first preset gain, switching the HDR mode to the DAG HDR mode, where the first preset gain is the analog gain of the short frame of the LOFIC HDR mode when the light intensity is the first light intensity; Controlling the exposure ratio to remain unchanged before and after the HDR mode switch.

5. The method according to claim 2, characterized in that, the exposure gain includes an analog gain and a digital gain, and controlling the exposure gain of the LOFIC HDR mode to increase as the light intensity decreases and controlling the exposure ratio of the LOFIC HDR mode to decrease as the light intensity decreases specifically includes: Controlling the analog gain of the long frame of the LOFIC HDR mode to remain unchanged; Controlling the analog gain of the short frame of the LOFIC HDR mode to increase as the light intensity decreases; Controlling the digital gain of the LOFIC HDR mode to increase as the light intensity decreases.

6. The method according to claim 5, characterized in that, when the light intensity is less than the first light intensity, switching the HDR mode to the DAG HDR mode specifically includes: Determine the analog gain of the short frame in the LOFIC HDR mode and the digital gain of the LOFIC HDR mode according to the light intensity; When the digital gain of the LOFIC HDR mode is greater than a second preset gain, switch the HDR mode to the DAG HDR mode, where the second preset gain is the digital gain of the LOFIC HDR mode when the light intensity is a first light intensity; Control the exposure ratio before and after the HDR mode switch to remain unchanged.

7. The method according to claim 2, wherein, the exposure gain includes an analog gain and a digital gain, and controlling the exposure gain of the LOFIC HDR mode to increase as the light intensity decreases and controlling the exposure ratio of the LOFIC HDR mode to decrease as the light intensity decreases specifically includes: controlling the digital gain of the LOFIC HDR mode to remain unchanged, and controlling the increasing rate of the analog gain of the short frame of the LOFIC HDR mode to increase as the light intensity decreases to be greater than the increasing rate of the analog gain of the long frame of the LOFIC HDR mode to increase as the light intensity decreases; when the analog gain of the long frame of the LOFIC HDR mode is greater than a third preset gain, controlling the analog gain of the long frame of the LOFIC HDR mode to remain unchanged, the analog gain of the short frame of the LOFIC HDR mode to increase as the light intensity decreases, and the digital gain of the LOFIC HDR mode to increase as the light intensity decreases.

8. The method according to claim 7, wherein, when the light intensity is less than the first light intensity, switching the HDR mode to the DAG HDR mode specifically includes: Determine the analog gain of the short frame in the LOFIC HDR mode and the digital gain of the LOFIC HDR mode according to the light intensity; When the digital gain of the LOFIC HDR mode is greater than a fourth preset gain, switch the HDR mode to the DAG HDR mode, where the fourth preset gain is the digital gain of the LOFIC HDR mode when the light intensity is a first light intensity; Control the exposure ratio before and after the HDR mode switch to remain unchanged.

9. The method according to claim 4 or 6 or 8, wherein, controlling the exposure ratio before and after the HDR mode switch to remain unchanged specifically includes: When the maximum exposure ratio corresponding to the current exposure time is less than the exposure ratio before the HDR mode switch, decrease the exposure time and increase the analog gain of the long frame of the DAG HDR mode to control the exposure ratio before and after the HDR mode switch to remain unchanged, where the exposure time is negatively correlated with the maximum exposure ratio corresponding to the exposure time.

10. The method according to any one of claims 1-9, wherein, the method further includes: Control the picture brightness to be consistent before and after the HDR mode switch.

11. The method according to claim 10, wherein, controlling the picture brightness to be consistent before and after the HDR mode switch specifically includes: The product of the exposure time before switching the HDR mode and the exposure gain of the long frame in the LOFIC HDR mode is equal to the product of the exposure time after switching the HDR mode and the exposure gain of the long frame in the DAG HDR mode.

12. An electronic device, characterized in that the electronic device includes a processor and a memory, the memory is used to store a program, and when the program is run by the processor, it executes the method for switching the high dynamic range mode according to any one of claims 1-11.

13. A storage medium, characterized in that a computer program is stored on the storage medium, and when the computer program is executed by an electronic device, it implements the method for switching the high dynamic range mode according to any one of claims 1-11.

Images