Photographing method and electronic device

Abstract

The application discloses a photographing method and an electronic device, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a scene dynamic range corresponding to a current photographing scene; determining a target exposure mode conforming to the scene dynamic range from at least two exposure modes according to the scene dynamic range; the at least two exposure modes comprise a multiple exposure mode and at least one single exposure mode, the single exposure mode indicates that at least two images are acquired through different gains in a single exposure process, and the multiple exposure mode indicates that at least two images are acquired by controlling an exposure time; photographing the current photographing scene based on the target exposure mode to obtain a high dynamic range image; and the target exposure mode comprises at least one of the multiple exposure mode and the single exposure mode.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a shooting method and an electronic device. Background Technology

[0002] Cameras have limitations in their imaging performance of bright and dark areas in the same scene. Brighter areas may appear white due to overexposure, while darker areas may appear black due to underexposure. This limitation is called dynamic range, and dynamic range affects the image quality after imaging.

[0003] Among related technologies, High Dynamic Range Imaging (HDRI / HDR) technology can achieve a greater dynamic range. Specifically, multiple exposure HDR technology is the multiple exposure mode of a camera device, which can fuse multiple frames of images with different exposure times to obtain an image with more details in both bright and dark areas.

[0004] However, when reading multiple frames of images in multi-exposure mode, there is a time difference, or frame difference, which may cause motion blur in moving objects in the resulting image after fusing multiple frames. Summary of the Invention

[0005] The purpose of this application is to provide a shooting method that can solve the problem of motion blur caused by frame difference when using multiple exposure HDR technology.

[0006] In a first aspect, embodiments of this application provide a shooting method, the method comprising:

[0007] Obtain the scene dynamic range corresponding to the current shooting scene;

[0008] Based on the scene dynamic range, a target exposure mode that conforms to the scene dynamic range is determined from at least two exposure modes; the at least two exposure modes include a multiple exposure mode and at least one single exposure mode, wherein the single exposure mode indicates that at least two frames of images are acquired through different gains during a single exposure, and the multiple exposure mode indicates that at least two frames of images are acquired by controlling the exposure time.

[0009] A high dynamic range image is obtained by shooting in the current shooting scene based on the target exposure mode; the target exposure mode includes at least one of the multiple exposure mode and the single exposure mode.

[0010] Secondly, embodiments of this application provide a shooting device, the device comprising:

[0011] The first acquisition module is used to acquire the scene dynamic range corresponding to the current shooting scene;

[0012] The first determining module is configured to determine a target exposure mode that conforms to the scene dynamic range from at least two exposure modes based on the scene dynamic range; the at least two exposure modes include a multiple exposure mode and at least one single exposure mode, the single exposure mode indicating that at least two frames of images are acquired through different gains in a single exposure process, and the multiple exposure mode indicating that at least two frames of images are acquired by controlling the exposure time.

[0013] The shooting module is used to shoot in the current shooting scene based on the target exposure mode to obtain a high dynamic range image; the target exposure mode includes at least one of the multiple exposure mode and the single exposure mode.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores a program or instructions executable on the processor, and the program or instructions, when executed by the processor, implement the steps of the shooting method as described in the first aspect.

[0015] Fourthly, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the shooting method as described in the first aspect.

[0016] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the shooting method as described in the first aspect.

[0017] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the shooting method as described in the first aspect.

[0018] In this embodiment, the dynamic range of the scene corresponding to the current shooting scene is obtained; based on the scene dynamic range, a target exposure mode that conforms to the scene dynamic range is determined from at least two exposure modes; and a high dynamic range image is obtained by shooting in the current shooting scene based on the target exposure mode. Thus, since a single exposure mode indicates that at least two frames are acquired through different gains during a single exposure, compared to the operation using multiple exposure modes in related technologies, a single exposure has no exposure time difference, i.e., no frame difference. Furthermore, since the target exposure mode includes both multiple exposure modes and single exposure modes, shooting in the current shooting scene based on the target exposure mode combines the advantages of both multiple exposure modes and single exposure modes, improving the dynamic range of the camera device in the current shooting scene. Moreover, while meeting the scene dynamic range requirements, it avoids motion blur caused by frame differences, thereby improving image quality. Attached Figure Description

[0019] Figure 1 This is a flowchart of the shooting method provided in the embodiments of this application;

[0020] Figure 2 This is a flowchart illustrating the automatic exposure statistics provided in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the AE processing flow provided in the embodiments of this application;

[0022] Figure 4 This is a schematic diagram of the ISP image processing flow provided in the embodiments of this application;

[0023] Figure 5 This is a system block diagram of the high dynamic range mode provided in the embodiments of this application;

[0024] Figure 6 This is a schematic diagram illustrating the principle of multiple exposure modes in related technologies;

[0025] Figure 7 These are signal-to-noise ratio curves for various high dynamic range modes provided in the embodiments of this application;

[0026] Figure 8 This is a schematic diagram of the HDR mode region of the image sensor provided in the embodiments of this application;

[0027] Figure 9 This is a signal processing flowchart of the multiple exposure combined with single exposure mode provided in the embodiments of this application;

[0028] Figure 10 This is a schematic diagram of the circuit structure of the dual analog gain mode provided in the embodiments of this application;

[0029] Figure 11This is a schematic diagram of the counting principle of the counter provided in the embodiments of this application;

[0030] Figure 12 This is a timing diagram of the comparator and counter provided in an embodiment of this application;

[0031] Figure 13 This is a block diagram of the imaging device provided in the embodiments of this application;

[0032] Figure 14 This is one of the structural schematic diagrams of the electronic device provided in the embodiments of this application;

[0033] Figure 15 This is the second schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0035] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0036] In related technologies, the compact camera module (CCM) is currently the mainstream lens module used in mobile phones. A CCM mainly consists of a lens, a voice coil motor (VCM), an infrared filter (IR filter), a CMOS image sensor (CIS), a digital signal processor (DSP), and a flexible printed circuit board (FPC). The CCM's workflow is as follows: the voice coil motor drives the lens to a precise focusing position. External light passes through the lens, and after the infrared filter removes unwanted infrared light, visible light is focused by pixel-level microlenses. Then, it passes through RGB color filters and illuminates different photodiodes (PDs) on the image sensor. The photodiodes convert the sensed light signals into electrical signals. These signals are then amplified and converted into a digital signal matrix (i.e., an image) by an analog-to-digital converter (ADC). Finally, the image is processed by the DSP, compressed, and stored.

[0037] The shooting method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0038] Figure 1 This is a flowchart of the shooting method provided in the embodiments of this application, such as... Figure 1 As shown, the method includes:

[0039] Step 101: Obtain the scene dynamic range corresponding to the current shooting scene.

[0040] In this embodiment, at least one original image can be obtained by capturing the current shooting scene without activating the high dynamic range mode. The camera device can be a camera, mobile phone, tablet computer, etc., and this embodiment does not limit its use. High dynamic range mode refers to a shooting mode using high dynamic range imaging (HDR) technology. Specifically, it can include a multiple exposure mode using multiple exposure HDR technology and a single exposure mode using single exposure HDR technology. The original image has not undergone image processing or compression, so it retains the most original image data information captured by the camera device. The single exposure mode can be a dual conversion gain (DCG) mode or a dual analog gain (DAG) mode; these are merely illustrative examples, and this embodiment does not limit their use.

[0041] Understandably, the dynamic range (DR) of the real world is 10. -6~10 9 candela per square meter (cd / m) 2 The dynamic range of human vision is 1:10. 5 (cd / m 2 In actual shooting scenarios, the dynamic range of human vision is more than ten times that of a camera device. The latitude of a camera device refers to the maximum number of stops of overexposure or underexposure it can tolerate when exposure errors occur, while still being able to obtain an acceptable (usable) image from the photosensitive material. For example, the latitude of color negative film can be up to 4, corresponding to a dynamic range of 80 dB, while the latitude of a camera can be up to 15, corresponding to a dynamic range of 90 dB.

[0042] Table 1 Comparison of Brightness and Dynamic Range

[0043]

[0044] As shown in Table 1, cd / m 2 It is a unit of brightness. The unit of dynamic range (DR) is dB. F-stop represents the ratio of the lens focal length f to the aperture diameter D of the camera device. EV represents the exposure value (EV). The image processing software's data processing bit depth of 16 bits means that it can process up to 16 bits of image data. The image processing software's data processing bit depth of 12 bits means that it can process up to 12 bits of image data.

[0045] The formula for calculating dynamic range (DR) is shown in the following formula (1):

[0046] DR = 20log 10 (Maximum brightness / Minimum brightness) (1)

[0047] According to formula (1), the dynamic range corresponding to image processing and mobile phone display can be calculated. In order to meet the requirement of "what you see is what you get" and allow the camera to capture the same scene as the human eye, the camera needs to activate the high dynamic range mode to meet the scene's dynamic range.

[0048] In this embodiment, the single-frame exposure parameters corresponding to the current shooting scene can be determined based on the original image. Then, the maximum and minimum auto exposure (AE) parameters corresponding to the current shooting scene can be determined based on the single-frame exposure parameters. The methods for obtaining the single-frame exposure parameters, maximum AE parameters, and minimum AE parameters can be found in related technologies, and this embodiment does not limit these methods. For example, the maximum and minimum AE parameters can be obtained by performing auto exposure (AE) statistics on the original image using an image signal processor (ISP). This is merely an example, and this embodiment does not limit these methods.

[0049] The Image Signal Processor (ISP) is used to perform 3A calculations and feedback on the image signal output from the image sensor, as well as raw image preprocessing, depigmentation, color correction and enhancement, noise reduction, color conversion, multi-frame synthesis, brightness mapping, and detail enhancement. The 3A calculations include: Auto White Balance (AWB), Auto Exposure (AE), and Auto Focus (AF). In the 3A calculations, the AF calculation result determines the image sharpness, the AWB calculation result determines whether the image has color cast, and the AE calculation result determines whether the exposure is correct.

[0050] In this embodiment, after determining the single-frame exposure parameters corresponding to the current shooting scene based on the original image, it is possible to determine whether the current shooting scene is overexposed or underexposed based on the single-frame exposure parameters. If overexposure or underexposedness exists, the high dynamic range (HDR) mode of the camera device needs to be enabled to perform high dynamic range shooting in the current shooting scene. Overexposure refers to a situation where the brighter areas in the original image lack detail or gradation in an overly bright environment, while underexposedness refers to a situation where the original image is too dark to accurately reflect the colors of objects in the current shooting scene.

[0051] Specifically, the Image Signal Processor (ISP) processes the raw image output from the image sensor. By determining the sharpness or phase difference of the raw image, the ISP feeds back to control the camera's lens and motor, enabling the camera to focus and achieve sharpness—this is autofocus (AF). The ISP also performs AE (Automatic Exposure) statistics on the raw image output from the image sensor and feeds back to control the image sensor to modify the shutter / gain registers, achieving accurate exposure—this is automatic exposure (AE). Figure 2As shown, the ISP performs automatic exposure statistics (AE stats) to determine if the current shooting scene is overexposed. If overexposed, the ISP feeds back to control the image sensor to reduce the AE parameters until it is no longer overexposed and records the minimum AE parameter. The ISP then determines if the current shooting scene is underexposed. If underexposed, the ISP feeds back to control the image sensor to increase the AE parameters until it is no longer underexposed and records the maximum AE parameter. The AE processing flow of the ISP feedback control of the image sensor includes: Figure 3 As shown, the initial exposure parameters (AEC) statistical index of automatic exposure are transmitted to the image sensor. AE statistics are performed based on the raw image (raw image) output by the image sensor, light is measured and target brightness is calculated, convergence is determined, the exposure parameters for the next frame are calculated, and the exposure parameters to eliminate water ripples are obtained and returned to the image sensor.

[0052] Furthermore, the dynamic range of the image sensor of the camera device is obtained, and the scene dynamic range corresponding to the current shooting scene is calculated based on the maximum AE parameter, the minimum AE parameter, and the dynamic range of the image sensor. For example, if the dynamic range of the image sensor is 48dB, the maximum AE parameter is 5x gain and 500ms exposure time, and the minimum AE parameter is 1x gain and 1ms exposure time, then the scene dynamic range corresponding to the current shooting scene is as shown in the following formula (2):

[0053]

[0054] The maximum value of the dynamic range can be calculated according to formula (2), and the minimum value of the dynamic range is 0dB. The range corresponding to the maximum value and the minimum value of the dynamic range is then determined as the scene dynamic range.

[0055] Step 102: Based on the scene dynamic range, determine a target exposure mode that conforms to the scene dynamic range from at least two exposure modes; the at least two exposure modes include a multiple exposure mode and at least one single exposure mode, wherein the single exposure mode indicates that at least two frames of images are acquired through different gains during a single exposure, and the multiple exposure mode indicates that at least two frames of images are acquired by controlling the exposure time.

[0056] In this embodiment, the multiple exposure mode refers to adjusting the exposure time of the photodiode (PD) by controlling the timing of the pixel transistors of the image sensor of the camera device. Shorter exposure times can be used for brighter areas to avoid overexposure, while longer exposure times can be used for darker areas to improve the clarity of dark areas. The multiple exposure mode can be set to two, three, or even more exposures to enhance the dynamic range of the image and its flexibility in application scenarios. This is merely an example, and this embodiment does not impose any limitations.

[0057] In this embodiment, at least one single-exposure mode may include a dual conversion gain (DCG) mode and a dual analog gain (DAG) mode. The dual conversion gain (DCG) includes high conversion gain (HCG) and low conversion gain (LCG). The camera acquires an image signal through a single exposure, and then uses the capacitor (C1) of the floating node (FD) to achieve a high pixel voltage conversion gain (HCG), obtaining a frame image corresponding to HCG. This results in lower readout noise, improves the light sensitivity of darker areas, and contributes to the image quality in darker areas. Conversely, two capacitors (C1 and C2) are used to reduce the pixel voltage conversion gain (LCG), obtaining a frame image corresponding to LCG. This allows the FD to have greater charge storage capacity, preventing overexposure of pixels in brighter areas and preserving more image details. The gains of HCG and LCG can be set to any ratio, such as 1:2, 1:4, or 1:8, etc., which are merely illustrative examples and are not limited in this embodiment.

[0058] In this embodiment, the Dual Analog Gain (DAG) mode refers to using the analog-to-digital converter (ADC) circuit of the image sensor to take the analog image signal from the pixel module of the image sensor as the input signal of the ADC circuit. Two different ramp generators are used to generate two different readout slopes, AG-1 and AG-2. The same analog image signal will be read by the ADC circuit according to the slope of AG-1 to obtain the value of "output 1", and then read according to the slope of AG-2 to obtain the value of "output 2", thus obtaining two gain image digital signals. Then, two frames of high dynamic range images are generated based on the two gain image digital signals, thereby expanding the dynamic range of the image.

[0059] In one feasible implementation, the analog-to-digital signal conversion circuit of the image sensor can be a column parallel analog-to-digital converter (ADC). The ADC reads the input signal from the previous stage twice with different analog gains and outputs two frames of digital signals with different gains. The analog gain used for the two reads by the ADC can be any gain value, such as 1 to 64 times. That is, the different gain values ​​used for the two reads of the DAG can be arbitrarily set to any analog gain value between 1 and 64 times according to the requirements of the scene. For example, AG-1 uses 1 times analog gain and AG-2 uses 4.5 times analog gain. This is just an example and the embodiments of this application do not limit it.

[0060] In one feasible implementation, at least one single-exposure mode may include a multi-conversion gain (MCG) mode and a multi-analog gain (MAG) mode. The MCG mode may be a triple conversion gain (TCG) mode, which includes HCG, LCG, and medium conversion gain (Medium CG, MCG). The TCG mode can further enhance the dynamic range of the image. The MAG mode can use more analog gain to read the analog signal from the image sensor to obtain the corresponding digital signal, thereby obtaining multiple frames of high dynamic range images and enhancing the dynamic range of the image. This is merely an illustrative example, and the embodiments of this application are not intended to limit the scope of the invention.

[0061] In this embodiment, the dynamic range corresponding to the multiple exposure mode and the dynamic range corresponding to at least one single exposure mode can be predetermined. Then, based on the scene dynamic range of the current shooting scene, a combination of modes whose summed dynamic ranges satisfy the scene dynamic range is selected from the multiple exposure modes and at least one single exposure mode, and this combination is determined as the target exposure mode corresponding to the current shooting scene. Specifically, from different combination of modes whose summed dynamic ranges are close to the scene dynamic range, the combination with the smallest frame difference and the least noise can be selected as the target exposure mode corresponding to the current shooting scene. This allows for obtaining a higher quality high dynamic range image while meeting the scene dynamic range requirements.

[0062] The dynamic range corresponding to the multiple exposure mode and the dynamic range corresponding to at least one single exposure mode can be determined by pre-capturing common shooting scenes using a camera device in either multiple exposure mode or at least one single exposure mode, acquiring original images corresponding to different modes, determining the dynamic range corresponding to each mode based on the original images, and storing these images. In practical applications, after obtaining the scene dynamic range corresponding to the current shooting scene, it can be directly calculated based on the scene dynamic range and the pre-stored dynamic ranges corresponding to the multiple exposure mode and at least one single exposure mode. The combination of modes whose dynamic ranges, when added together, satisfy the scene dynamic range is selected as the target exposure mode for the current shooting scene. This improves the efficiency of obtaining the target exposure mode.

[0063] Alternatively, in another feasible implementation, for the current shooting scene, the camera device can be used to capture images in the current shooting scene using multiple exposure modes and at least one single exposure mode to obtain the original images corresponding to different exposure modes in the current shooting scene. Then, based on the original images in the current shooting scene, the dynamic range corresponding to the multiple exposure mode and the dynamic range corresponding to each of the at least one single exposure mode are determined. Then, based on the scene dynamic range of the current shooting scene, a combination of modes whose dynamic ranges can be added together to satisfy the scene dynamic range is selected from the multiple exposure modes and at least one single exposure mode to determine the target exposure mode corresponding to the current shooting scene. In this way, the target exposure mode can be better matched with the current shooting scene, thereby improving the image quality of the high dynamic range image obtained by shooting in the current shooting scene based on the target exposure mode.

[0064] Step 103: Take a picture in the current shooting scene based on the target exposure mode to obtain a high dynamic range image; the target exposure mode includes at least one of the multiple exposure mode and the single exposure mode.

[0065] In this embodiment of the application, the high dynamic range (HDR) function of the image sensor of the camera device can be activated, and the target exposure mode can be used as the HDR mode of the image sensor. Then, the current shooting scene is captured to obtain the image signal output by the image sensor. By processing the image signal, a high dynamic range image corresponding to the current shooting scene is generated.

[0066] Specifically, image signals can be processed by an Application Processor (AP) chip to synthesize high dynamic range images. The AP chip may include an Image Signal Processor (ISP), which processes the image signals. (See [link to relevant documentation]). Figure 4 The image signal output by the image sensor is sent to the image signal processor (ISP). The ISP performs Bayer array processing, automatic exposure (AE) statistics, color interpolation, global tone mapping (GTM) algorithm for color correction matrix, gamma correction, denoising (luminance) NR (luma) processing, basic color space processing (YUV420 format), warping, denoising and multi-frame noise reduction processing (NR & Multi-frame Proc), local tone mapping (LTM) algorithm processing, detail edge enhancement processing, and image enhancement / upscale processing.

[0067] In this embodiment, the AP chip can synthesize a high dynamic range image based on the image processed by the ISP, such as an image in the Joint Photographic Experts Group (JPEG) format, and then send it to the display device of the camera device for user preview, or send it to the storage device of the camera device for local storage.

[0068] In this embodiment, a dedicated high dynamic range (HDR) processing chip can be set in the camera device to perform HDR calculations, that is, based on the scene's dynamic range, a target exposure mode that conforms to the scene's dynamic range is determined from multiple exposure modes and at least one single exposure mode. For example... Figure 5 As shown, the image sensor sends image signals to the AP chip according to the Mobile Industry Processor Interface (MIPI) transmission protocol. The AP chip includes firmware, Dynamic Random Access Memory (DRAM), and an Image Signal Processor (ISP). The ISP performs AE calculations and feeds back AE information (AE info) from the image sensor via an Inter-Integrated Circuit (IIC) bus or a Serial Peripheral Interface (SPI) bus. It also feeds back AF information (AF info) via Pulse Width Modulation (PWM) to control the camera lens for autofocus. After the AP chip synthesizes the original image, it sends the original image (RGB) and scene dynamic range to the image processor (HDR) chip. Based on the scene dynamic range, the HDR chip determines the target exposure mode that matches the scene dynamic range from multiple exposure modes and at least one single exposure mode, and feeds back a dynamic range scaling table (HDRmap) to the image sensor, allowing the image sensor to set its HDR mode according to the HDR map. The image processor (HDR) chip also feeds back to the application processing unit (AP) chip to drive the image sensor to perform the exposure action corresponding to the target exposure mode, that is, to shoot according to the target exposure mode in the current shooting scene. The image processor (HDR) chip can also output a preview image to the camera device for screen preview, allowing users to view the original image.

[0069] In this embodiment, the dynamic range of the scene corresponding to the current shooting scene is obtained; based on the scene dynamic range, a target exposure mode that conforms to the scene dynamic range is determined from at least two exposure modes; and a high dynamic range image is obtained by shooting in the current shooting scene based on the target exposure mode. In this way, since the single exposure mode indicates that at least two frames are acquired through different gains during a single exposure, compared to the operation using multiple exposure modes in related technologies, a single exposure has no exposure time difference, i.e., no frame difference. Furthermore, since the target exposure mode includes both multiple exposure modes and single exposure modes, shooting in the current shooting scene based on the target exposure mode can combine the advantages of both multiple exposure modes and single exposure modes, improving the dynamic range of the camera device in the current shooting scene. Moreover, while meeting the scene dynamic range requirements, it avoids motion blur caused by frame differences, thereby improving image quality.

[0070] Figure 6 This is a schematic diagram illustrating the principle of multiple exposure modes in related technologies, such as... Figure 6 As shown, in multi-exposure mode, the camera device fuses multiple frames with different exposure times to obtain an image with more detail in both bright and dark areas, thereby expanding the high dynamic range of the image. However, multi-frame fusion can cause motion blur for moving objects, requiring subsequent complex deblurring algorithms. Short exposure frames are used as the reference frames for moving objects to perform motion object recognition, segmentation, and deblurring, while long exposure frames are used to extract details from darker areas (low-light regions). However, this method produces worse image quality for moving objects in low-light scenes and is not suitable for all shooting scenarios. Currently, a 1:16 ratio of long to short exposure times is commonly used. While a larger exposure ratio yields a higher dynamic range, it can cause discontinuities in the signal after multi-frame fusion, resulting in noticeable image stitching lines and poor image quality in the synthesized image. The shooting method in this embodiment obtains the scene dynamic range corresponding to the current shooting scene.

[0071] Based on the scene dynamic range, a target exposure mode that matches the scene dynamic range is determined from at least two exposure modes. Then, the current shooting scene is shot based on the target exposure mode. The resulting high dynamic range image is more closely matched to the current shooting scene. Furthermore, by combining multiple exposure modes and at least one single exposure mode, the dynamic range of the image is expanded. The HDR mode combination can be freely switched according to the needs of different shooting scenes, while reducing motion blur.

[0072] In some embodiments, the method further includes:

[0073] Step 201: Obtain the original images under the at least two exposure modes.

[0074] In this embodiment, the original images under at least two exposure modes include original images under multiple exposure modes and original images under at least one single exposure mode. The original image under multiple exposure modes can be an image obtained in advance by capturing any shooting scene in multiple exposure mode using a camera device that supports multiple exposure mode. The original image under any single exposure mode can be an image obtained in advance by capturing any shooting scene in single exposure mode using a camera device that supports that single exposure mode.

[0075] In another feasible implementation, for the current shooting scene, the high dynamic range mode of the camera device can be activated. Specifically, the image sensor of the camera device can be set to multiple exposure mode to capture images in the current shooting scene, obtaining the original image in multiple exposure mode. Additionally, the image sensor can be set to individual single exposure modes and captured images in the current shooting scene to obtain the original image in each single exposure mode. The image sensor supports both multiple exposure mode and individual exposure modes.

[0076] Step 202: Determine the signal-to-noise ratio curves corresponding to the at least two exposure modes based on the original image.

[0077] In this embodiment, for the multiple exposure mode, the original images corresponding to different exposure ratios in the multiple exposure mode are obtained. Based on the original images corresponding to different exposure ratios, the read noise value (RN) corresponding to different exposure ratios is determined. Then, the full well capacity (FWC) of the pixels corresponding to the multiple exposure mode is obtained. The signal-to-noise ratio (SNR) corresponding to different exposure ratios is calculated according to the signal-to-noise ratio (SNR) formula, as shown in the following formula (3). Wherein, the exposure ratio refers to the preset brightness value, for example, when the exposure ratio is 0.4%, the brightness value is 1 lux, 0.8% corresponds to 2 lux, 1.6% corresponds to 4 lux, etc. The specific implementation method of determining the read noise value (RN) corresponding to different exposure ratios based on the original images corresponding to different exposure ratios in the multiple exposure mode can refer to related technologies, and this embodiment does not limit it.

[0078]

[0079] Specifically, an exposure ratio range, such as 0.4% to 6400%, can be preset. Then, based on the multiple exposure mode, original images corresponding to different exposure ratios within the exposure ratio range are acquired. The corresponding readout noise value (RN) is then determined based on the original images corresponding to different exposure ratios. Finally, the signal-to-noise ratio (SNR) corresponding to different exposure ratios is calculated using formula (3), thus obtaining the SNR range corresponding to the exposure ratio range. Furthermore, an SNR curve in the multiple exposure mode can be generated based on the exposure ratio range and the SNR range.

[0080] For any single exposure mode, such as DCG mode, the original image corresponding to different exposure ratios in the single exposure mode can be obtained according to the preset exposure ratio range. Based on the original image corresponding to different exposure ratios, the readout noise value (RN) corresponding to different exposure ratios is determined. Then, the full well capacity (FWC) of the pixel corresponding to the single exposure mode is obtained. The signal-to-noise ratio corresponding to different exposure ratios is calculated according to formula (3) to obtain the signal-to-noise ratio range corresponding to the exposure ratio range. Furthermore, the signal-to-noise ratio curve corresponding to the single exposure mode can be generated based on the exposure ratio range and the signal-to-noise ratio range.

[0081] For example, Table 2 is a comparison table of exposure ratio range and signal-to-noise ratio range. The preset exposure ratio range is 0.4% to 6400%, and at least two exposure modes are included: multiple exposure mode (triple exposure 3stag 4x), dual conversion gain mode (DCG 32x), and dual analog gain mode (DAG 4x). The signal-to-noise ratios corresponding to different exposure ratios under each mode are shown in Table 2. Here, x represents the relationship between the gain and illuminance (ISO) of the HDR mode, and the value of x is 100. For example, 32x corresponds to ISO 3200.

[0082] Furthermore, the signal-to-noise ratio curves for each high dynamic range mode can be generated by referring to Table 2, such as... Figure 7 As shown.

[0083] Table 2 Comparison of Exposure Ratio Range and Signal-to-Noise Ratio Range

[0084] 0.4％ 0.8％ 1.6％ 3.1％ 6.3％ 12.5％ DAG (4x) -0.4211 3.9420 7.8287 11.3539 14.6466 17.8052 0.4％ 0.8％ 1.6％ 3.1％ 6.3％ 12.5％ DCG (32x) 3.9420 7.8287 11.3539 14.6466 17.8052 20.8916 0.4％ 0.8％ 1.6％ 3.1％ 6.3％ 12.5％ 3stag (4x) 3.9420 7.8287 11.3539 14.6466 17.8052 20.8916 25.0％ 50.0％ 100.0％ 200.0％ 400.0％ 800.0％ DAG (4x) 20.8916 23.9405 26.9702 29.9902 33.0054 36.0182 25.0％ 50.0％ 100.0％ 100.0％ 200.0％ 400.0％ DCG (32x) 23.9405 26.9702 29.9902 29.8464 32.9328 35.9817 25.0％ 50.0％ 100.0％ 200.0％ 400.0％ 800.0％ 3stag (4x) 23.9405 26.9702 29.9902 33.0054 36.0182 39.0297 800.0％ 1600.0％ 3200.0％ 6400.0％ DAG (4x) 23.9405 26.9702 29.9902 32.9328 800.0％ 1600.0％ 3200.0％ 3200.0％ 6400.0％ DCG (32x) 39.0114 42.0314 45.0466 42.0314 45.0466 800.0％ 1600.0％ 3200.0％ 6400.0％ 3stag (4x) 29.9902 33.0054 36.0182 39.0297

[0085] See Figure 7 The horizontal axis represents the preset exposure ratio range (0.4% to 6400%), and the vertical axis represents the signal-to-noise ratio corresponding to different exposure ratios in each high dynamic range mode, in dB.

[0086] In some embodiments, step 102 may include the following steps:

[0087] Step 1021: Determine the target exposure mode that conforms to the scene dynamic range based on the scene dynamic range and the signal-to-noise ratio curves corresponding to the at least two exposure modes.

[0088] In related technologies, the ratio between pixel full-well capacity and readout noise value determines the dynamic range that an image sensor can cover. Pixel full-well capacity (FWC) represents the maximum number of electrons that each pixel of the image sensor can receive, i.e., the maximum brightness. Readout noise value (RN) represents the noise generated when there is no light signal and the shortest exposure time is used, i.e., the minimum brightness.

[0089] In this embodiment, formula (3), the signal-to-noise ratio (SNR) calculation formula, includes the ratio of full-well capacity (FWC) to readout noise (RN), while formula (1), the dynamic range (VR) calculation formula, includes the ratio of maximum brightness to minimum brightness. Since the units for SNR and VR are the same (dB), the dynamic range can be approximated by SNR. Based on the scene dynamic range of the current shooting scene and the SNR curves corresponding to at least two exposure modes, and based on the exposure ratio range corresponding to the current shooting scene, a combination of modes whose SNR satisfies the scene dynamic range of the current shooting scene is selected from multiple exposure modes and at least one single exposure mode. The selected combination of modes is then determined as the target exposure mode corresponding to the current shooting scene. Specifically, as shown... Figure 7 As shown, with the horizontal axis representing an exposure ratio of 800%, both the signal-to-noise ratio (SNR) curves for the 3stag (4x) mode and the DAG (4x) mode show a downward trend. This indicates that beyond an exposure ratio of 800%, the SNR performance of both the 3stag and DAG modes is not good. To achieve better shooting results while meeting the scene dynamic range requirements of the current shooting scenario, it is necessary to combine multiple HDR modes. (See [link to relevant documentation]). Figure 7 The target exposure mode can be determined by combining the advantages of each HDR mode under different exposure ratios, based on the signal-to-noise ratio curves corresponding to each HDR mode. For example, for an exposure ratio of 1600%, the signal-to-noise ratio curve of the 3stag mode drops, while the signal-to-noise ratio curve of the DCG mode is still in a linear upward phase. Therefore, combining the high gain (maximum 32x) of the 3stag mode and the excellent signal-to-noise ratio performance of the DCG mode, the 3stag mode and the DCG mode are determined as the target exposure modes. This improves the flexibility of target exposure mode selection, allowing the selection of the optimal target exposure mode based on the current shooting scene and the user's shooting needs, thus enhancing the practicality of the shooting method in this embodiment.

[0090] In some embodiments, for certain shooting scenarios, a single high dynamic range (HDR) mode can cover the scene's dynamic range. Therefore, an HDR mode can be selected from multiple exposure modes or individual exposure modes, and the image is shot in that scenario using this single HDR mode to obtain a HDR image. For example, when a user takes a photo during the day, the camera's ISO can be 100. If the brightness range (exposure ratio range) is 0.4% to 400%, then for better HDR results, multiple exposure modes and at least one individual exposure mode, such as dual conversion gain (DCG) mode, are needed together. See also Figure 7 Within the exposure range of 0.4% to 400%, the signal-to-noise ratio (SNR) curves of the 3stag, DCG, and DAG modes all exhibit a linear upward trend. This allows for flexible selection of mode combinations as the target exposure mode based on the scene's dynamic range and the individual SNR curves. For example, at any exposure ratio within the 0.4% to 400% range, the mode with the highest SNR value can be directly selected as the target exposure mode for the current shooting scene. When taking photos at night, the camera's ISO can be set to 3200, allowing the camera to choose DCG mode to satisfy the scene's dynamic range requirements while avoiding the frame difference issues associated with multiple exposure modes.

[0091] In this embodiment, original images under at least two exposure modes are acquired; based on the original images, signal-to-noise ratio (SNR) curves corresponding to the at least two exposure modes are determined; and based on the scene dynamic range and the SNR curves corresponding to the at least two exposure modes, a target exposure mode conforming to the scene dynamic range is determined. This allows for convenient determination of the target exposure mode based on the SNR curves corresponding to at least two exposure modes, improving the efficiency of target exposure mode determination and thus enhancing the shooting efficiency of the shooting method in this embodiment.

[0092] In some embodiments, step 201 may include the following steps:

[0093] Step 2011: Obtain the first original image corresponding to the multiple exposure mode and the second original image corresponding to the at least one single exposure mode.

[0094] In this embodiment of the application, the image sensor of the camera device can be a sensor that simultaneously has multiple exposure modes and at least one single exposure mode. The pixels on the image sensor are not partitioned, and the image sensor can be controlled to switch between multiple exposure modes and each single exposure mode. The image is captured in the current shooting scene according to the multiple exposure mode and each single exposure mode respectively, to obtain a first original image corresponding to the multiple exposure mode and a second original image corresponding to at least one single exposure mode.

[0095] In some embodiments, step 202 may include the following steps:

[0096] In 2021, based on the first original image, the signal-to-noise ratio curve corresponding to the multiple exposure mode was determined.

[0097] In this embodiment, first original images corresponding to different exposure ratios under multiple exposure mode can be obtained according to a pre-set exposure ratio range. Readout noise values ​​(RN) corresponding to different exposure ratios are determined based on the first original images corresponding to different exposure ratios. Then, the signal-to-noise ratio (SNR) under different exposure ratios is calculated according to the pixel full-well capacity (FWC) corresponding to the multiple exposure mode and formula (3), thus obtaining the SNR range corresponding to the multiple exposure mode. Further, an SNR curve corresponding to the multiple exposure mode can be generated based on the exposure ratio range and the SNR range.

[0098] 2022, based on the second original image corresponding to the at least one single exposure mode, determine the signal-to-noise ratio curve corresponding to the at least one single exposure mode.

[0099] In this embodiment, for any single exposure mode, a second original image corresponding to different exposure ratios under the single exposure mode can be obtained according to a pre-set exposure ratio range. The readout noise value (RN) corresponding to different exposure ratios under the single exposure mode is determined based on the second original image. Then, the signal-to-noise ratio (SNR) under different exposure ratios is calculated according to the pixel full-well capacity (FWC) corresponding to the single exposure mode and formula (3), thus obtaining the SNR range corresponding to the single exposure mode. Further, an SNR curve corresponding to the single exposure mode can be generated based on the exposure ratio range and the SNR range.

[0100] In this embodiment, by acquiring a first original image corresponding to the multiple exposure mode and a second original image corresponding to the at least one single exposure mode; and by determining the signal-to-noise ratio (SNR) curve corresponding to the multiple exposure mode based on the first original image, the determined SNR curve can better match the multiple exposure mode; similarly, by determining the SNR curve corresponding to the at least one single exposure mode based on the second original image corresponding to the at least one single exposure mode, the determined SNR curve can better match the corresponding single exposure mode. This makes the target exposure mode determined based on the SNR curve more accurate, and to some extent improves the image quality of high dynamic range images obtained by shooting according to the target exposure mode.

[0101] In some embodiments, step 103 may include the following steps:

[0102] Step 1031: Take a picture in the current shooting scene based on the multiple exposure mode to obtain at least two first images.

[0103] In this embodiment, the multiple exposure mode can be a two-exposure mode, a three-exposure (3Stagger) mode, etc. The two-exposure mode includes a long exposure mode and a short exposure mode with different exposure lengths, while the three-exposure mode includes a long exposure mode, a medium exposure mode, and a short exposure mode with different exposure lengths. For the three-exposure mode, the camera device can use different shutter speeds to perform three exposures, obtaining images corresponding to the long exposure mode, medium exposure mode, and short exposure mode respectively, thus obtaining three first frames.

[0104] Step 1032: Take a picture in the current shooting scene based on the target single exposure mode in the target exposure mode to obtain at least two frames of second images.

[0105] In this embodiment, the target single exposure mode is a single exposure mode among target exposure modes. For example, the target single exposure mode can be a dual conversion gain (DCG) mode, which includes a high conversion gain (HCG) mode and a low conversion gain (LCG) mode. In the dual conversion gain mode, the camera device can perform a single exposure, then read the analog image signals output by the image sensor pixels using both high conversion gain (HCG) and low conversion gain (LCG), and then generate two second images based on both HCG and LCG. It should be noted that, since the dual conversion gain (DCG) mode is a single exposure, its advantage over multiple exposure modes is the absence of frame difference; however, its disadvantages are poorer dynamic range and lower noise compared to multiple exposure modes.

[0106] Step 1033: Generate the high dynamic range image based on the at least two first images and the at least two second images.

[0107] In this embodiment, at least two frames of a first image and at least two frames of a second image can be preprocessed using an image signal processor (ISP), such as depigmentation, color correction and enhancement, noise reduction, color conversion, brightness mapping, and detail enhancement, to obtain a processed image. Then, an application processing unit (AP) chip synthesizes the ISP-processed image into a multi-frame image to obtain a high dynamic range (HDR) image. Specifically, a preset number of first images and a preset number of second images can be selected from at least two first images, and then the selected first and second images are synthesized to obtain the HDR image. During image synthesis, the first and second images can be cropped according to the optimal HDR mode corresponding to different areas of the current shooting scene, and then the cropped first and second images are synthesized into a HDR image. For example, if a certain area has high brightness and the multiple exposure mode has a large dynamic range, the first image corresponding to the multiple exposure mode can be cropped; if a certain area has low brightness and better signal-to-noise ratio is desired, then the DCG mode is advantageous, and the second image corresponding to the DCG mode can be cropped. Finally, the cropped first and second images are synthesized into a HDR image. In this way, the advantages of each HDR mode can be combined to improve the image quality of high dynamic range images and achieve better shooting results.

[0108] In this embodiment, at least two first images are obtained by shooting the current shooting scene based on the multiple exposure mode; at least two second images are obtained by shooting the current shooting scene based on the target single exposure mode in the target exposure mode; and the high dynamic range image is generated based on the at least two first images and the at least two second images. In this way, a high dynamic range image corresponding to the current shooting scene can be obtained by combining the multiple exposure mode and the target single exposure mode, improving the dynamic range of the camera device for the current shooting scene. Furthermore, while meeting the scene's dynamic range requirements, the combination of multiple exposure mode and single exposure mode avoids the motion blur problem caused by using the multiple exposure mode, thus improving image quality.

[0109] In another feasible implementation, pixels on the image sensor of the camera device can be divided into regions corresponding to different high dynamic range (HDR) modes. For example... Figure 8As shown, the image sensor includes a normal area (non-HDR mode), a multiple exposure mode (stagger) area, and a single exposure mode-double conversion gain (DCG) mode area. When acquiring the original image corresponding to the current shooting scene, the pixels in the normal area of ​​the image sensor can first be used to sense the light across the entire current shooting scene to obtain an image analog signal. Then, the image processor (ISP) performs AE statistics on the image analog signal to determine the overexposed and underexposed areas in the current shooting scene. The operation of the image processor (ISP) performing AE statistics on the image analog signal can be referred to the relevant description in step 101 of the aforementioned embodiment, and will not be repeated here.

[0110] Since multiple exposure modes typically have a large dynamic range, they can be used to expose overexposed areas in the current shooting scene, thus defining the overexposed areas of the current shooting scene as the shooting scene corresponding to the multiple exposure mode. Similarly, since dual conversion gain (DCG) mode has relatively good noise performance in low-light areas, it can be used to shoot underexposed areas in the current shooting scene, thus defining the underexposed areas of the current shooting scene as the shooting scene corresponding to the DCG mode.

[0111] In this embodiment, for the multiple exposure mode, the light-sensitive area corresponding to the multiple exposure mode on the image sensor can be used to sense light in the shooting scene corresponding to the multiple exposure mode, such as the brighter area in the current shooting scene. For other areas of the current shooting scene (excluding overexposed areas), the pixels of the normal area (non-HDR mode) on the image sensor can be used to sense light in the other areas, obtaining at least two first images corresponding to the multiple exposure mode. This is equivalent to sensing light in all areas of the current shooting scene based on the multiple exposure mode without partitioning, obtaining at least two first images. In the case of partitioning, some shooting scenes are sensed using the multiple exposure mode, while other shooting scenes are sensed using the non-HDR mode, obtaining at least two first images.

[0112] In this embodiment, for the target single exposure mode, light can be captured in the shooting scene corresponding to the target single exposure mode, such as a darker area in the current shooting scene, using the photosensitive area corresponding to the target single exposure mode on the image sensor. For other areas of the current shooting scene (excluding the overly dark areas), light can be captured using pixels from the normal area (non-HDR mode) on the image sensor, resulting in at least two frames of second images corresponding to the target single exposure mode. This is equivalent to capturing light in all areas of the current shooting scene based on the target single exposure mode without partitioning, thus obtaining at least two frames of second images. In the case of partitioning, some shooting scenes are captured using the target single exposure mode, while other shooting scenes are captured using a non-HDR mode, resulting in at least two frames of second images.

[0113] In this way, with the image sensor divided into pixel regions, by using regions with different high dynamic range (HDR) modes on the image sensor to sense some overexposed and / or underexposed areas in the current shooting scene, and by using regions with non-HDR modes to sense other areas in the current shooting scene, the frequency of use of image sensor pixels can be reduced, the power consumption of the image sensor and the amount of image data transmitted can be reduced, the system load can be reduced, and the lifespan of the image sensor can be improved to a certain extent.

[0114] In some embodiments, the gain includes conversion gain and analog gain, and step 1032 may include the following steps:

[0115] Step 1032a: Read the exposure output signal of the current shooting scene based on the conversion gain to obtain the first gain image signal.

[0116] Step 1032b: Read the first gain image signal based on the analog gain to obtain the second gain image signal.

[0117] Step 1032c: Based on the second gain image signal, obtain the at least two frames of the second image; wherein the exposure output signal is the output signal under the multiple exposure mode, the conversion gain includes at least two gain values, and the analog gain includes at least two gain values.

[0118] In this embodiment of the application, the exposure output signal can be multiple image signals corresponding to different exposure durations in the multiple exposure mode, or it can be a single image signal corresponding to a single exposure in the non-HDR mode.

[0119] In this embodiment, reading the exposure output signal of the current shooting scene based on the conversion gain is a conversion gain mode, wherein the conversion gain includes at least two gain values. For example, a dual conversion gain (DCG) mode.

[0120] In this embodiment, the camera device can perform multiple exposures in the current shooting scene based on a multiple exposure mode. For any single exposure, different conversion gains, i.e., at least two gain values, such as high conversion gain (HCG) and low conversion gain (LCG), can be used to read the analog image signal output from the image sensor pixels, thus obtaining at least two first gain image signals corresponding to a single exposure.

[0121] In this embodiment, reading the first gain image signal based on analog gain specifically means that when performing analog-to-digital conversion on any first gain image signal, it is read using different analog gains, i.e., at least two gain values, which is the analog gain mode. For any first gain image signal, it can be read based on at least two gains to obtain at least two second gain image signals. Thus, since the conversion gain includes at least two gain values ​​and the analog gain includes at least two gain values, at least two first gain image signals can be obtained, and consequently, at least four second gain image signals can be obtained.

[0122] It should be noted that analog gain modes, such as dual analog gain (DAG) mode, have the advantage of no frame difference compared to multiple exposure modes, but the disadvantages are that the dynamic range and noise performance are worse than DCG mode.

[0123] In this embodiment, for any second gain image signal, the second gain image signal can be preprocessed by an image signal processor (ISP), and then an image of a preset format can be synthesized by an application processing unit (AP) chip based on the second gain image signal processed by the ISP to obtain a second image frame. Since at least four second gain image signals can be obtained in step 1032b, at least two second images can be obtained accordingly.

[0124] like Figure 9As shown, the multiple exposure mode is the 3Stagger-HDR mode, which includes long exposure, medium exposure, and short exposure. The single exposure mode has the dual conversion gain (DCG) mode and the dual analog gain (DAG) mode. Each frame of image signal of Stagger-HDR can be output to the next stage DCG. DCG outputs image analog signals with superimposed HCG gain signals or LCG gain signals respectively. Each image analog signal at the HCG or LCG end is output to DAG. DAG uses two different gains, such as AG-1 and AG-2, to sample and read the image analog signal respectively. Therefore, each frame of image signal of Stagger-HDR will output four frames of images with different gains after passing through DCG and DAG. That is, the multiple frame image signals of 3Stagger-HDR will have a total of 12 frames of image digital signals with different exposure and gain combinations after passing through DCG and DAG.

[0125] In certain high frame rate scenarios where the DAG functionality is not required, such as... Figure 9 As shown, the DAG can be bypassed, and a normal analog-to-digital converter (ADC) can be used to read the image digital signal. The image digital signal can be compressed and transmitted via a data compression encoder, reducing the load on the transmission interface and increasing the frame rate. Accordingly, the image sensor outputs the compressed image signal to an external chip, such as an AP chip. The external chip needs to be configured with a corresponding data compression decoder to perform compression and decompression, and then the decompressed image signal is input to the AP chip's image signal processor (ISP). Alternatively, the image can be directly output to the IAP for image signal processing without data compression. This is merely an example, and the embodiments of this application do not limit the scope of the invention.

[0126] In this embodiment, a first gain image signal is obtained by reading the exposure output signal of the current shooting scene based on the conversion gain; a second gain image signal is obtained by reading the first gain image signal based on the analog gain; and the at least two frames of the second image are obtained based on the second gain image signal. In this way, the exposure output signal can be amplified by conversion gain and analog gain based on the multiple exposure mode, further improving the dynamic range of the camera device in the current shooting scene. This makes the generated high dynamic range image more closely match the scene dynamic range of the current shooting scene, thereby improving image quality to a certain extent.

[0127] In some embodiments, the analog gain includes a first analog gain value and a second analog gain value, and step 1032b may include the following steps:

[0128] Step 1032b1: Read the first gain image signal based on the first ramp generator to obtain the first read image signal.

[0129] Step 1032b2: Read the first gain image signal based on the second ramp generator to obtain a second read image signal; wherein, the first ramp generator corresponds to the first analog gain value, the second ramp generator corresponds to the second analog gain value, the read slope corresponding to the first ramp generator is different from the read slope corresponding to the second ramp generator, and the second gain image signal includes the first read image signal and the second read image signal.

[0130] In this embodiment, the analog gain includes a first analog gain value and a second analog gain value, so the analog gain mode can be a dual analog gain (DAG) mode. The circuit structure of the DAG mode is as follows: Figure 10 As shown, the circuit on the left includes a multiple exposure mode circuit and a dual conversion gain (DCG) mode circuit. The analog output of the pixel, i.e., the image analog signal, is input to two comparators, and the ramp signals output from two ramp generators with different slopes are input to two comparators. The image analog signal is compared with the two ramp signals with different slopes output from the two ramp generators, and the output of the comparators is the comparison result.

[0131] In related technologies, the voltage of the ramp generator increases linearly. When the voltage rises to a level greater than or equal to the analog image signal, the difference between the start and end times of the ramp generator is the digital image signal. The comparator's output is then converted into a digital image signal by a counter; the counter's function is to convert the time difference into a digital signal. Figure 10 In this diagram, PD represents a photodiode, TX represents a transfer transistor, RST represents a reset transistor, FD represents a floating node, SF represents an amplifier, and RSL represents a row select transistor.

[0132] Step 1032b1: Read the first gain image signal based on the first ramp generator to obtain the first read image signal.

[0133] Step 1032b2: Read the first gain image signal based on the second ramp generator to obtain a second read image signal; wherein, the first ramp generator corresponds to the first analog gain value, the second ramp generator corresponds to the second analog gain value, the read slope corresponding to the first ramp generator is different from the read slope corresponding to the second ramp generator, and the second gain image signal includes the first read image signal and the second read image signal.

[0134] In this embodiment, the reading slope of the first ramp generator can be set to the first analog gain value, and the reading slope of the second ramp generator can be set to the second analog gain value. The first analog gain value and the second analog gain value are different, so the reading slope corresponding to the first ramp generator is different from the reading slope corresponding to the second ramp generator.

[0135] In this embodiment of the application, a first ramp signal from a first ramp generator and a second ramp signal from a second ramp generator are acquired, and the first ramp signal is input into a first comparator, the second ramp signal is input into a second comparator, and the first gain image signal is input into the first comparator and the second comparator, respectively.

[0136] In this embodiment, the comparison result output by the first comparator can be input to a first counter. The first counter counts the difference between the start and end times corresponding to when the first ramp signal is greater than the first gain image signal, i.e., the first time difference, and converts the first time difference into a first readout image signal. Similarly, the comparison result output by the second comparator can be input to a second counter. The second counter counts the difference between the start and end times corresponding to when the second ramp signal is greater than the first gain image signal, i.e., the second time difference, and converts the second time difference into a second readout image signal. Furthermore, the first and second readout image signals can be determined as the second gain image signal.

[0137] Figure 11 This is a schematic diagram of the counting principle of the counter provided in the embodiments of this application, such as... Figure 11 As shown, the horizontal axis represents the signal output time of the ramp generator, the vertical axis represents the voltage value of the image analog signal, AG-1 represents the first analog gain value, i.e. the slope of the first ramp signal, AG-2 represents the second analog gain value, i.e. the slope of the second ramp signal, and the counter output 1 is the first image reading signal, and output 2 is the second image reading signal.

[0138] Figure 12 This is a timing diagram of the comparator and counter provided in an embodiment of this application, as shown below. Figure 12 As shown, the pixels of the image sensor output two types of image analog signals: a pixel reset analog signal A, which includes noise signals such as dark current, and a pixel transmission analog signal B. These signals are then processed by a comparator-ramp generator-counter to obtain the corresponding pixel reset digital signal C and pixel transmission digital signal D. Finally, the actual digital signal output by the pixel is the difference E between C and D. The gain of the counter's output signal is achieved by the slope of the ramp signal generated by the ramp generator.

[0139] In this embodiment, a first read image signal is obtained by reading the first gain image signal based on a first ramp generator; a second read image signal is obtained by reading the first gain image signal based on a second ramp generator. Since the first ramp generator corresponds to the first analog gain value and the second ramp generator corresponds to the second analog gain value, it is convenient to read the first gain image signal using the first ramp generator and the second ramp generator to obtain second gain image signals with different analog values, thereby improving the practicality of the shooting method of this application.

[0140] The shooting method provided in this application can be executed by a shooting device. This application uses a shooting device executing the shooting method as an example to illustrate the shooting device provided in this application.

[0141] See Figure 13 This application provides a shooting device, the device 30 including:

[0142] The first acquisition module 301 is used to acquire the scene dynamic range corresponding to the current shooting scene;

[0143] The first determining module 302 is used to determine a target exposure mode that conforms to the scene dynamic range from at least two exposure modes according to the scene dynamic range; the at least two exposure modes include a multiple exposure mode and at least one single exposure mode, the single exposure mode indicates that at least two frames of images are acquired through different gains in a single exposure process, and the multiple exposure mode indicates that at least two frames of images are acquired by controlling the exposure time.

[0144] The shooting module 303 is used to shoot the current shooting scene based on the target exposure mode to obtain a high dynamic range image; the target exposure mode includes at least one of the multiple exposure mode and the single exposure mode.

[0145] In some embodiments, the device 30 further includes:

[0146] The second acquisition module is used to acquire the original images under the at least two exposure modes;

[0147] The second determining module is used to determine the signal-to-noise ratio curves corresponding to the at least two exposure modes based on the original image.

[0148] The first determining module 302 is specifically used to: determine the target exposure mode that conforms to the scene dynamic range based on the scene dynamic range and the signal-to-noise ratio curves corresponding to the at least two exposure modes.

[0149] In some embodiments, the second acquisition module is specifically used to: acquire a first original image corresponding to the multiple exposure mode and a second original image corresponding to the at least one single exposure mode;

[0150] The second determining module is specifically used for:

[0151] Based on the first original image, determine the signal-to-noise ratio curve corresponding to the multiple exposure mode;

[0152] Based on the second original image corresponding to the at least one single exposure mode, determine the signal-to-noise ratio curve corresponding to the at least one single exposure mode.

[0153] In some embodiments, the shooting module 303 is specifically used for:

[0154] Based on the multiple exposure mode, a shot is taken in the current shooting scene to obtain at least two first images;

[0155] Based on the target single exposure mode in the target exposure mode, the current shooting scene is captured to obtain at least two frames of second images;

[0156] The high dynamic range image is generated based on the at least two first images and the at least two second images.

[0157] In some embodiments, the gain includes conversion gain and analog gain, and the imaging module 303 is further configured to:

[0158] The exposure output signal of the current shooting scene is read based on the conversion gain to obtain a first gain image signal;

[0159] The first gain image signal is read based on the analog gain to obtain the second gain image signal;

[0160] Based on the second gain image signal, the at least two frames of the second image are obtained;

[0161] The exposure output signal is the output signal under the multiple exposure mode, the conversion gain includes at least two gain values, and the analog gain includes at least two gain values.

[0162] In some embodiments, the analog gain includes a first analog gain value and a second analog gain value, and the imaging module 303 is further configured to:

[0163] The first gain image signal is obtained by reading the first gain image signal based on the first ramp generator;

[0164] The second read image signal is obtained by reading the first gain image signal based on the second ramp generator;

[0165] Wherein, the first ramp generator corresponds to the first analog gain value, the second ramp generator corresponds to the second analog gain value, the read slope corresponding to the first ramp generator is different from the read slope corresponding to the second ramp generator, and the second gain image signal includes the first read image signal and the second read image signal.

[0166] The advantages of the shooting device and the shooting method described in the foregoing embodiments over related technologies are the same, and will not be repeated here.

[0167] The shooting device in this application embodiment can be an electronic device or a component of an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. The embodiments of this application do not specifically limit the scope.

[0168] The shooting device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0169] The imaging device provided in this application embodiment can achieve... Figures 1 to 5 , Figures 7 to 12 The various processes implemented in the method embodiments achieve the same technical effect, and will not be described again here to avoid repetition.

[0170] In some embodiments, such as Figure 14As shown, this application embodiment also provides an electronic device M40, including a processor M401 and a memory M402. The memory M402 stores a program or instructions that can run on the processor M401. When the program or instructions are executed by the processor M401, they implement the various steps of the above-described shooting method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0171] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0172] Figure 15 A schematic diagram of the hardware structure of another electronic device to implement an embodiment of this application.

[0173] The electronic device 500 includes, but is not limited to, components such as: radio frequency unit 501, network module 502, audio output unit 503, input unit 504, sensor 505, display unit 506, user input unit 507, interface unit 508, memory 509, and processor 510.

[0174] Those skilled in the art will understand that the electronic device 500 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 510 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 15 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0175] The processor 510 is configured to acquire the scene dynamic range corresponding to the current shooting scene; determine a target exposure mode that conforms to the scene dynamic range from at least two exposure modes based on the scene dynamic range; the at least two exposure modes include a multiple exposure mode and at least one single exposure mode, wherein the single exposure mode indicates acquiring at least two frames of images through different gains during a single exposure, and the multiple exposure mode indicates acquiring at least two frames of images by controlling the exposure time; and shoot the current shooting scene based on the target exposure mode to obtain a high dynamic range image; wherein the target exposure mode includes at least one of the multiple exposure mode and the single exposure mode.

[0176] The processor 510 is further configured to acquire original images under the at least two exposure modes; determine the signal-to-noise ratio curves corresponding to the at least two exposure modes based on the original images; and determine the target exposure mode that conforms to the scene dynamic range based on the scene dynamic range and the signal-to-noise ratio curves corresponding to the at least two exposure modes.

[0177] The processor 510 is further configured to acquire a first original image corresponding to the multiple exposure mode and a second original image corresponding to the at least one single exposure mode; determine the signal-to-noise ratio curve corresponding to the multiple exposure mode based on the first original image; and determine the signal-to-noise ratio curve corresponding to the at least one single exposure mode based on the second original image corresponding to the at least one single exposure mode.

[0178] The processor 510 is further configured to take a picture in the current shooting scene based on the multiple exposure mode to obtain at least two first images; take a picture in the current shooting scene based on the target single exposure mode in the target exposure mode to obtain at least two second images; and generate the high dynamic range image based on the at least two first images and the at least two second images.

[0179] The processor 510 is further configured to read the exposure output signal of the current shooting scene based on the conversion gain to obtain a first gain image signal; read the first gain image signal based on the analog gain to obtain a second gain image signal; and obtain the at least two frames of the second image based on the second gain image signal; wherein the exposure output signal is the output signal under the multiple exposure mode, the conversion gain includes at least two gain values, the analog gain includes at least two gain values, and the gain includes conversion gain and analog gain.

[0180] The processor 510 is further configured to read the first gain image signal based on a first ramp generator to obtain a first read image signal; and to read the first gain image signal based on a second ramp generator to obtain a second read image signal; wherein the first ramp generator corresponds to the first analog gain value, the second ramp generator corresponds to the second analog gain value, the read slope corresponding to the first ramp generator is different from the read slope corresponding to the second ramp generator, the second gain image signal includes the first read image signal and the second read image signal; and the analog gain includes the first analog gain value and the second analog gain value.

[0181] The electronic device and the shooting method described in the foregoing embodiments have the same advantages over related technologies, which will not be repeated here.

[0182] It should be understood that, in this embodiment, the input unit 504 may include a graphics processing unit (GPU) 5041 and a microphone 5042. The GPU 5041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 507 includes at least one of a touch panel 5071 and other input devices 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0183] The memory 509 can be used to store software programs and various data. The memory 509 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 509 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 509 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0184] Processor 510 may include one or more processing units; in some embodiments, processor 510 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the modem processor may also not be integrated into processor 510.

[0185] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described shooting method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0186] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0187] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described shooting method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0188] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0189] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described shooting method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0190] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0191] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0192] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A shooting method, characterized in that, The method includes: Obtain the scene dynamic range corresponding to the current shooting scene; Based on the scene dynamic range, a target exposure mode that conforms to the scene dynamic range is determined from at least two exposure modes; the at least two exposure modes include a multiple exposure mode and at least one single exposure mode, wherein the single exposure mode indicates that at least two frames of images are acquired through different gains during a single exposure, and the multiple exposure mode indicates that at least two frames of images are acquired by controlling the exposure time. A high dynamic range (HDL) image is obtained by shooting in the current shooting scene based on the target exposure mode; the target exposure mode includes at least one of the multiple exposure mode and the single exposure mode; the step of obtaining the HDL image by shooting in the current shooting scene based on the target exposure mode includes: shooting in the current shooting scene based on the multiple exposure mode to obtain at least two first images; shooting in the current shooting scene based on the target single exposure mode in the target exposure mode to obtain at least two second images; and generating the HDL image based on the at least two first images and the at least two second images.

2. The method according to claim 1, characterized in that, The method further includes: Acquire the original images under the at least two exposure modes; Based on the original image, determine the signal-to-noise ratio curves corresponding to the at least two exposure modes respectively; The step of determining a target exposure mode that conforms to the scene dynamic range from at least two exposure modes based on the scene dynamic range includes: Based on the scene dynamic range and the signal-to-noise ratio curves corresponding to the at least two exposure modes, the target exposure mode that conforms to the scene dynamic range is determined.

3. The method according to claim 2, characterized in that, The step of obtaining the original images corresponding to the at least two exposure modes includes: Acquire the first original image corresponding to the multiple exposure mode, and the second original image corresponding to the at least one single exposure mode; The step of determining the signal-to-noise ratio curves corresponding to the at least two exposure modes based on the original image includes: Based on the first original image, determine the signal-to-noise ratio curve corresponding to the multiple exposure mode; Based on the second original image corresponding to the at least one single exposure mode, determine the signal-to-noise ratio curve corresponding to the at least one single exposure mode.

4. The method according to claim 1, characterized in that, The gain includes conversion gain and analog gain. The target single exposure mode based on the target exposure mode is used to capture images in the current shooting scene to obtain at least two frames of the second image, including: The exposure output signal of the current shooting scene is read based on the conversion gain to obtain a first gain image signal; The first gain image signal is read based on the analog gain to obtain the second gain image signal; Based on the second gain image signal, the at least two frames of the second image are obtained; The exposure output signal is the output signal under the multiple exposure mode, the conversion gain includes at least two gain values, and the analog gain includes at least two gain values.

5. The method according to claim 4, characterized in that, The analog gain includes a first analog gain value and a second analog gain value. The step of reading the first gain image signal based on the analog gain to obtain the second gain image signal includes: The first gain image signal is obtained by reading the first gain image signal based on the first ramp generator; The second read image signal is obtained by reading the first gain image signal based on the second ramp generator; Wherein, the first ramp generator corresponds to the first analog gain value, the second ramp generator corresponds to the second analog gain value, the read slope corresponding to the first ramp generator is different from the read slope corresponding to the second ramp generator, and the second gain image signal includes the first read image signal and the second read image signal.

6. A shooting device, characterized in that, The device includes: The first acquisition module is used to acquire the scene dynamic range corresponding to the current shooting scene; The first determining module is configured to determine a target exposure mode that conforms to the scene dynamic range from at least two exposure modes based on the scene dynamic range; the at least two exposure modes include a multiple exposure mode and at least one single exposure mode, the single exposure mode indicating that at least two frames of images are acquired through different gains in a single exposure process, and the multiple exposure mode indicating that at least two frames of images are acquired by controlling the exposure time. A shooting module is configured to capture images in the current shooting scene based on the target exposure mode to obtain a high dynamic range image; the target exposure mode includes at least one of the multiple exposure mode and the single exposure mode; capturing images in the current shooting scene based on the target exposure mode to obtain a high dynamic range image includes: capturing images in the current shooting scene based on the multiple exposure mode to obtain at least two first images; capturing images in the current shooting scene based on the target single exposure mode to obtain at least two second images; and generating the high dynamic range image based on the at least two first images and the at least two second images.

7. The apparatus according to claim 6, characterized in that, The device further includes: The second acquisition module is used to acquire the original images under the at least two exposure modes; The second determining module is used to determine the signal-to-noise ratio curves corresponding to the at least two exposure modes based on the original image. The first determining module is specifically used to: determine the target exposure mode that conforms to the scene dynamic range based on the scene dynamic range and the signal-to-noise ratio curves corresponding to the at least two exposure modes.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the shooting method as described in any one of claims 1-5.

9. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the shooting method as described in any one of claims 1-5.

Images