Video shooting method and electronic device

Abstract

Provided in the present application are a video shooting method and an electronic device. The electronic device can switch to different working modes of an image sensor on the basis of different ambient brightness levels. For example, under the condition that the ambient brightness level is less than a first illuminance value, the electronic device controls the image sensor to work in a low-brightness high-dynamic mode. Under the condition that the ambient brightness level is between the first illuminance value and a second illuminance value, the electronic device controls the image sensor to work in a medium-brightness high-dynamic mode. Under the condition that the ambient brightness level is greater than or equal to the second illuminance value, the electronic device controls the image sensor to work in a high-brightness high-dynamic mode. According to the method, in one aspect, a dynamic range of the electronic device in the video shooting process can be improved. In the other aspect, the video shooting effect is improved by means of the different working modes of the image sensor.

Description

Video shooting method and electronic equipment

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 25, 2023, with application number 202310601145.6 and application name “A Video Shooting Method and Electronic Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of image processing, and in particular to a video shooting method and electronic equipment. Background Art

[0003] The dynamic range of a camera sensor, which refers to its ability to simultaneously capture both highlights and shadows within an image, is limited. When the dynamic range of a captured video scene exceeds the sensor's dynamic range, a high dynamic range (HDR) problem occurs, resulting in an inability to distinguish between shadows and bright areas within the video frame. Further research is needed to address HDR issues encountered during video capture using electronic devices.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a video shooting method and an electronic device, which improve the dynamic range of the electronic device during video shooting and enhance the video shooting effect.

[0006] In a first aspect, the present application provides a video shooting method, an electronic device includes a camera module, the camera module includes an image sensor and a dual conversion gain sensor, the dual conversion gain sensor includes a first sensor and a second sensor, the conversion gain of the first sensor is less than the conversion gain of the second sensor, and the method includes: the electronic device displays a shooting interface, the shooting interface is used to display the image captured by the camera; when the ambient brightness is less than a first illuminance value, the electronic device processes the first original image output by the image sensor into a first image according to the first sensitivity through the second sensor, and processes the first original image into a second image according to the second sensitivity through the second sensor, and the first sensitivity is less than the second sensitivity; the electronic device synthesizes the first image and the second image into a third image, and displays the third image on the shooting interface; when the ambient brightness is between the first illuminance value and the second illuminance value, the electronic device processes the second original image output by the image sensor into a fourth image according to the third sensitivity through the first sensor, and processes the second original image into a fifth image according to the fourth sensitivity through the second sensor, the third sensitivity is less than the fourth sensitivity, and the fourth sensitivity is less than the second sensitivity; the electronic device synthesizes the fourth image and the fifth image into a sixth image, and displays the sixth image on the shooting interface.

[0007] The dual conversion gain sensor may be located inside the image sensor, or may be independent of the image sensor.

[0008] The first sensor may be referred to as a low conversion gain sensor, and the second sensor may be referred to as a high conversion gain sensor.

[0009] Among them, the shooting interface can be a photo shooting preview interface, a video shooting preview interface, or a video shooting interface.

[0010] The first illuminance value and the second illuminance value may be a specific value or a range.

[0011] Through this method, on the one hand, the dynamic range of the electronic device during video shooting can be improved. On the other hand, the electronic device can automatically switch between different shooting modes based on the ambient brightness. Specifically, in low-light scenes, the signal-to-noise ratio of the first sensor is low, and the signal-to-noise ratio of the second sensor is relatively high. By outputting two frames of images through the second sensor to synthesize a video frame, there will be no noise stratification problem, and the imaging effect of the video frame is also good. In medium-light scenes, the signal-to-noise ratio of the first sensor is comparable to that of the second sensor, and the signal-to-noise ratio of the first sensor is also relatively high. By outputting two frames of images through the first sensor and the second sensor to synthesize a video frame, the imaging effect of the video frame is also relatively good.

[0012] In combination with the first aspect, in a possible implementation, the method also includes: when the ambient brightness is greater than or equal to the second illuminance value, the electronic device controls the image sensor to output the third original image with the first exposure time, and controls the image sensor to output the fourth original image with the second exposure time; the electronic device processes the third original image into a seventh image according to the third sensitivity through the first sensor, and processes the third original image into an eighth image according to the fourth sensitivity through the second sensor; the electronic device processes the fourth original image into a ninth image based on a merging method; the electronic device synthesizes the seventh image, the eighth image and the ninth image into a tenth image, and displays the tenth image on the shooting interface.

[0013] In this way, in bright scenes, long-exposure frames (the seventh and eighth images) and a short-exposure frame (the ninth image) are output. The seventh, eighth, and ninth images are then fused to create a single image, the tenth image. This solves the problem of overexposed images caused by overflowing the full well capacity of a single pixel in bright scenes. The overexposed image can be restored using the tenth image.

[0014] In combination with the first aspect, in a possible implementation, the shooting interface also includes a recording end button; the method also includes: the electronic device receives a first operation of the user on the recording end button; in response to the first operation, the electronic device saves a first video, the first video includes the third image and the sixth image, or the first video includes the third image, the sixth image and the tenth image.

[0015] In this way, the method can be applied to video shooting scenarios, and videos can be shot based on the method.

[0016] In combination with the first aspect, in a possible implementation, the shooting interface further includes a video recording start button.

[0017] In this way, the method can be applied to video shooting preview scenarios. In the video shooting preview interface, the electronic device first works in the low-dynamic mode. When it is determined that the dynamic range of the environment is greater than the dynamic range of the image sensor, the electronic device can switch to a high-dynamic mode, such as a low-brightness high-dynamic mode, a medium-brightness high-dynamic mode, or a high-brightness high-dynamic mode. Then, after the electronic device enters the video shooting interface, the electronic device can directly shoot and save the video in the low-brightness high-dynamic mode, the medium-brightness high-dynamic mode, or the high-brightness high-dynamic mode.

[0018] In combination with the first aspect, in a possible implementation, the method further includes: when the ambient brightness is between the first illuminance value and the second illuminance value, the brighter the ambient brightness is, the smaller the sensitivity of the first sensor and the sensitivity of the second sensor are.

[0019] In this way, in low-light scenes, as the environment brightens, the sensitivity of the second sensor that obtains the first image and the sensitivity of the second sensor that obtains the second image can be gradually reduced. As the environment darkens, the sensitivity of the second sensor that obtains the first image and the sensitivity of the second sensor that obtains the second image can be gradually increased. In this way, the video frames captured by the electronic device also gradually brighten or darken, which can improve the video shooting effect.

[0020] In combination with the first aspect, in a possible implementation, the method further includes: when the ambient brightness is between the first illuminance value and the second illuminance value, the brighter the ambient brightness is, the smaller the sensitivity of the first sensor and the sensitivity of the second sensor are.

[0021] In this way, in a medium-bright scene, as the environment becomes brighter, the sensitivity of the first sensor and the second sensor can be gradually reduced. As the environment becomes darker, the sensitivity of the first sensor and the second sensor can be gradually increased. In this way, the video frames captured by the electronic device also gradually brighten or darken, which can improve the video shooting effect.

[0022] With reference to the first aspect, in a possible implementation, exposure durations of the first sensor and the second sensor are both the first exposure duration.

[0023] In this way, during the mode switching process, the exposure time of the first sensor and the second sensor remains unchanged, and only the sensitivity is changed, thereby avoiding flickering between two adjacent video frames and affecting the visual effect.

[0024] With reference to the first aspect, in a possible implementation, a ratio between the sensitivity of the second sensor that obtains the first image and the sensitivity of the second sensor that obtains the second image remains unchanged.

[0025] In this way, in a low-light scene, as the ambient brightness becomes brighter or darker, the sensitivity of the second sensor that obtains the first image and the sensitivity of the second sensor that obtains the second image increase or decrease proportionally, avoiding differences in dynamic range.

[0026] With reference to the first aspect, in a possible implementation, the ratio between the sensitivity of the first sensor and the sensitivity of the second sensor remains unchanged.

[0027] In this way, in a medium-brightness scene, as the ambient brightness becomes brighter or darker, the sensitivity of the first sensor and the second sensor increases or decreases proportionally, avoiding a difference in dynamic range.

[0028] In combination with the first aspect, in a possible implementation, the method further includes: when the ambient brightness is greater than or equal to the second illuminance value, the brighter the ambient brightness is, the longer the first exposure time remains unchanged and the shorter the second exposure time becomes.

[0029] In this way, in a bright scene, as the environment becomes brighter, the exposure time for the ninth image can be gradually reduced. As the environment becomes darker, the exposure time for the ninth image can be gradually increased. In this way, the video frames captured by the electronic device also gradually brighten or darken, which can improve the video shooting effect.

[0030] In combination with the first aspect, in a possible implementation manner, the method further includes: when the ambient brightness is equal to the second illuminance value, the first exposure duration is equal to the second exposure duration.

[0031] In combination with the first aspect, in one possible implementation, the method includes: the electronic device adjusts the sensitivity of the first sensor by adjusting the analog gain and / or digital gain of the first sensor; the electronic device adjusts the sensitivity of the second sensor by adjusting the analog gain and / or digital gain of the second sensor.

[0032] In combination with the first aspect, in a possible implementation, the first sensor also includes a first capacitor, and the electronic device adjusts the sensitivity of the first sensor by adjusting the analog gain and / or digital gain of the first sensor. Specifically, when the first capacitor is a LOFIC capacitor, the electronic device adjusts the sensitivity of the first sensor by adjusting the digital gain of the first sensor; when the first capacitor is not a LOFIC capacitor, the electronic device adjusts the sensitivity of the first sensor by adjusting the analog gain and / or digital gain of the first sensor.

[0033] In a second aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute a method as in the first aspect or any one of the embodiments of the first aspect.

[0034] In a third aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when the computer program product is run on an electronic device, enables the electronic device to execute a method as in the first aspect or any one of the implementations of the first aspect.

[0035] In a fourth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the method of the first aspect or any one of the implementation methods of the first aspect.

[0036] For the description of the beneficial effects in the second to fourth aspects, we can refer to the description of the beneficial effects in the first aspect, and this application will not repeat them here. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figures 1A-1E are a set of UI diagrams provided by this application;

[0038] FIG2 is a schematic diagram of the imaging principle of a DCG image provided by this application;

[0039] FIG3 is a schematic diagram of the dynamic range of a DCG image provided by this application;

[0040] FIG4 shows the signal-to-noise ratios of images obtained by an LCG channel and an HCG channel provided by the present application at different illumination values;

[0041] Figures 5A-5C are another set of UI diagrams provided by this application;

[0042] FIG6 is a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application;

[0043] FIG7 is a schematic diagram of a software structure of an electronic device 100 provided in an embodiment of the present application;

[0044] 8A-8C are schematic diagrams of a set of video preview interfaces provided by the present application;

[0045] FIG8D is a schematic diagram of starting video recording provided by the present application;

[0046] 8E-8F are schematic diagrams of a set of recorded video frames output based on a low-brightness high-dynamic mode according to an embodiment of the present application;

[0047] 8G-8H are schematic diagrams of a set of recorded video frames output based on a medium-brightness high dynamic mode according to an embodiment of the present application;

[0048] FIG8I is a schematic diagram of outputting recorded video frames based on a highlight and high dynamic mode according to an embodiment of the present application;

[0049] FIG9 is a flow chart of a method for displaying preview video frames and recording video frames by an electronic device 100 according to an embodiment of the present application;

[0050] FIG10 is a schematic diagram of an image sensor switching between different operating modes provided by an embodiment of the present application;

[0051] FIG11 is a schematic diagram of software and hardware collaboration of an electronic device 100 provided in an embodiment of the present application;

[0052] FIG12 is a flow chart of a video shooting method provided in this application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment can be included in at least one embodiment of the present embodiment application. The appearance of this phrase in various positions in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It can be understood explicitly and implicitly by those skilled in the art that the embodiments described herein can be combined with other embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of this application.

[0054] In the specification, claims, and accompanying drawings of this application, the terms "first," "second," "third," and the like are used to distinguish different objects and are not used to describe a particular order. Furthermore, the terms "including," "comprising," "having," and any variations thereof are intended to cover non-exclusive inclusions. For example, a list of steps or elements may be included, or alternatively, steps or elements not listed may be included, or other steps or elements may be included that are inherent to the process, method, product, or apparatus.

[0055] Only part relevant to the present application is shown in the accompanying drawings, not all of it. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processing or methods depicted as flow charts. Although flow charts describe various operations (or steps) as sequential processing, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of various operations can be rearranged. When its operation is completed, the processing can be terminated, but can also have additional steps not included in the accompanying drawings. The processing can correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0056] As used in this specification, the terms "component," "module," "system," "unit," and the like are used to refer to computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a unit can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or distributed between two or more computers. In addition, these units can be executed from various computer-readable media having various data structures stored thereon. Units can communicate, for example, through local and / or remote processes based on signals having one or more data packets (e.g., data from a second unit interacting with another unit in a local system, a distributed system, and / or a network. For example, the Internet interacts with other systems via signals).

[0057] First, the terms involved in this application are explained.

[0058] (1) Ambient brightness and brightness scene

[0059] Ambient brightness refers to the intensity of light (also known as illuminance) reflected from the object captured by the image sensor. Based on differences in light intensity, ambient brightness can be categorized into different brightness scenarios, including but not limited to high-brightness, medium-brightness, and low-brightness.

[0060] The image sensor mentioned in this application may be in the form of a semiconductor chip, including but not limited to a charge coupled device (CCD) and a complementary metal oxide (CMO) semiconductor.

[0061] For example, different brightness scenes may correspond to different brightness ranges. For example, the brightness range corresponding to the highlight scene may be greater than 2000 lux, the brightness range corresponding to the medium brightness scene may be 2000 lux-200 lux, and the brightness range corresponding to the low brightness scene may be 200 lux-0 lux.

[0062] It can be understood that the brightness scenes described in the embodiments of the present application may not be limited to the above three; and the brightness ranges corresponding to the three brightness scenes are only used as an example. The values ​​of the brightness ranges corresponding to different brightness scenes may also be other values, which is not limited in the embodiments of the present application.

[0063] (2) ISO

[0064] In the film era, ISO sensitivity was an internationally standardized measure of the speed of film used in traditional cameras. This sensitivity was determined by the film purchased, and the value was fixed, a property of the film itself. Digital cameras and mobile phone cameras, on the other hand, use electronic image sensors (CCDs) or COMS to detect the intensity of incoming light. To unify the measurement units with the film used in traditional cameras, the ISO sensitivity concept was introduced. Therefore, the ISO value of a digital camera also reflects its speed. A higher ISO number indicates greater sensitivity to light, while a lower ISO number indicates less sensitivity.

[0065] Taking CMOS as an example, the working principle of CMOS is as follows: (1) Photons enter the photodiode, and the photodiode absorbs the energy of the photons and is excited to generate electrons; (2) The electrons are output to the potential well; (3) An analog voltage signal is generated at both ends of the potential well; (4) The analog voltage signal is analog-amplified using a programmable gain amplifier to obtain an amplified voltage signal; (5) The amplified voltage signal is input into the analog circuit, and the amplified voltage signal is further amplified based on the analog gain (AG) in the analog circuit; (6) The further amplified voltage signal is converted from analog to digital; (7) The digital signal is input into the digital circuit, and the digital signal is amplified based on the digital gain (DG) in the digital circuit.

[0066] The amplification factor of the analog voltage signal in step (4) is the conversion gain (CG). The CG in step (4), the AG in step (5), and the DG in step (7) can be collectively understood as ISO. In some embodiments, CG*AG*DG=ISO. CG is fixed. ISO can be changed by adjusting AG and / or DG.

[0067] Typically, the voltage swing allowed in a pixel is determined during sensor design, and the voltage swing is then fixed.

[0068] (3) Dynamic range (DR)

[0069] Image sensors have a dynamic range, which refers to their ability to perceive the darkest and brightest illumination values ​​in a scene, or in other words, their ability to represent image brightness values. In most scenarios, the larger the dynamic range of an image sensor, the wider the range of light intensities it can perceive, resulting in richer image detail in the captured image. The dynamic range that an image sensor can perceive is limited and is generally determined by its full well capacity (FWC).

[0070] Full well capacity refers to the total number of electrons that the potential well of a single pixel can accommodate. When photons strike the image sensor surface, absorbed photons are converted into electrons, which are stored in the pixel's potential well. The maximum charge that a potential well can hold is the full well capacity, also known as the maximum well capacity. When the full well capacity is reached, an overexposed image will result. The larger the full well capacity of an image sensor, the more electrons the potential well of a single pixel can accommodate, making overexposition less likely.

[0071] After the sensor is designed, the voltage swing is fixed, and ISO can have a significant impact on the sensor's full well capacity. Specifically, assuming the pixel's allowable voltage swing is Vmax, Vmax = CG * ISO * V, where V represents the voltage generated across the potential well. With Vmax and CG fixed, a larger ISO value reduces the analog voltage V allowed across the potential well. This means the number of electrons Q = C * V that the potential well can accommodate decreases, and the full well capacity decreases. C represents the capacitance of the potential well, which is typically constant.

[0072] In other words, the full well capacity will be affected by ISO. The smaller the ISO, the larger the full well capacity, and the larger the ISO, the smaller the full well capacity. The full well capacity of a single pixel can be adjusted by adjusting the ISO size.

[0073] In some embodiments, the dynamic range may be obtained based on the darkest and brightest illumination values ​​in the captured scene perceived by the image sensor.

[0074] For example, the expression for dynamic range can be: DR = 20log10(bright / dark); where DR refers to dynamic range, bright refers to the brightest illuminance value perceived by the image sensor, and dark refers to the darkest illuminance value perceived by the image sensor. This expression can establish a corresponding relationship between the dynamic range of an environment and the ambient light intensity.

[0075] (4) High Dynamic Range (HDR)

[0076] A high dynamic range (HDR) scene can refer to a situation where the sensor's dynamic range is smaller than the scene's dynamic range. Consequently, the resulting image cannot accurately represent scene content that exceeds the sensor's brightness range. For example, when the ambient brightness exceeds the image sensor's maximum dynamic range, the captured image lacks bright areas. When the ambient brightness falls below the image sensor's minimum dynamic range, the captured image lacks detail in dark areas, resulting in poor imaging.

[0077] Typically, the dynamic range of nature exceeds 100dB, while high-end sensors can only reach 78dB, and consumer-grade sensors have a dynamic range of approximately 60dB. When shooting, if the sensor's dynamic range is smaller than the dynamic range of the scene being filmed, dynamic range overflow will occur, and the camera image will lose detail in dark areas or content in bright areas.

[0078] For example, if the dynamic range of an image sensor is 20dB to 78dB, while the dynamic range of the environment is between 1dB and 100dB, the captured image will lack details in dark areas and content in bright areas, resulting in poor imaging quality.

[0079] For example, FIG1A is a natural scene image observed by the human eye provided in an embodiment of the present application, wherein the brightness value of the scene ranges from 1dB to 100dB, wherein the brightness value of the first region 1011 ranges from 1dB to 15dB, and the brightness value of the second region 1012 ranges from 80dB to 100dB.

[0080] The dynamic range of the sensor is 20dB to 78dB. Since the dynamic range of the scene exceeds the dynamic range of the sensor, when taking a photo of the scene, the captured image cannot accurately present the content of the scene that exceeds the brightness value range of the sensor. Figure 1B is the effect diagram obtained by taking a photo of the natural scene in Figure 1A. In Figure 1A, the brightness value range of the first area 1011 is 1dB to 15dB, which belongs to the shadow area, and the brightness value range of the second area 1012 is 80dB to 100dB, which belongs to the bright area. Since the brightness value range of these two areas exceeds the brightness value range of the sensor, an HDR problem occurs in Figure 1B, that is: in Figure 1B, the first area looks black as a whole, and the objects in the area cannot be distinguished. The second area looks white as a whole, and the blue sky and white clouds cannot be distinguished.

[0081] (5) Dual conversion gain (DCG)

[0082] An image sensor with DCG capability has two potential wells per pixel, each corresponding to different full well capacities and conversion gains. A large full well capacity corresponds to low conversion gain (LCG) and low sensitivity, while a small full well capacity corresponds to high conversion gain (HCG) and high sensitivity. This allows the image sensor to utilize two potential wells (two sensitivities) and two conversion gains for the same scene, capturing two images in a single exposure: one in high-sensitivity mode and one in low-sensitivity mode. The high-sensitivity and low-sensitivity images are captured at the same time and exposure time. Electronics then combine the high-sensitivity and low-sensitivity images into a single image, creating a DCG image. The high-sensitivity image aims to restore detail in dark areas, while the low-sensitivity image aims to suppress overexposed areas and restore bright areas. HCG reduces the minimum dynamic range of the image sensor, while LCG increases the maximum dynamic range, thereby improving the sensor's dynamic range.

[0083] For example, for an ordinary image sensor, the perceptible dynamic range is between 20 dB and 78 dB, while for an image sensor with DCG capability, the perceptible dynamic range can reach between 1 dB and 100 dB.

[0084] For example, if the dynamic range of the environment is between 10dB and 80dB, and the dynamic range of the image sensor with DCG capability is also between 1dB and 100dB, then the image sensor with DCG capability can obtain the dark area details and bright area content of the captured image, and the imaging effect is better.

[0085] When the dynamic range of the image sensor is smaller than that of the environment, the DCG solution can be used to improve the dynamic range of the image sensor. Specifically, LCG can output images in low-sensitivity mode, and HCG can output images in high-sensitivity mode. The low-sensitivity image can be used to suppress overexposed areas and restore bright areas, while the high-sensitivity image can be used to restore dark area details.

[0086] For example, Figure 1C is an image output by the image sensor in low-sensitivity mode. It can be seen from Figure 1C that objects in the dark area of ​​the image (for example, the first area 1011) are relatively clear, and the content in the bright area of ​​the image cannot be displayed clearly (for example, the blue sky and white clouds in the second area 1012 cannot be distinguished).

[0087] For example, FIG1D is an image output by the image sensor in high-sensitivity mode. It can be seen from FIG1D that the content in the dark area of ​​the image cannot be displayed clearly (for example, a black area is displayed in the first area 1011, and the objects in the area cannot be distinguished), and the content can be clearly displayed in the bright area of ​​the image (for example, the blue sky and white clouds in the first area 3012 can be clearly distinguished).

[0088] The image in the low-sensitivity mode shown in FIG1C and the image in the high-sensitivity mode shown in FIG1D are synthesized to obtain the DCG image shown in FIG1E . As can be seen from FIG1E , the contents of both the dark and bright areas in the DCG image are clearly displayed (for example, the objects in the first area 1011 and the blue sky and white clouds in the second area 1012 are clearly displayed).

[0089] FIG2 shows the imaging principle of a DCG image.

[0090] As shown in Figure 2, binning is an image readout mode. When an electronic device captures an image, light reflected from the target object is captured by the camera and transmitted to the image sensor. The image sensor includes multiple photosensitive elements, and the charge collected by each photosensitive element constitutes a pixel. The charges sensed in adjacent pixels are added together and read out in a single-pixel mode, and a binning operation is performed on the pixel information. Specifically, binning can combine n×n pixels into a single pixel. For example, binning can combine 2×2 adjacent pixels into a single pixel, that is, the pixel information of these 2×2 adjacent pixels is presented as a single pixel. For example, as shown in Figure 2, when the image is a 4×4 pixel, binning can combine the adjacent 2×2 pixels into a single pixel, allowing the image sensor to combine the 4×4 image into a 2×2 image. This 2×2 image is then used as the binned image generated by the image sensor, which constitutes a single frame. Outputting images using pixel binning mode can increase the pixel's photosensitive area, thereby improving the sensitivity of dark areas to light.

[0091] When the image sensor supports DCG capability, the image sensor can output images in a DCG manner to enhance the dynamic range of the image and improve the imaging effect.

[0092] As shown in Figure 2, after the image sensor combines adjacent n×n pixels into a single pixel and before outputting an image based on the combined pixels, it can further utilize two conversion gains. For example, it can obtain output image data using two conversion gains, respectively, based on the HCG pathway and the LCG pathway. For example, the combined pixels can be input into the LCG pathway to synthesize an image in low-sensitivity mode, and then into the HCG pathway to synthesize an image in high-sensitivity mode. Specifically, the image sensor processes the combined pixels through the LCG circuit, analog amplification, analog-to-digital conversion, and digital amplification, and then outputs an LCG image. The image sensor also processes the combined pixels through the HCG circuit, analog amplification, analog-to-digital conversion, and digital amplification, and then outputs an HCG image. The image sensor then fuses the LCG and HCG images to obtain a fused DCG image and outputs the DCG image. It should be noted that the different ISOs in the LCG and HCG pathways result in different dynamic ranges for the corresponding LCG and HCG images.

[0093] As shown in Figure 3, the dynamic range of the HCG image can be 10dB-50dB, the dynamic range of the LCG image can be 20dB-100dB, and the dynamic range of the synthesized DCG image can be 10dB-100dB. This improves the dynamic range of the image sensor.

[0094] (6) Signal-to-noise ratio

[0095] An important criterion for measuring image quality is the image's signal-to-noise ratio (SNR). The SNR of images obtained through the LCG and HCG pathways has the following characteristics.

[0096] Figure 4 shows the signal-to-noise ratios of the images obtained by the LCG and HCG pathways at different illumination levels. As can be seen from Figure 4, in low-illuminance scenes (also known as low-brightness scenes), the HCG pathway outputs a higher signal-to-noise ratio (HCG) image, resulting in a clearer image. However, in low-illuminance scenes (also known as low-brightness scenes), the LCG pathway outputs a significantly reduced signal-to-noise ratio (LCG) image, resulting in a more blurred image.

[0097] In low-light scenes, if the HCG image and the LCG image are merged to obtain the DCG image, although the dynamic range of the imaging is improved, obvious noise stratification will appear in the fused part of the HCG image and the LCG image.

[0098] For example, Figure 5A shows an LCG image output by the LCG path in a low-light scene. Comparing Figure 5A with Figure 1C , it can be seen that in low-light scenes, the signal-to-noise ratio of the LCG image output by the LCG path is reduced, and the image is blurred.

[0099] FIG5B shows an HCG image output by the HCG path in a low-light scene. As can be seen from FIG5B , in a low-light scene, the HCG image output by the HCG path is clearer than the LCG image output by the LCG path.

[0100] The DCG image shown in Figure 5C is synthesized by combining the LCG image shown in Figure 5A with the HCG image shown in Figure 5B. As can be seen in Figure 5C, the blue sky and white clouds in the second region 1012 of the DCG image are clearly displayed because the blue sky and white clouds in the second region 1012 are synthesized from the HCG image. However, the object image in the first region 1011 is synthesized from the LCG image, and the signal-to-noise ratio of the LCG image differs significantly from that of the HCG image. Therefore, the clarity of the object image in the first region 1011 in Figure 5C is inconsistent with that of the images in other regions, resulting in noise stratification.

[0101] Therefore, the HCG image obtained through the HCG pathway is more suitable for application in low-light scenes.

[0102] Based on the above introduction, when the dynamic range of the image sensor is smaller than the dynamic range of the environment in video shooting, DCG can be used to improve the dynamic range of the image sensor, thereby improving the dynamic range of the video frame. However, the DCG solution has the following drawbacks in scenes that are too dark or too bright.

[0103] In low-light scenes (such as night scene mode), the light intensity is weak, and the illuminance value that the image sensor can perceive is also relatively low. If two image frames are output through the LCG path and the HCG path respectively to synthesize a DCG image, and the DCG image is used as the video frame acquired in the low-light scene, it can be seen from the embodiments of Figures 4, 5A, and 5C that the HCG image output by the HCG path and the LCG image output by the LCG path will have obvious noise stratification in low-light scenes, resulting in poor imaging effect.

[0104] In bright scenes (such as outdoor scenes), the light intensity is strong, and the image sensor can perceive high illumination values. However, the full well capacity of a single pixel is limited. When the environment is too bright, the full well capacity of a single pixel overflows, resulting in an overexposed image. In this case, the imaging quality is also poor.

[0105] Therefore, the DCG solution cannot solve the problem of poor imaging effects in different brightness scenes.

[0106] Based on this, in order to solve the problem of poor imaging effects in video recording, the present application provides a video shooting method that can improve video shooting effects in scenes with different brightness.

[0107] The video recording may include a preview state before the video recording starts and a preview state after the video recording starts.

[0108] Image sensors can monitor ambient brightness and dynamic range in real time.

[0109] When the dynamic range of the environment is smaller than the dynamic range of the image sensor, the image sensor can operate in a low dynamic mode and output video frames in a binning manner.

[0110] When the dynamic range of the environment is greater than the dynamic range of the image sensor, the image sensor can switch from the low dynamic mode to the high dynamic mode. In the high dynamic mode, the dynamic range of the image sensor can be increased so that the dynamic range of the image sensor in the high dynamic mode is greater than the dynamic range of the environment.

[0111] After the image sensor switches to high dynamic mode, it can determine the ambient brightness and determine the image sensor's operating mode based on the ambient brightness. The image sensor's operating mode in high dynamic mode is different within different ambient brightness ranges. This can improve the image sensor's imaging effect in high dynamic mode, avoiding noise in low-light scenes or overexposed images in high-light scenes. For the various operating modes of the image sensor in high dynamic mode and how to switch the various operating modes of the image sensor in high dynamic mode, please refer to the detailed description below.

[0112] The following embodiments of the present application will take how the image sensor switches the working mode in the preview state after starting recording as an example to illustrate.

[0113] Next, the hardware structure of the electronic device 100 is introduced.

[0114] The electronic device 100 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) or virtual reality (VR) device, etc. The embodiment of the present application does not impose any particular limitation on the specific type of the electronic device 100.

[0115] Please refer to Figure 6, which is a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0116] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0117] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0118] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0119] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0120] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0121] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0122] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise and brightness. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0123] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0124] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0125] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0126] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0127] The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194 .

[0128] The gyroscope sensor 180B can be used to determine the motion of the electronic device 100. In some embodiments, the gyroscope sensor 180B can be used to determine the angular velocity of the electronic device 100 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can also be used for image stabilization. The gyroscope sensor 180B can also be used for navigation and somatosensory gaming scenarios.

[0129] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0130] The magnetic sensor 180D includes a Hall sensor, and the electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case.

[0131] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the terminal's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0132] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0133] The proximity light sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 100 emits infrared light through the LED. The photodiode detects infrared light reflected from nearby objects, automatically turning off the screen to save power. The proximity light sensor 180G can also be used to automatically unlock and lock the screen in holster mode and pocket mode.

[0134] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0135] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0136] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy.

[0137] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0138] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone in a human vocal part.

[0139] In the embodiment of the present application, the software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device 100.

[0140] As shown in FIG7 , the electronic device may include: an application layer, an application framework, a hardware abstraction layer (HAL) layer, and a kernel layer.

[0141] The application layer can include a series of application packages. As shown in Figure 7, the application packages may include applications such as the camera application, gallery, calendar, map, music, short message, and call applications. The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. As shown in Figure 7, the application framework layer may include a window manager, content provider, view system, telephony manager, resource manager, notification manager, etc.

[0142] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0143] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0144] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0145] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0146] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0147] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0148] The hardware abstraction layer can include multiple functional modules, such as a perception module and an image processor.

[0149] The perception module is used to determine the dynamic range and ambient brightness of the environment based on the video frames captured by the image sensor. The image sensor's operating mode is then determined based on the dynamic range and brightness of the environment, enabling dynamic switching of the image sensor's operating mode. For example, if the perception module determines that the dynamic range of the environment is greater than the dynamic range of the image sensor for m consecutive frames, the perception module may determine that the image sensor needs to switch to high dynamic mode. The perception module then determines the image sensor's operating mode in high dynamic mode based on the perceived ambient brightness. The image sensor's operating mode in high dynamic mode varies within different ambient brightness ranges.

[0150] The image processor is used to perform pre-processing and post-processing on the video frames to obtain preview video frames and recorded video frames.

[0151] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, etc.

[0152] The camera driver is used to trigger the camera to turn on when receiving a trigger command sent by the camera application at the application layer.

[0153] The hardware layer primarily includes components within the display and camera modules, such as cameras. Cameras can include lenses, mirrors, and image sensors. These components are used to start operating under the control of corresponding drivers within the core layer. For example, a camera can start and capture images under the control of a camera driver. The image sensor within a camera can also switch its operating mode under the control of the camera driver within the core layer.

[0154] The following introduces some shooting scenes provided by this application.

[0155] 1. Enter the video preview interface (Figure 8A to Figure 8C).

[0156] FIG. 8A exemplarily shows an exemplary user interface 710 on the electronic device 100 for displaying applications installed on the electronic device 100 .

[0157] User interface 710 displays a page with application icons, which may include multiple application icons (e.g., a weather application icon, a calendar application icon, an album application icon, a note application icon, an email application icon, an application store application icon, a settings application icon, etc.). A page indicator may also be displayed below the multiple application icons to indicate the positional relationship between the currently displayed page and other pages. Below the page indicator are multiple application icons (e.g., a camera application icon 711, a browser application icon, a message application icon, and a dial application icon). The application icons remain displayed when the page is switched.

[0158] It is understood that the camera application icon 711 is an icon of a camera application. The camera application icon 711 can be used to trigger the launch of the camera application. The camera application is an image capture application on electronic devices such as smartphones and tablets. This application does not limit the name of the application.

[0159] It should be noted that the user interface 710 shown in FIG8A is only an example provided by the present application and should not be considered as limiting the present application. In other words, the user interface 710 can display more or less content, and the present application does not limit this.

[0160] The electronic device 100 can detect a user operation (such as a touch / click operation) on the camera application icon 711. In response to the operation, the electronic device 100 can display the shooting interface 720 shown in Figure 8B. The shooting interface 720 can be the user interface of the default shooting mode of the camera application, and the user can preview the image and complete the shooting on this interface. In other words, the user can open the shooting interface 720 of the camera application by clicking the camera application icon 711.

[0161] It is understandable that the user operations mentioned in this application may include but are not limited to touch, click, voice control, gestures and other operations, and this application does not limit this.

[0162] As shown in FIG. 8B , the shooting interface 720 may include a mode bar 721 , a shooting control 722 , a preview window 723 , a review control 724 , a camera flip control 725 , and a quick function area 726 .

[0163] The mode bar 721 may include multiple shooting mode options, such as "night scene", "portrait", "photo", "video recording", etc. Different shooting modes can provide users with shooting services with different effects. Users can select any shooting mode from multiple shooting modes to shoot according to different needs. For example, "photo recording" may be the default shooting mode for taking photos. "Video recording" is used to record videos. The "night scene" mode is suitable for shooting scenes with dim light, such as at night. The "portrait" mode is suitable for shooting scenes where the subject is a person. The electronic device 100 can also provide more shooting modes, such as "large aperture", "movie", "professional", etc., which are not listed here one by one.

[0164] The capture control 722 is used to trigger a photo. The electronic device 100 can detect whether a user action, such as a click, is applied to the capture control 722. Upon detecting a user action on the capture control 722, the electronic device 100 can generate a capture instruction. Based on the capture instruction, the electronic device 100 can retrieve the image reported by the camera at the corresponding timestamp and save it as a photo.

[0165] The preview window 723 can be used to display the image reported by the camera in real time. In different shooting modes, the electronic device 100 can process the image reported by the camera to improve the image display effect. For example, in "Portrait" mode, the electronic device 100 can blur the background of the image reported by the camera to highlight the portrait. Here, the preview window 723 can display the image processed by the image processing algorithm corresponding to the different shooting modes in real time, so that the user can perceive the shooting effect corresponding to the different shooting modes in real time.

[0166] The review control 724 can be used to browse thumbnails of photos / videos that have been taken. When a user operation acting on the review control 724 is detected, the electronic device 100 can also display the best photo corresponding to the thumbnail.

[0167] The camera flip control 725 can be used to monitor user operations that trigger camera flipping. The electronic device 100 can detect a user operation, such as a touch operation, acting on the camera flip control 725. In response to the operation, the electronic device 100 can flip the camera used for shooting, such as switching the rear camera to the front camera, or switching the front camera to the rear camera.

[0168] The quick function area 726 may include a control 726A for the protagonist recording mode, an AI scene recognition control 726B, a flash control 726C, a color mode control 726D, a setting control 726E, and the like. The control 726A for the protagonist recording mode can be used to trigger the electronic device 100 to identify the protagonist among multiple characters in the preview screen when it is turned on. The AI ​​scene recognition control 726B can be used to trigger the electronic device 100 to identify the shooting scene in the preview screen when it is turned on. The current AI scene recognition control 726B is in the off state. The flash control 726C can be used to trigger the electronic device 100 to turn on or off the flash. The color mode control 726D can be used to trigger the electronic device 100 to use a color filter to process the image captured by the camera. The setting control 726E can be used to set the shooting parameters of the electronic device 100 (for example, image size, image storage format, etc.).

[0169] The shooting interface 720 may also include more or fewer controls, which is not limited in the embodiment of the present application.

[0170] The electronic device 100 can detect user operations on the shooting mode options in the mode bar 721 and change the currently used shooting mode according to the above user operations. The above user operations are, for example, left / right swipe operations. For example, when it is detected that the mode bar 721 is dragged and slid to the left (left swipe operation) and the float stops at the "Record" option, the electronic device 100 can switch to the "Record" mode and display the recording interface 810 shown in Figure 8C. The recording interface 810 can also be called a recording preview interface.

[0171] As shown in Figure 8C, after entering the "recording" mode, the electronic device 100 can display a preview video frame in the preview window 723, and the electronic device 100 can switch the preview interface corresponding to the original "photographing" mode to the preview interface corresponding to the "recording" mode. Specifically, the electronic device 100 can switch the shooting control 722 of the original "photographing" mode to the start recording control 812 of the "recording" mode. At the same time, the electronic device 100 can display a timestamp control 811 in the preview window 723. The timestamp control 811 is used to indicate the length of the recorded video. Before starting to record the video, the time displayed in the timestamp control 811 is 0. After starting to record the video, the time displayed in the timestamp control 811 increases as the length of the recorded video increases.

[0172] Optionally, after the electronic device 100 enters the "recording" mode and displays the recording preview interface, the electronic device 100 may start to determine the dynamic range of the environment and determine whether to enter the low dynamic range mode or the high dynamic range mode.

[0173] 2. If the dynamic range of the environment is smaller than the dynamic range of the image sensor, the image sensor operates in a low dynamic range mode and outputs a preview video frame based on the low dynamic range mode.

[0174] If the dynamic range of the environment is greater than the dynamic range of the image sensor, the image sensor operates in a high dynamic mode and outputs a preview video frame based on the high dynamic mode.

[0175] Optionally, after the electronic device 100 enters the "recording" mode and displays the video preview interface, the electronic device 100 may begin to determine the dynamic range of the environment. It may also determine whether to enter low dynamic mode or high dynamic mode. Thus, if the dynamic range of the environment is greater than the dynamic range of the image sensor in the electronic device 100, the image sensor may have already entered high dynamic mode before the electronic device 100 begins capturing video. The image sensor may output preview video frames based on the high dynamic mode. After the electronic device 100 begins capturing video, the image sensor may directly output recorded video frames based on the high dynamic mode.

[0176] After the electronic device 100 enters the “recording” mode and displays the recording preview interface, the electronic device 100 may determine the size relationship between the environment dynamic range and the image sensor dynamic range based on the preview video frame.

[0177] If the electronic device 100 determines that the dynamic range of the environment is greater than the dynamic range of the image sensor based on n consecutive preview video frames, where n is a positive integer greater than or equal to 1, the electronic device 100 may switch the image sensor from low dynamic range mode to high dynamic range mode. Otherwise, the image sensor operates in low dynamic range mode. Optionally, the default operating mode of the image sensor in the electronic device 100 is low dynamic range mode.

[0178] The low dynamic range mode may refer to the electronic device 100 generating video frames by binning. In the low dynamic range mode, the dynamic range of the image sensor is not changed.

[0179] High dynamic range (HDR) refers to an image sensor's ability to enhance its dynamic range through dual-sensor modes, or through long and short exposure modes. HDR can be categorized as low-light HDR, medium-light HDR, and high-light HDR. For a more detailed explanation of HDR, please refer to the following description.

[0180] Before the image sensor of the electronic device 100 switches to the high dynamic range mode, it is necessary to determine which operating mode within the high dynamic range mode to switch to. Based on the above description, in low-light scenes (such as night scenes), the HCG image output by the HCG path and the LCG image output by the LCG path will produce a synthesized image with noise stratification, resulting in poor imaging quality.

[0181] In bright scenes (such as outdoor scenes), the full well capacity of a single pixel is also limited. When the environment is too bright, the full well capacity of a single pixel overflows, resulting in an overexposed image. In this case, the imaging effect is also poor.

[0182] Therefore, the electronic device 100 also needs to determine the ambient brightness and enter different operating modes in the high dynamic mode based on the ambient brightness.

[0183] For example, if the electronic device 100 determines based on the preview video frame that the ambient dynamic range is greater than the dynamic range of the image sensor in the electronic device 100, and the ambient brightness is less than the first illuminance value, the electronic device 100 can control the image sensor to enter the low-brightness high-dynamic mode.

[0184] If the electronic device 100 determines based on the preview video frame that the ambient dynamic range is greater than the dynamic range of the image sensor in the electronic device 100, and the ambient brightness is greater than the first illuminance value and less than the second illuminance value, the electronic device 100 can control the image sensor to enter the medium-brightness high dynamic mode.

[0185] If the electronic device 100 determines based on the preview video frame that the ambient dynamic range is greater than the dynamic range of the image sensor in the electronic device 100 and the ambient brightness is greater than the second illuminance value, the electronic device 100 can control the image sensor to enter the high-brightness high-dynamic mode.

[0186] For example, as shown in FIG8D , when the electronic device 100 confirms that the dynamic range of the environment is greater than the dynamic range of the image sensor in the electronic device 100 and the ambient brightness is less than the first illuminance value, the image sensor can output a preview video frame based on the low-brightness high-dynamic mode and display the preview video frame in the preview window 723 shown in FIG8D .

[0187] The electronic device 100 may receive a user operation on the start video recording control 812 and start recording the video.

[0188] As shown in FIG8D , electronic device 100 may detect a user operation on the start recording control 812. In response to the operation, electronic device 100 may start recording a video. Accordingly, electronic device 100 may display the video recording interface shown in FIG8E . Electronic device 100 may switch the original start recording control 812 to an end recording control 814.

[0189] As shown in Figure 8E, the video recording interface 810 may further include a control 813. The control 813 may be used to receive a user's shooting operation during the video recording process, capture a photo, and save the photo.

[0190] Optionally, in other embodiments, the electronic device 100 may determine the size relationship between the environmental dynamic range and the image sensor dynamic range based on the recorded video frames after starting to shoot the video, and determine whether to switch the working mode of the image sensor. This application does not limit this.

[0191] (1) When the ambient brightness is less than the first illumination value, the image sensor operates in a low-brightness high-dynamic mode and outputs a recorded video frame based on the low-brightness high-dynamic mode ( FIG. 8E - FIG. 8F ).

[0192] As shown in FIG8E , the image sensor in the electronic device 100 may output a recorded video frame based on the low-brightness high-dynamic mode, and display the recorded video frame in a preview window 723 shown in FIG8E .

[0193] Optionally, if during the process of video recording by the electronic device 100, the ambient brightness becomes brighter or darker due to a change in the shooting position or a change in the natural environment, and the ambient brightness is still less than the first illumination value, the image sensor still operates in the low-brightness high-dynamic mode. The electronic device 100 can change the sensitivity of the image sensor so that the brightness of the recorded video frames synthesized by the image sensor gradually brightens or darkens, thereby preventing sudden changes in the brightness of the recorded video frames from affecting the user's visual experience.

[0194] For example, when the ambient brightness becomes dim, the electronic device 100 may increase the sensitivity of the image sensor so that the brightness of the recorded video frames synthesized by the image sensor gradually dims.

[0195] For another example, when the ambient brightness increases, the electronic device 100 can reduce the sensitivity of the image sensor so that the brightness of the recorded video frame synthesized by the image sensor gradually increases. For example, as shown in Figure 8F, when the ambient brightness increases but is less than the first illumination value, the electronic device 100 can display the recorded video frame shown in Figure 8F. The brightness of the recorded video frame shown in Figure 8F is higher than that of the recorded video frame shown in Figure 8E.

[0196] As the recording time increases, the time displayed in the timestamp control 811 also gradually increases.

[0197] In some embodiments, if the ambient brightness becomes brighter, causing the ambient brightness to be greater than the first illuminance value but less than the second illuminance value, the electronic device 100 can control the image sensor to enter the medium-brightness high dynamic mode.

[0198] Optionally, when the electronic device 100 determines that the ambient brightness is greater than the first illuminance value and less than the second illuminance value for m consecutive video frames, the electronic device 100 may control the image sensor to enter the medium-brightness high dynamic range mode. m is a positive integer greater than or equal to 1.

[0199] (2) When the ambient brightness is greater than the first illuminance value and less than the second illuminance value, the image sensor operates in the medium-brightness high-dynamic mode and outputs the recorded video frame based on the medium-brightness high-dynamic mode (Figure 8G-Figure 8H).

[0200] As shown in FIG8G , the image sensor in the electronic device 100 may output a recorded video frame based on the medium-brightness high dynamic mode, and display the recorded video frame in a preview window 723 shown in FIG8G .

[0201] Optionally, if during the process of video recording by the electronic device 100, the ambient brightness becomes brighter or darker due to a change in the shooting position or a change in the natural environment, and the ambient brightness is still greater than the first illuminance value and less than the second illuminance value, and the image sensor is still operating in the medium-brightness high dynamic range mode, the electronic device 100 may change the sensitivity of the image sensor so that the brightness of the recorded video frames synthesized by the image sensor gradually brightens or darkens, thereby preventing sudden changes in the brightness of the recorded video frames from affecting the user's visual experience.

[0202] For example, when the ambient brightness becomes darker, if the ambient brightness is less than the first illuminance value, the electronic device 100 may switch the working mode of the image sensor to the low-brightness high-dynamic working mode.

[0203] For example, when the ambient brightness becomes darker, but the ambient brightness is greater than the first illuminance value and less than the second illuminance value, the electronic device 100 can increase the sensitivity of the image sensor so that the brightness of the recorded video frame synthesized by the image sensor slowly darkens.

[0204] For another example, when the ambient brightness becomes brighter, but the ambient brightness is greater than the first illuminance value and less than the second illuminance value, the electronic device 100 can reduce the sensitivity of the image sensor so that the brightness of the recorded video frame synthesized by the image sensor gradually brightens. For example, as shown in Figure 8H, when the ambient brightness becomes brighter, but the ambient brightness is greater than the first illuminance value and less than the second illuminance value, the electronic device 100 can display the recorded video frame shown in Figure 8H. The brightness of the recorded video frame shown in Figure 8H is higher than that of the recorded video frame shown in Figure 8G.

[0205] As the recording time increases, the time displayed in the timestamp control 811 also gradually increases.

[0206] In some embodiments, if the ambient brightness becomes brighter, causing the ambient brightness to be greater than the second illuminance value, the electronic device 100 may control the image sensor to enter a high-brightness high-dynamic mode.

[0207] Optionally, when the electronic device 100 determines that the ambient brightness is greater than the second illumination value in m consecutive video frames, the electronic device 100 may control the image sensor to enter the high-brightness high-dynamic mode.

[0208] (3) When the ambient brightness is greater than the second illumination value, the image sensor operates in the high-brightness high-dynamic mode and outputs a recorded video frame based on the high-brightness high-dynamic mode ( FIG. 8I ).

[0209] High-brightness high-dynamic mode differs from medium-brightness high-dynamic mode or low-brightness high-dynamic mode in that it operates in high-brightness environments. In high-brightness environments, the full well capacity of a pixel is also limited and has reached its maximum value, resulting in the pixel's potential well being unable to carry more charge, leading to charge overflow and an overblown image. To restore the overblown image, the electronic device 100 can capture a recorded video frame through a short exposure, and use the short-exposure recorded video frame to restore the overblown image.

[0210] As shown in FIG8I , the image sensor in the electronic device 100 may output a recorded video frame based on the high-brightness high-dynamic mode, and display the recorded video frame in the preview window 723 shown in FIG8I .

[0211] Optionally, if during the process of video recording by the electronic device 100, the ambient brightness becomes brighter or darker due to a change in the shooting position or a change in the natural environment, and the ambient brightness is still greater than the second illumination value, and the image sensor is still operating in the high-brightness high-dynamic mode, the electronic device 100 can change the exposure duration of the short-exposure frame so that the brightness of the recorded video frame synthesized by the image sensor gradually brightens or darkens, thereby preventing sudden changes in the brightness of the recorded video frame from affecting the user's visual experience.

[0212] For example, when the ambient brightness becomes darker, if the ambient brightness is less than the second illuminance value, the electronic device 100 may switch the operating mode of the image sensor to the medium-brightness high-dynamic operating mode.

[0213] When the ambient brightness becomes darker, but the ambient brightness is greater than the second illuminance value, the electronic device 100 can increase the exposure time of the short exposure frame so that the brightness of the recorded video frame synthesized by the image sensor gradually darkens.

[0214] For another example, when the ambient brightness becomes brighter, but the ambient brightness is greater than the second illuminance value, the electronic device 100 can reduce the exposure time of the short exposure frame so that the brightness of the recorded video frame synthesized by the image sensor slowly becomes brighter.

[0215] As the recording time increases, the time displayed in the timestamp control 811 also gradually increases.

[0216] Optionally, the above embodiment only divides the operating modes of the image sensor in the high dynamic range mode into three categories: low-brightness high dynamic range mode, medium-brightness high dynamic range mode, and high-brightness high dynamic range mode. In other embodiments, the operating modes of the image sensor in the high dynamic range mode are not limited to these three categories, and may be divided into more or fewer categories. This application does not impose any limitation on this.

[0217] In some embodiments, before switching the image sensor in electronic device 100 to high dynamic range (HDR), electronic device 100 determines that the ambient brightness is greater than a second illuminance value. Electronic device 100 may first switch to medium-brightness HDR mode and gradually reduce the image sensor's sensitivity. Thereafter, the image sensor in electronic device 100 switches to high-brightness HDR mode. This allows the brightness of the recorded video frames generated by electronic device 100 to gradually increase.

[0218] In some embodiments, before switching the image sensor in the electronic device 100 to the high dynamic range, the electronic device 100 determines that the ambient brightness is greater than the second illumination value. The electronic device 100 may also directly switch the image sensor to the high-brightness high dynamic range mode.

[0219] It can be seen from the embodiments shown in Figures 8D-8E above that as the ambient brightness changes, the electronic device 100 can dynamically switch the working mode of the image sensor, which not only improves the dynamic range of the video shot by the electronic device 100, but also the brightness of the recorded video frames in the video can change with the change of the ambient brightness, thereby improving the imaging effect of the recorded video frames.

[0220] Below, in conjunction with Figure 9, the specific process of how the electronic device 100 displays preview video frames and recorded video frames is described in detail. Please refer to Figure 9, which is a flow chart of a video shooting method provided in an embodiment of the present application. The implementation of this method can be based on the interaction and cooperation between the application layer (such as the camera application) and the hardware abstraction layer (HAL) in the electronic device, wherein the application layer mainly involves the CameraUI and image storage modules in the camera application; the HAL layer mainly involves the perception module. As shown in Figure 9, the specific process of the electronic device displaying preview video frames and recording video frames is as follows:

[0221] A video preview interface is displayed and a preview video frame is displayed ( S901 - S915 ).

[0222] S901. Start the camera application.

[0223] The system desktop can detect the user's operation of opening the camera application, such as the user clicking the desktop icon of "Camera". Exemplarily, as shown in the embodiment of Figure 8A above, when the system desktop detects an input operation for the camera application icon 711, the CameraUI module is started. After the camera application is started, the CameraUI module can display the shooting interface 720 shown in Figure 8B above. The camera application is an application with a camera function in the electronic device 100. The CameraUI module can be responsible for the human-computer interaction of the camera application, such as controlling the display of the shooting interface and its interface elements and responding to user operations occurring in the shooting interface.

[0224] S902: The CameraUI module receives a first input operation from the user.

[0225] Exemplarily, the first input may be a user operation on a shooting mode option in the mode bar 721 . For example, the first input may be a user operation of dragging the mode bar 721 to the left and causing the float to stop at the “recording” option.

[0226] S903: In response to the first input operation, the CameraUI module displays a video preview interface.

[0227] In response to the first input operation, the electronic device 100 enters the “recording” mode and displays the recording preview interface shown in FIG8C . The CameraUI module can display the preview video frame in the preview window 723 .

[0228] S904 : In response to the first input operation, the image sensor outputs a preview video frame based on the low-dynamic mode.

[0229] After entering the "recording" mode, the electronic device 100 defaults to the image sensor outputting preview video frames in low dynamic range mode. In other words, in low dynamic range mode, the dynamic range of the image sensor is not adjusted, and the electronic device 100 generates preview video frames through binning.

[0230] S905. The image sensor sends the preview video frame generated in real time to the perception module.

[0231] S906 : The image sensor sends the preview video frame generated in real time to the CameraUI module.

[0232] After generating the preview video frame, the image sensor sends the preview video frame to the perception module and the CameraUI module respectively.

[0233] The perception module can obtain the ambient brightness and ambient dynamic range based on the preview video frame.

[0234] The CameraUI module may display the preview video frame in the preview window 723 .

[0235] S906 may also be executed before S905, or simultaneously with S905, which is not limited in this application.

[0236] S907. The CameraUI module displays the preview video frame output in the low dynamic mode in the preview window.

[0237] After receiving the preview video frame sent by the image sensor, the CameraUI module may display the preview video frame output by the image sensor based on the low dynamic mode in the preview window 723 .

[0238] For example, the preview video frame output by the image sensor based on the low dynamic mode may be the preview video frame displayed in the preview window 723 in FIG. 8C .

[0239] S905 and S907 may be executed in real time, periodically, or irregularly, so that the CameraUI module can refresh the preview video frame displayed in the preview window 723 .

[0240] S908. The perception module obtains the environmental dynamic range and environmental brightness based on the preview video frame.

[0241] S908 may also be executed before S907, and S908 may also be executed simultaneously with S907, which is not limited in this application.

[0242] S909. The perception module sends the ambient dynamic range and ambient brightness to the image sensor.

[0243] After acquiring the ambient dynamic range and ambient brightness, the perception module sends the ambient dynamic range and ambient brightness to the image sensor.

[0244] Optionally, the perception module may determine the ambient brightness based on a grayscale histogram or an average brightness value of the preview video frame.

[0245] As described above, different brightness scenes can be divided based on the illuminance value of the ambient brightness. For example, low-light scenes, medium-light scenes, and high-light scenes have different brightness ranges. For example, the illuminance value of a low-light scene is less than a first illuminance value, the illuminance value of a medium-light scene is greater than the first illuminance value but less than a second illuminance value, and the illuminance value of a high-light scene is greater than the second illuminance value.

[0246] In a possible implementation, the electronic device 100 stores grayscale histograms corresponding to different brightness scenes. The grayscale histogram is used to represent the brightness distribution of pixels in an image. The brightness can be understood as the brightness of the image in YUV format.

[0247] Each brightness scene may include one or more grayscale histograms. The perception module may obtain a grayscale histogram of the preview video frame based on the preview video frame. The similarity between the grayscale histogram of the preview video frame and the grayscale histograms corresponding to different brightness scenes is calculated respectively. When the similarity is greater than a first threshold, the perception module may determine the current brightness scene. For example, if the similarity calculated by the perception module between the grayscale histogram of the preview video frame and the preset grayscale histogram under the low-light scene is greater than the first threshold, the perception module may determine that the current shooting scene is a low-light scene and determine the current ambient brightness.

[0248] In other possible implementations, the perception module may calculate the average illuminance value of each pixel in the preview video frame and, based on the average illuminance value, determine the ambient brightness, thereby determining the brightness scene. For example, if the perception module determines that the average illuminance value of the pixels in the preview video frame is less than the first illuminance value, the perception module may determine that the current shooting scene is a low-light scene, and the average illuminance value of the pixels in the preview video frame is the current ambient brightness.

[0249] Not limited to the above two implementation methods, the perception module can also determine the ambient brightness in other ways, which is not limited in this application.

[0250] S906, S908 and S909 may also be executed in real time / periodically / irregularly, so that the image sensor can dynamically switch between different working modes based on the environmental dynamic range and the environmental brightness.

[0251] Optionally, steps S906, S908, and S909 may not be performed. Ambient brightness and ambient dynamic range may be determined using another light metering sensor. A light metering sensor may be provided in the camera of the electronic device 100. It is understood that the light metering sensor may be used to measure the brightness of light reflected from the subject, i.e., reflective light metering. The light metering sensor may include a light metering element. Incident light is refracted through the lens and reflector of the electronic device 100 and enters the built-in light metering sensor, which may then acquire the ambient brightness and ambient dynamic range.

[0252] S910: The image sensor needs to determine whether the dynamic range of the environment is greater than the dynamic range of the image sensor.

[0253] The dynamic range of the image sensor is set at the factory. After receiving the ambient dynamic range and ambient brightness from the perception module, the image sensor can compare the dynamic range of the image sensor with the ambient dynamic range.

[0254] When the dynamic range of the environment is greater than the dynamic range of the image sensor, step S911 is executed, and the image sensor switches to the high dynamic mode.

[0255] As shown in FIG8C , when the dynamic range of the image sensor is smaller than the ambient dynamic range, the image details in the preview video frame output by the image sensor based on the low dynamic mode are not clear. For example, the bright area content (such as blue sky and white clouds) and the dark area details (objects) in FIG8C are not clear.

[0256] When the dynamic range of the environment is smaller than the dynamic range of the image sensor, S914 is executed, and the image sensor still operates in the low dynamic mode.

[0257] Optionally, if the image sensor confirms that the dynamic range of the environment is greater than the dynamic range of the image sensor based on n consecutive preview video frames, n is a positive integer greater than or equal to 1, and the image sensor executes S911 again. Otherwise, the image sensor executes S914.

[0258] S911: The image sensor needs to confirm whether the ambient brightness is greater than a first illumination value.

[0259] As can be seen from the above description, in medium-brightness scenes, the signal-to-noise ratio of the HCG image output by the HCG path and the LCG image output by the LCG path are both relatively good. However, as the ambient brightness decreases, in low-brightness scenes (such as night scenes), the signal-to-noise ratio of the LCG image output by the LCG path will be significantly reduced. As a result, the preview video frame synthesized by the HCG image output by the HCG path and the LCG image output by the LCG path will show obvious noise stratification, resulting in poor imaging effect.

[0260] As ambient brightness increases, the full well capacity of a single pixel is limited in bright scenes (such as outdoor scenes). When the ambient brightness is too bright, the full well capacity of a single pixel overflows, resulting in an overexposed image. The preview video frame synthesized by the HCG image output by the HCG path and the LCG image output by the LCG path will have overexposed areas, resulting in poor imaging quality.

[0261] Then the image sensor can switch to different working modes based on different ambient brightness ranges in high dynamic mode to solve the imaging problems existing in the above-mentioned low-light scenes or high-light scenes.

[0262] Therefore, before determining to enter the high dynamic mode, the image sensor needs to obtain the ambient brightness and switch to different high dynamic working modes based on different ambient brightness ranges.

[0263] The reason why the ambient dynamic range is greater than the image sensor's dynamic range can be due to the environment being too bright, or due to the environment being too dark, resulting in the ambient dynamic range being less than the image sensor's dynamic range. Therefore, the image sensor must also determine whether the ambient brightness is greater than the first illumination value.

[0264] When the ambient brightness is less than the first illuminance value, it indicates that the ambient dynamic range is greater than the dynamic range of the image sensor because the ambient brightness is too dark. In this case, the image sensor may switch to a low-brightness, high-dynamic operating mode, that is, execute S912.

[0265] If the ambient brightness is greater than the first illuminance value, it indicates that the ambient dynamic range is greater than the dynamic range of the image sensor due to the ambient brightness being too bright, and the image sensor can be switched to a medium brightness high dynamic range operating mode, that is, executing S913 .

[0266] Optionally, the ambient brightness greater than the first illuminance value may include two situations: the ambient brightness greater than the second illuminance value and the ambient brightness greater than the first illuminance value but less than the second illuminance value. That is, the ambient brightness greater than the first illuminance value may include two situations: a high-brightness scene and a medium-brightness scene.

[0267] If the current brightness scene is a medium-bright scene, the image sensor can first switch to the medium-brightness high dynamic working mode.

[0268] If the current brightness scene is a high-brightness scene, the image sensor can first switch to the medium-brightness high-dynamic working mode, and then switch to the high-brightness high-dynamic working mode, so that the brightness of the preview video frame displayed by the electronic device 100 is a gradually brightening process.

[0269] If the current brightness scene is a high-brightness scene, the image sensor can also directly switch to the high-brightness and high-dynamic working mode, which is not limited in this application.

[0270] It should be noted that S910 and S911 may also be performed by the perception module. When the perception module determines that the image sensor needs to switch from the low dynamic mode to the low-brightness high dynamic mode, the perception module may send a switching notification to the image sensor, and the image sensor switches to the low-brightness high dynamic mode after receiving the notification.

[0271] When the perception module determines that the image sensor needs to switch from the low dynamic mode to the medium brightness high dynamic mode, the perception module can send a switching notification to the image sensor, and the image sensor switches to the medium brightness high dynamic mode after receiving the switching notification.

[0272] When the perception module determines that the image sensor does not need to switch operating modes, the perception module may not send a notification to the image sensor, and the image sensor continues to operate in the low dynamic mode. The perception module may also send a notification to the image sensor, and the image sensor continues to operate in the low dynamic mode after receiving the switching notification.

[0273] S912: The image sensor switches from the low-dynamic mode to the low-brightness high-dynamic mode, and outputs a preview video frame based on the low-brightness high-dynamic mode.

[0274] When the ambient dynamic range is greater than the dynamic range of the image sensor and the ambient brightness is less than the first illumination value, the image sensor may switch to a low-brightness high-dynamic operating mode and output a preview video frame based on the low-brightness high-dynamic mode.

[0275] Next, we will introduce how the image sensor switches from low dynamic range mode to low-light high dynamic range mode.

[0276] In some embodiments, in low-light scenes, the signal-to-noise ratio of the HCG image output by the HCG channel is relatively good. Two frames of HCG images can be output through the HCG channel based on different sensitivities, and the two frames of HCG images can be synthesized to obtain a preview video frame in low-light high-dynamic mode.

[0277] In low-light scenes, the two HCG images obtained through the HCG path can be referred to as a first HCG image and a second HCG image. The sensitivity of the HCG path used to obtain the first HCG image differs from that used to obtain the second HCG image. For example, the sensitivity of the HCG path used to obtain the first HCG image is higher than that of the HCG path used to obtain the second HCG image. The first HCG image and the second HCG image are synthesized to obtain a preview video frame. The first HCG image can be used to obtain dark area details in the preview video frame, while the second HCG image can be used to obtain bright area content in the preview video frame.

[0278] In one possible implementation, the operating mode of the image sensor in a low-light scene can be set to an operating mode in which, in the low-light scene, the sensitivity of the HCG path for obtaining the first HCG image and the sensitivity of the HCG path for obtaining the second HCG image are fixed.

[0279] In other possible implementations, the low-light scene can also be divided into multiple brightness ranges, and the operating mode of the image sensor corresponding to each brightness range is different. That is, the operating mode of the image sensor in the low-light scene can be set to multiple operating modes. In this implementation, in the low-light scene, the sensitivity of the HCG path for obtaining the first HCG image and the sensitivity of the HCG path for obtaining the second HCG image can change with changes in the ambient brightness. In this way, in the low-light scene, as the ambient brightness becomes darker or brighter, the image sensor can also dynamically adjust the sensitivity of the HCG path, so that the brightness of the preview video frame captured by the electronic device 100 in the low-light scene can change with changes in the ambient brightness.

[0280] Optionally, in a low-light scene, as the ambient brightness changes, the sensitivity of the HCG path for obtaining the first HCG image and the sensitivity of the HCG path for obtaining the second HCG image may change by the same amount. For example, as the environment brightens, the sensitivity of the HCG path for obtaining the first HCG image and the sensitivity of the HCG path for obtaining the second HCG image decrease by the same amount. As the environment darkens, the sensitivity of the HCG path for obtaining the first HCG image and the sensitivity of the HCG path for obtaining the second HCG image also increase by the same amount. In this way, the ratio of the sensitivity of the HCG path for obtaining the first HCG image to the sensitivity of the HCG path for obtaining the second HCG image is the same, and the dynamic range of the two preview video frames remains unchanged, ensuring that the image content of the two preview video frames does not change suddenly.

[0281] Optionally, in low-light scenes, as the ambient brightness changes, the exposure duration of the HCG path for obtaining the first HCG image and the exposure duration of the HCG path for obtaining the second HCG image remain constant and are integer multiples of the artificial light source flicker period, where the artificial light source flicker period can be the reciprocal of twice the frequency of the AC power connected to the electronic device 100. This prevents AC flicker, which is when the preview video frame captured by the electronic device 100 flickers over time.

[0282] As can be seen from the above description, the sensitivity of the image sensor is determined by CG, AG, and DG. CG is fixed, and the sensitivity of the image sensor can be adjusted by adjusting the size of AG and / or DG.

[0283] The capacitors used to carry electrons in image sensors can be divided into LOFIC capacitors and non-LOFIC capacitors. If the capacitors used to carry electrons in the image sensor are non-LOFIC capacitors, the sensitivity of the image sensor can be adjusted by adjusting the size of AG and / or DG. If the capacitors used to carry electrons in the image sensor are LOFIC capacitors, the sensitivity of the image sensor can only be adjusted by adjusting the size of DG.

[0284] Preferably, when the capacitor carrying electrons is a non-LOFIC capacitor, the noise caused by adjusting the sensitivity of the image sensor through AG is smaller than the noise caused by adjusting the sensitivity of the image sensor through DG, and the imaging effect is good.

[0285] The following embodiments of this application will take how to adjust AG to adjust the sensitivity of the image sensor as an example to illustrate. For the specific implementation of adjusting the sensitivity of the image sensor by adjusting DG, please refer to the specific implementation of adjusting the sensitivity of the image sensor by adjusting AG, which will not be repeated in this application.

[0286] Before adjusting AG and / or DG, the sensitivity of the LCG and HCG pathways is preset, and the ratio of the sensitivity of the LCG pathway to the sensitivity of the HCG pathway is fixed. For example, the sensitivity of the LCG pathway can be 100*AG*DG, where the value of CG can be 100. The sensitivity of the HCG pathway can be 400*AG*DG, where the value of CG can be 400. The sensitivity of the LCG pathway: the sensitivity of the HCG pathway = 1:4.

[0287] It should be noted that in other embodiments, the initial AG and initial DG of the LCG pathway and the HCG pathway may be different, the initial AG and initial DG of the LCG pathway and the HCG pathway may also be other values, and the CG of the LCG pathway and the HCG pathway may also be other values. This application does not limit this.

[0288] Table 1

[0289] Table 1 shows the values ​​of the image sensor shooting parameters in a low-light scene. The shooting parameters include but are not limited to sensitivity and exposure time.

[0290] In Table 1, when the ambient brightness is less than 200 Lux, electronic device 100 can determine that the current brightness scene is a low-light scene, and the image sensor can operate in low-light high-dynamic mode. When the image sensor operates in low-light high-dynamic mode, the sensitivity of the HCG path of the image sensor for the first HCG image is 400*2AG*DG, and the exposure time is 10ms. The sensitivity of the HCG path of the image sensor for the second HCG image is 400*32AG*DG, and the exposure time is also 10ms.

[0291] For example, when the image sensor determines that it needs to switch from low-dynamic mode to low-light high-dynamic mode, the image sensor can set the exposure time to 10ms and set the sensitivities in the HCG path to 400*2AG*DG and 400*32AG*DG, respectively, to obtain a first HCG image and a second HCG image. The first and second HCG images are then synthesized to obtain a preview video frame obtained by the image sensor in low-dynamic mode.

[0292] In the embodiment shown in Table 1, after the image sensor enters the low-brightness high-dynamic mode, if the ambient brightness becomes darker or brighter, as long as the ambient brightness does not exceed 200 Lux, the image sensor can continue to obtain preview video frames according to the shooting parameters shown in Table 1.

[0293] It should be noted that the values ​​of the image sensor shooting parameters in the low-light high-dynamic mode shown in Table 1 can also be other values, and this application does not limit this.

[0294] Table 2

[0295] Table 2 shows the values ​​of shooting parameters in the image sensor in another low-light scene.

[0296] The embodiment shown in Table 2 further divides the low-light scene into a plurality of different brightness ranges. The sensitivity of the HCG path for obtaining the first HCG image and the second HCG image in different brightness ranges is different.

[0297] For example, the low-light scene can be further divided into four brightness ranges: 200 Lux to 150 Lux, 150 Lux to 100 Lux, 100 Lux to 50 Lux, and 50 Lux to 0 Lux.

[0298] Not limited to the above four brightness ranges, low-light scenes can also be divided into more or fewer brightness ranges, which is not limited in this application.

[0299] When the ambient brightness is between 200 Lux and 150 Lux, the exposure time can be set to 10 ms, and the sensitivity of the HCG channel can be set to 400*AG*DG to obtain the first HCG image. Then, the sensitivity of the HCG channel can be set to 400*AG*DG to obtain the second HCG image.

[0300] When the ambient brightness is between 150 Lux and 100 Lux, the exposure time can be set to 10 ms, and the sensitivity of the HCG channel can be set to 400*2AG*DG to obtain the first HCG image. Then, the sensitivity of the HCG channel can be set to 400*32AG*DG to obtain the second HCG image.

[0301] When the ambient brightness is between 100 Lux and 50 Lux, the exposure time can be set to 10 ms, and the sensitivity of the HCG channel can be set to 400*4AG*DG to obtain the first HCG image. Then, the sensitivity of the HCG channel can be set to 400*64AG*DG to obtain the second HCG image.

[0302] When the ambient brightness is between 50 Lux and 0 Lux, the exposure time can be set to 10 ms, and the sensitivity of the HCG channel can be set to 400*8AG*DG to obtain the first HCG image. Then, the sensitivity of the HCG channel can be set to 400*128AG*DG to obtain the second HCG image.

[0303] Optionally, in a low-light scene, when the ambient brightness dims or brightens, the image sensor adjusts the sensitivity of the HCG path so that the ratio of the sensitivity of the HCG path of the first HCG image to the sensitivity of the second HCG image is the same before and after the adjustment. For example, when the ambient brightness is within the brightness range of 200 Lux to 150 Lux, the ratio of the sensitivity of the HCG path of the first HCG image to the sensitivity of the second HCG image is 1:16. When the ambient brightness is within the brightness range of 150 Lux to 100 Lux, the ratio of the sensitivity of the HCG path of the first HCG image to the sensitivity of the second HCG image is also 1:16. When the ambient brightness is within the brightness range of 100 Lux to 50 Lux, the ratio of the sensitivity of the HCG path of the first HCG image to the sensitivity of the second HCG image is also 1:16. When the ambient brightness is within a range of 50 Lux to 0 Lux, the ratio of the sensitivity of the HCG path for obtaining the first HCG image to the sensitivity of the second HCG image is also 1:16.

[0304] Optionally, in a low-light scene, as the ambient brightness changes, the exposure value of the HCG path for obtaining the first HCG image and the exposure time of the HCG path for obtaining the second HCG image are always 10ms, the AC frequency can be 50hz, and the artificial light source flickering period is 1 / 100=10ms.

[0305] For example, when the image sensor determines that it needs to switch from low dynamic mode to low brightness high dynamic mode. The image sensor needs to determine the shooting parameters of the image sensor based on the ambient brightness. For example, if the current ambient brightness is 140 Lux, the image sensor can determine that the current ambient brightness is within the brightness range of 150 Lux to 100 Lux. The image sensor can set the exposure time to 10 ms and set the sensitivity in the HCG channel to 400*AG*DG and 400*16AG*DG respectively to obtain multiple frames of preview video frames. The image sensor then sets the sensitivity in the HCG channel to 400*2AG*DG and 400*32AG*DG respectively to obtain multiple frames of preview video frames. If the ambient brightness continues to be within the brightness range of 150 Lux to 100 Lux, the image sensor can continue to output preview video frames based on the sensitivities of 400*2AG*DG and 400*32AG*DG.

[0306] In the embodiment shown in Table 2, after the image sensor enters low-light, high-dynamic range mode, if the ambient brightness darkens or brightens, the image sensor can switch to different shooting parameters in real time based on the ambient brightness. For example, when the ambient brightness darkens, the image sensor can increase the sensitivity, and when the ambient brightness brightens, the image sensor can decrease the sensitivity. This allows the brightness of the preview video frames captured by the image sensor in low-light scenes to change with changes in ambient brightness.

[0307] It should be noted that the values ​​of the image sensor shooting parameters in the low-light high-dynamic mode shown in Table 2 can also be other values, and this application does not limit this.

[0308] In some embodiments, the values ​​of the shooting parameters of the image sensor in the low-light scene may be stored in a configuration file, which may be called.

[0309] S913: The image sensor switches from the low dynamic mode to the medium-brightness high dynamic mode, and outputs a preview video frame based on the medium-brightness high dynamic mode.

[0310] When the ambient dynamic range is greater than the dynamic range of the image sensor and the ambient brightness is greater than the first illuminance value, the image sensor may switch to a medium-brightness high-dynamic operating mode. The image sensor may output a preview video frame based on the medium-brightness high-dynamic mode.

[0311] Next, we will introduce how the image sensor switches from low dynamic range mode to medium brightness high dynamic range mode.

[0312] In moderately bright scenes, the signal-to-noise ratio (SNR) of both the HCG and LCG images output by the HCG and LCG pathways is relatively good, resulting in better imaging quality. Therefore, we can generate an HCG image through the HCG pathway and an LCG image through the LCG pathway, respectively, and then synthesize the HCG and LCG images to generate a preview video frame.

[0313] In one possible implementation, the operating mode of the image sensor in a medium-brightness scene can be set to one operating mode. In this implementation, in a medium-brightness scene, the sensitivity of the HCG path and the sensitivity of the LCG path are fixed.

[0314] In other possible implementations, the medium-brightness scene can also be divided into multiple brightness ranges, and the operating mode of the image sensor corresponding to each brightness range is different. In other words, the operating mode of the image sensor in the low-brightness scene can be set to multiple operating modes. In this implementation, in the medium-brightness scene, the sensitivity of the HCG path and the sensitivity of the LCG path can change with the change of the ambient brightness. In this way, in the medium-brightness scene, as the ambient brightness becomes darker or brighter, the image sensor can also dynamically adjust the sensitivity of the HCG path and the LCG path, so that the brightness of the preview video frame captured by the electronic device 100 in the medium-brightness scene can change with the change of the ambient brightness.

[0315] Similar to the principle of adjusting the sensitivity of the image sensor in S912, this application also takes adjusting the sensitivity of the image sensor through AG as an example for explanation.

[0316] Optionally, in moderately bright scenes, as ambient brightness changes, the sensitivity of the HCG and LCG pathways can change by the same amount. For example, as the ambient light brightens, the sensitivity of the HCG and LCG pathways decreases by the same amount. As the ambient light darkens, the sensitivity of the HCG and LCG pathways also increases by the same amount. This ensures that the ratio of the sensitivity of the HCG and LCG pathways remains the same, maintaining the dynamic range of the two preview video frames, and preventing sudden changes in the image content of the two preview video frames.

[0317] Optionally, in a medium-brightness scene, as the ambient brightness changes, the exposure duration of the HCG channel and the exposure duration of the LCG channel remain unchanged and are integer multiples of the artificial light source flicker period, so that AC flicker does not occur.

[0318] Table 3

[0319] Table 3 shows the values ​​of the image sensor shooting parameters in a medium-bright scene. The shooting parameters include but are not limited to sensitivity and exposure time.

[0320] In Table 3, when the ambient brightness is greater than 200 Lux and less than 2000 Lux, the electronic device 100 can determine that the current brightness scene is a medium-bright scene, and the image sensor can operate in medium-brightness high dynamic range mode. When the image sensor operates in medium-brightness high dynamic range mode, the sensitivity of the LCG path used to obtain the LCG image is 100*2AG*DG, and the exposure time is 10ms. The sensitivity of the LCG path used to obtain the HCG image is 400*8AG*DG, and the exposure time is also 10ms.

[0321] For example, when the image sensor determines that it needs to switch from low dynamic range mode to medium-brightness high dynamic range mode, the image sensor may set the exposure time to 10ms, then set the sensitivity of the LCG channel to 100*2AG*DG and the sensitivity of the HCG channel to 400*8AG*DG, thereby obtaining an LCG image and an HCG image. The HCG image and the HCG image are then synthesized to obtain a preview video frame obtained by the image sensor in medium-brightness high dynamic range mode.

[0322] In the embodiment shown in Table 3, after the image sensor enters the medium-brightness high dynamic mode, if the ambient brightness becomes darker or brighter, as long as the ambient brightness is greater than 200 Lux and less than 2000 Lux, the image sensor can continue to obtain preview video frames according to the shooting parameters shown in Table 3.

[0323] It should be noted that the values ​​of the image sensor shooting parameters in the medium-brightness high dynamic mode shown in Table 3 can also be other values, and this application does not limit this.

[0324] Table 4

[0325] Table 4 shows the values ​​of the shooting parameters of the image sensor in another medium-bright scene.

[0326] The embodiment shown in Table 4 further divides the medium-brightness scene into multiple different brightness ranges. The sensitivity of the LCG channel and the HCG channel in different brightness ranges is different.

[0327] For example, the medium-brightness scene can be further divided into four brightness ranges: 2000 Lux to 1400 Lux, 1400 Lux to 800 Lux, 800 Lux to 400 Lux, and 400 Lux to 200 Lux.

[0328] The brightness ranges are not limited to the above four. The medium-brightness scene can also be divided into more or fewer brightness ranges, which is not limited in this application.

[0329] When the ambient brightness is between 2000 Lux and 1400 Lux, the exposure time can be set to 10ms, and the sensitivity of the LCG channel can be set to 100*AG*DG to obtain an LCG image. The sensitivity of the HCG channel can be set to 400*4AG*DG to obtain an HCG image.

[0330] When the ambient brightness is between 1400 Lux and 800 Lux, the exposure time can be set to 10ms, and the sensitivity of the LCG channel can be set to 100*4AG*DG to obtain an LCG image. The sensitivity of the HCG channel can be set to 400*16AG*DG to obtain an HCG image.

[0331] When the ambient brightness is between 800 Lux and 400 Lux, the exposure time can be set to 10ms, and the sensitivity of the LCG channel can be set to 100*16AG*DG to obtain an LCG image. The sensitivity of the HCG channel can be set to 400*64AG*DG to obtain an HCG image.

[0332] When the ambient brightness is between 400 Lux and 200 Lux, the exposure time can be set to 10ms, and the sensitivity of the LCG channel can be set to 100*64AG*DG to obtain an LCG image. The sensitivity of the HCG channel can be set to 400*256AG*DG to obtain an HCG image.

[0333] Optionally, in a medium-brightness scene, when the ambient brightness dims or brightens, the image sensor adjusts the sensitivities of the LCG and HCG channels so that the ratio of the sensitivity of the LCG channel to the sensitivity of the HCG channel is the same before and after the adjustment. For example, when the ambient brightness is within the brightness range of 2000 Lux to 1400 Lux, the ratio of the sensitivity of the LCG channel to the sensitivity of the HCG channel is 1:16. When the ambient brightness is within the brightness range of 1400 Lux to 800 Lux, the ratio of the sensitivity of the LCG channel to the sensitivity of the HCG channel is also 1:16. When the ambient brightness is within the brightness range of 800 Lux to 400 Lux, the ratio of the sensitivity of the LCG channel to the sensitivity of the HCG channel is also 1:16. When the ambient brightness is within the brightness range of 400 Lux to 200 Lux, the ratio of the sensitivity of the LCG channel to the sensitivity of the HCG channel is also 1:16.

[0334] Optionally, in a low-light scene, as the ambient brightness changes, the exposure value of the HCG path for obtaining the first HCG image and the exposure duration of the HCG path for obtaining the second HCG image are always 10 ms.

[0335] For example, when the image sensor determines that it needs to switch from low dynamic range mode to medium-brightness high dynamic range mode, the image sensor needs to determine the values ​​of its shooting parameters based on the ambient brightness. For example, if the current ambient brightness is 1500 Lux, the image sensor can determine that the current ambient brightness is within the brightness range of 1400 Lux to 800 Lux. The image sensor can set the exposure time to 10ms, set the sensitivity of the LCG channel to 100*AG*DG and the sensitivity of the LCG channel to 400*4AG*DG, and obtain multiple preview video frames. The sensitivity of the LCG channel can then be set to 100*4AG*DG and the sensitivity of the LCG channel can be set to 400*16AG*DG, and obtain multiple preview video frames. If the ambient brightness subsequently remains within the brightness range of 1400 Lux to 800 Lux, the image sensor can continue to output preview video frames based on the sensitivities of 100*4AG*DG and 400*16AG*DG.

[0336] In the embodiment shown in Table 4, after the image sensor enters medium-brightness, high-dynamic range mode, if the ambient brightness darkens or brightens, the image sensor can switch to different shooting parameters in real time based on the ambient brightness. For example, when the ambient brightness darkens, the image sensor can increase the sensitivity, and when the ambient brightness brightens, the image sensor can decrease the sensitivity. This allows the brightness of the preview video frames captured by the image sensor in medium-brightness scenes to change with changes in ambient brightness.

[0337] It should be noted that the values ​​of the image sensor shooting parameters in the medium-brightness high dynamic mode shown in Table 4 can also be other values, and this application does not limit this.

[0338] In some embodiments, the values ​​of the shooting parameters of the image sensor in the medium-brightness scene may be stored in a configuration file, which may be called.

[0339] In some embodiments, as can be seen from the description in S911, determining that the ambient dynamic range is greater than the first illuminance value may also include a case where the ambient brightness is greater than the second illuminance value, that is, also includes a case of a highlight scene.

[0340] The image sensor can first be switched from the low dynamic range mode to the medium brightness high dynamic range mode according to the method in Table 3 or Table 4, and then switched from the medium brightness high dynamic range mode to the high brightness high dynamic range mode.

[0341] Next, we will introduce how the image sensor switches from medium-brightness high dynamic range mode to high-brightness high dynamic range mode.

[0342] In high-brightness scenes, the sensitivity and exposure time of the LCG and DCG channels must be kept constant. A preview video frame must be acquired through short exposure, and the overexposure image must be restored through the short-exposure preview video frame.

[0343] Table 5

[0344] Table 5 shows the values ​​of the shooting parameters of the image sensor in a high-brightness scene.

[0345] The embodiment shown in Table 5 divides the high-brightness scene into three different brightness ranges. The exposure time of the short-exposure preview video frame in different brightness ranges is different. If the ambient brightness continues to be within the brightness range of 150Lux to 100Lux, the image sensor can continue to output preview video frames based on the sensitivity of 400*2AG*DG and 400*32AG*DG.

[0346] For example, the high-brightness scene can be further divided into more than three brightness ranges: 2000 Lux to 2600 Lux, 2600 Lux to 3200 Lux, and above 3200 Lux.

[0347] Not limited to the above three brightness ranges, the highlight scene can also be divided into more or fewer brightness ranges, which is not limited in this application.

[0348] When the ambient brightness is between 2000 and 2600 lux, first set the exposure time to 10 ms and the sensitivity of the LCG channel to 100 x AG x DG to produce an LCG image. Set the sensitivity of the HCG channel to 400 x 4 AG x DG to produce an HCG image. Then set the exposure time to 10 ms to produce a short-exposure image frame. First, combine the LCG and HCG images into a long-exposure image frame, and then combine the long-exposure and short-exposure frames into a preview video frame.

[0349] When the ambient brightness is between 2600 and 3200 lux, first set the exposure time to 10 ms and the sensitivity of the LCG channel to 100 x AG x DG to produce an LCG image. Set the sensitivity of the HCG channel to 400 x 4 AG x DG to produce an HCG image. Then set the exposure time to 5 ms to produce a short-exposure image frame. First, combine the LCG and HCG images to form a long-exposure image frame, and then combine the long-exposure and short-exposure frames into a preview video frame.

[0350] When the ambient brightness is above 3200 Lux, first set the exposure time to 10ms and the sensitivity of the LCG channel to 100*AG*DG to produce an LCG image. Set the sensitivity of the HCG channel to 400*4AG*DG to produce an HCG image. Then set the exposure time to 2.5ms to produce a short-exposure image frame. First, combine the LCG and HCG images to form a long-exposure image frame, and then combine the long-exposure and short-exposure frames into a preview video frame.

[0351] In this way, in a bright scene, as the brightness increases, the exposure time of the short-exposure image frame can be gradually reduced to improve the dynamic range of the preview video frame, thereby restoring the bright area content.

[0352] For example, the dynamic range of preview video frames synthesized within the brightness range of 2600 Lux to 3200 Lux is greater than that of preview video frames synthesized within the brightness range of 2000 Lux to 2600 Lux. The dynamic range of preview video frames synthesized within the brightness range of 3200 Lux or above is greater than that of preview video frames synthesized within the brightness range of 2600 Lux to 3200 Lux.

[0353] Exemplarily, when the image sensor determines that it needs to switch from medium-brightness high dynamic mode to medium-brightness high dynamic mode, the image sensor needs to determine the value of the image sensor's shooting parameters based on the ambient brightness. Exemplarily, if the current ambient brightness is 2800 Lux, the image sensor can determine that the current ambient brightness is within the brightness range of 2600 Lux to 3200 Lux. The image sensor can first obtain multiple frames of preview video frames based on the shooting parameters of the image sensor in the brightness range of 2000 Lux to 2600 Lux. Finally, multiple frames of preview video frames are obtained by shooting based on the shooting parameters of the image sensor in the brightness range of 2600 Lux to 3200 Lux. If the ambient brightness continues to be within the brightness range of 2600 Lux to 3200 Lux, the image sensor can continue to output preview video frames based on the shooting parameters of the image sensor in the brightness range of 2600 Lux to 3200 Lux.

[0354] In the embodiment shown in Table 5, after the image sensor enters high-brightness, high-dynamic range mode, if the ambient brightness dims or brightens, the image sensor can switch to different shooting parameters in real time based on the ambient brightness. For example, when the ambient brightness dims, the image sensor can increase the exposure time of the short-exposure image frame. When the ambient brightness brightens, the image sensor can decrease the exposure time of the short-exposure image frame. This allows the brightness of the preview video frame captured by the image sensor in high-brightness scenes to change with changes in ambient brightness.

[0355] S914: The image sensor outputs a preview video frame based on the low-motion mode.

[0356] When the image sensor determines that the dynamic range of the environment is smaller than the dynamic range of the image sensor, the image sensor may operate in a low dynamic mode and output a preview video frame based on the low dynamic mode.

[0357] S915. The CameraUI module displays the preview video frame output in the low-brightness high-dynamic mode in the preview window.

[0358] For example, if the image sensor has switched to low-brightness, high-dynamic range mode before starting to shoot a video, the image sensor can obtain a preview video frame based on the shooting parameters shown in Table 1 or Table 2 and send the preview video frame to the CameraUI module. The CameraUI module can display the preview video frame output in the low-brightness, high-dynamic range mode in the preview window.

[0359] In this way, the image sensor has already switched to the high dynamic range mode before starting to record the video. After starting to record the video, the image sensor can directly record the video in the high dynamic range mode.

[0360] The video recording interface is displayed and video recording starts (S916-S924).

[0361] S916. The CameraUI module receives a second input operation from the user.

[0362] S917 . In response to the second input operation, the image sensor outputs a recorded video frame based on the low-brightness high-dynamic mode.

[0363] The implementation method of the image sensor outputting recorded video frames based on the low-light high-dynamic mode is similar to the implementation method of the image sensor outputting preview video frames based on the low-light high-dynamic mode. Please refer to the description in S912, and this application will not repeat it here.

[0364] S918. In response to the second input operation, the CameraUI module displays a video recording interface.

[0365] S919. The CameraUI module displays the recorded video frames output based on the low-brightness high-dynamic mode in the preview window of the video recording interface.

[0366] Exemplarily, the second input operation may be a user operation on the start recording control 812 shown in FIG8D .

[0367] In response to the second input operation, the electronic device 100 starts recording a video and displays the video recording interface shown in FIG. 8E . The CameraUI module may display the recorded video frames output based on the low-brightness high-dynamic mode in the preview window 723 .

[0368] S920: The image sensor sends the recorded video frames output based on the low-brightness high-dynamic mode to the image storage module.

[0369] In response to the second operation, the image sensor sends the recorded video frame output based on the low-brightness high-dynamic mode to the image storage module, so that the image storage module can save the recorded video frame output based on the low-brightness high-dynamic mode.

[0370] S921. When the ambient brightness is greater than the first illuminance value and less than the second illuminance value, the image sensor switches from the low-brightness high-dynamic mode to the medium-brightness high-dynamic mode, and outputs a recorded video frame based on the medium-brightness high-dynamic mode.

[0371] Optionally, if the image sensor determines that the ambient brightness is greater than the first illuminance value and less than the second illuminance value for m consecutive video frames, the image sensor may switch from the low-brightness high dynamic range mode to the medium-brightness high dynamic range mode, and output the recorded video frames based on the medium-brightness high dynamic range mode. m is a positive integer greater than or equal to 1.

[0372] The implementation method of the image sensor outputting the recorded video frame based on the medium-brightness high dynamic mode is similar to the implementation method of the image sensor outputting the preview video frame based on the medium-brightness high dynamic mode. Please refer to the description in S913, and this application will not repeat it here.

[0373] S922: The image sensor outputs a recorded video frame based on the medium-brightness high-dynamic mode and sends it to the image storage module.

[0374] After the image sensor enters the medium-brightness high dynamic mode, the image sensor can output recorded video frames based on the medium-brightness high dynamic mode, and send the recorded video frames output in the medium-brightness high dynamic mode to the image storage module, so that the image storage module can save the recorded video frames output based on the medium-brightness high dynamic mode.

[0375] S923: When the ambient brightness is greater than the second illumination value, the image sensor switches from the medium-brightness high-dynamic mode to the high-brightness high-dynamic mode, and outputs a recorded video frame based on the high-brightness high-dynamic mode.

[0376] Optionally, if the image sensor determines that the ambient brightness is greater than the second illuminance value for m consecutive video frames, the image sensor may switch from the medium-brightness high dynamic range mode to the high-brightness high dynamic range mode, and output the recorded video frames based on the high-brightness high dynamic range mode. m is a positive integer greater than or equal to 1.

[0377] The implementation method of the image sensor outputting the recorded video frame based on the highlight high dynamic mode is similar to the implementation method of the image sensor outputting the preview video frame based on the highlight high dynamic mode. Please refer to the description in S913, and this application will not repeat it here.

[0378] S924: The image sensor outputs the recorded video frame based on the high-brightness and high-dynamic mode and sends it to the image storage module.

[0379] After the image sensor enters the highlight high dynamic mode, the image sensor can output recorded video frames based on the highlight high dynamic mode, and send the recorded video frames output in the highlight high dynamic mode to the image storage module, so that the image storage module can save the recorded video frames output based on the highlight high dynamic mode.

[0380] In some embodiments, after S919 - S920 , when the image sensor detects that the ambient brightness is greater than the second illumination value, the image sensor may first execute S921 - S922 and then execute S923 - S924 .

[0381] In other embodiments, after S919-S920, when the image sensor detects that the ambient brightness is greater than the second illumination value, the image sensor may not execute S921-S922 and directly execute S923-S924.

[0382] Optionally, S919-S924 are merely examples and are not limiting. The image sensor may switch to different operating modes in real time based on the monitored ambient brightness.

[0383] FIG10 shows a schematic diagram of an image sensor switching between different operating modes.

[0384] As shown in FIG10 , the operating modes of the image sensor include but are not limited to a low dynamic mode, a low brightness high dynamic mode, a medium brightness high dynamic mode, and a high brightness high dynamic mode.

[0385] After the electronic device 100 opens the camera application and enters the "recording" mode, the image sensor operates in the low dynamic mode. The image sensor can monitor the dynamic range and brightness of the environment in real time to determine whether to switch the working mode.

[0386] In some embodiments, after the image sensor enters the low dynamic mode, if the image sensor detects that the ambient dynamic range is greater than the image sensor dynamic range and the ambient brightness is greater than the first illuminance value, the image sensor can switch from the low dynamic mode to the medium-brightness high dynamic mode and output a video frame based on the medium-brightness high dynamic mode.

[0387] After the image sensor enters the medium-brightness high dynamic mode, the image sensor can also dynamically switch to other working modes based on the ambient brightness.

[0388] Optionally, after the image sensor enters the medium-brightness high-dynamic mode, if the image sensor detects that the dynamic range of the environment is smaller than the dynamic range of the image sensor, the image sensor can switch to the low-dynamic mode and output video frames based on the low-dynamic mode.

[0389] Optionally, after the image sensor enters the medium-brightness high-dynamic mode, if the image sensor detects that the ambient dynamic range is greater than the image sensor dynamic range and the ambient brightness is less than the first illuminance value, the image sensor can switch from the medium-brightness high-dynamic mode to the low-brightness high-dynamic mode and output video frames based on the low-brightness high-dynamic mode.

[0390] Optionally, after the image sensor enters the medium-brightness high-dynamic mode, if the image sensor detects that the ambient dynamic range is greater than the image sensor dynamic range and the ambient brightness is greater than the second illuminance value, the image sensor can switch from the medium-brightness high-dynamic mode to the high-brightness high-dynamic mode and output video frames based on the high-brightness high-dynamic mode.

[0391] In other embodiments, after the image sensor enters the low dynamic mode, if the image sensor detects that the ambient dynamic range is greater than the image sensor dynamic range and the ambient brightness is less than the first illuminance value, the image sensor can switch from the low dynamic mode to the low-brightness high dynamic mode and output video frames based on the low-brightness high dynamic mode.

[0392] After the image sensor enters the low-light high-dynamic mode, the image sensor can also dynamically switch to other working modes based on the ambient brightness.

[0393] Optionally, after the image sensor enters the low-light high-dynamic mode, if the image sensor detects that the dynamic range of the environment is smaller than the dynamic range of the image sensor, the image sensor can switch to the low-dynamic mode again and output video frames based on the low-dynamic mode.

[0394] Optionally, after the image sensor enters the low-brightness high-dynamic mode, if the image sensor detects that the ambient dynamic range is greater than the image sensor dynamic range and the ambient brightness is greater than the first illuminance value and less than the second illuminance value, the image sensor can switch from the low-brightness high-dynamic mode to the medium-brightness high-dynamic mode and output video frames based on the medium-brightness high-dynamic mode.

[0395] Optionally, after the image sensor enters the low-brightness high-dynamic mode, if the image sensor detects that the ambient dynamic range is greater than the image sensor's dynamic range and the ambient brightness is greater than a second illuminance value, in one possible implementation, the image sensor may first switch from the low-brightness high-dynamic mode to the medium-brightness high-dynamic mode, and then switch from the medium-brightness high-dynamic mode to the high-brightness high-dynamic mode. In other possible implementations, the image sensor may directly switch from the low-brightness high-dynamic mode to the high-brightness high-dynamic mode.

[0396] After the image sensor enters the high-brightness high-dynamic mode, the image sensor can also dynamically switch to other working modes based on the ambient brightness.

[0397] Optionally, if the image sensor detects that the ambient brightness is greater than the first illuminance value and less than the second illuminance value, the image sensor can switch from the high-brightness high-dynamic mode to the medium-brightness high-dynamic mode and output video frames based on the medium-brightness high-dynamic mode.

[0398] Optionally, if the image sensor detects that the ambient brightness is less than the second illuminance value, in one possible implementation, the image sensor may first switch from the high-brightness high-dynamic mode to the medium-brightness high-dynamic mode, and then switch from the medium-brightness high-dynamic mode to the low-brightness high-dynamic mode. In other possible implementations, the image sensor may directly switch from the high-brightness high-dynamic mode to the low-brightness high-dynamic mode.

[0399] Finish recording the video, display the video preview interface, and save the recorded video (S925-S930).

[0400] S925. The CameraUI module receives a third input operation from the user.

[0401] For example, the third input operation may be an input operation (eg, a single click) for the end recording control 814. In response to the user's input operation, the electronic device 100 may switch the end recording control 814 to the start recording control 812 and save the recorded video.

[0402] S926. The image storage module saves the multiple recorded video frames and generates a first video based on the multiple recorded video frames.

[0403] In response to the third input operation, the image storage module saves the multiple recorded video frames and generates a first video based on the multiple recorded video frames. The image storage module can generate the first video based on the time sequence of the multiple recorded video frames, and the first video includes the multiple recorded video frames.

[0404] S927: In response to the third input operation, the CameraUI module displays a video preview interface.

[0405] In response to the third input operation, after stopping the video recording, the CameraUI module may display a video preview interface and display a preview video frame in a preview window of the video preview interface.

[0406] S928 . In response to the third input operation, the image sensor outputs a preview video frame based on the highlight-high dynamic mode.

[0407] Exemplarily, if the image sensor operates in the highlight high dynamic mode before the CameraUI module receives the third input from the user, when the CameraUI module receives the third input from the user, the image sensor will output a preview video frame based on the highlight high dynamic mode.

[0408] S929. The image sensor sends the preview video frame output based on the highlight high dynamic mode to the CameraUI module.

[0409] S930: The CameraUI module displays the preview video frame output by the image sensor based on the highlight high dynamic mode in the preview window.

[0410] Optionally, after the CameraUI module displays the preview video frame output by the image sensor based on the highlight high dynamic mode in the preview window, the electronic device 100 may execute the embodiments shown in S908-S914 to display the preview video frame in the preview window.

[0411] The specific implementation of this embodiment will be described below from the perspective of collaboration between software and hardware of an electronic device with reference to FIG11 .

[0412] As shown in Figure 11, a camera captures images. When light reflected from the subject passes through the lens, it is refracted by the mirror and converges on the image sensor. The image sensor converts the light signal into an analog electrical signal. This analog electrical signal bypasses the image sensor and the sensor front-end (SFE) and is then output through a digital-to-analog converter. The output of the digital-to-analog converter is the original digital image captured by the camera, i.e., the RAW video frame.

[0413] In low-brightness high dynamic range mode and medium-brightness high dynamic range mode, the image sensor can generate two video frames. In one possible implementation, the image sensor can combine the two video frames into one video frame. In other possible implementations, the image sensor can send the two video frames to the sensor front end, which combines the two video frames into one video frame.

[0414] In high-brightness and high-dynamic mode, the image sensor can generate three video frames. Two of them are long-exposure video frames, and one is short-exposure video frame. In one possible implementation, the image sensor combines the two long-exposure video frames into one video frame, and then sends the combined video frame to the sensor front end. The sensor front end then combines the combined video frame and the short-exposure video frame into one video frame. In other possible implementations, the image sensor can send the two long-exposure video frames and the short-exposure video frame to the sensor front end, and the sensor front end then combines the two long-exposure video frames and the short-exposure video frame into one video frame.

[0415] The image signal processor (ISP) processes the RAW video frames from the camera and generates video frames to be displayed. The video frames to be displayed can be in YUV (or luminance and chrominance) format. The image signal processor then sends the YUV formatted video frames to the display screen for display. The ISP includes an image processing engine (IPE), an image front-end engine (IFE), and an I / O control interface.

[0416] First, the process of processing the video frame at the front end of the image processor may include one or more of the following, for example: bad pixel correction processing, RAW domain noise reduction processing, black level correction processing, optical shading correction processing, automatic white balance processing, color interpolation processing, color correction processing, global tone mapping processing, or image conversion processing, etc. The embodiment of the present application does not limit the video frame pre-processing process.

[0417] The image processor front end then sends the pre-processed video frames to the image processor back end, which can perform post-processing on the pre-processed video frames. On the one hand, the image processor back end can use the post-processed video frame sequence as a preview video stream or a recorded video stream. The preview video stream or the recorded video stream can be displayed on the display screen through the I / O control interface. On the other hand, the image processor back end can send the post-processed video frame sequence to the perception module, so that the perception module can obtain the ambient dynamic range and ambient brightness based on the post-processed video frame sequence. The perception module then sends the ambient dynamic range and ambient brightness to the image sensor, so that the image sensor can switch different working modes based on the ambient dynamic range and ambient brightness.

[0418] Optionally, the camera may further include a light metering sensor, which may monitor ambient brightness and ambient dynamic range.

[0419] In some embodiments, during the process of recording a video, the electronic device 100 receives a user input operation (such as a single click) on the control 813, and the electronic device 100 can save the recorded video frame corresponding to the moment when the user clicks the control 813 in the gallery.

[0420] The video frame post-processing may include one or more of the following, for example: image correction and adjustment processing, local tone mapping processing, and gamma correction processing. In the image correction and adjustment processing, the image processor backend can perform anti-shake processing on the current video frame. In the local tone mapping processing, the image processor backend can adjust the overall brightness of the video frame so that the picture after brightness adjustment can be closer to the brightness presented in the real world. In the gamma correction processing, the image processor backend can adjust the brightness of the video frame so that it can retain more bright and dark details, compress the contrast, and retain more color information.

[0421] When the electronic device 100 receives the user's input operation (such as a single click) for the end video recording control 814, the recorded video stream can be saved in the gallery.

[0422] FIG12 is a flow chart of a video shooting method provided in this application.

[0423] S1201: The electronic device displays a shooting interface, where the shooting interface is used to display images captured by a camera.

[0424] The electronic device includes a camera module, which includes an image sensor and a dual-conversion-gain sensor. The dual-conversion-gain sensor includes a first sensor and a second sensor, and the conversion gain of the first sensor is smaller than the conversion gain of the second sensor.

[0425] The dual conversion gain sensor may be located inside the image sensor, or may be independent of the image sensor.

[0426] The first sensor may be referred to as a low conversion gain sensor, and the second sensor may be referred to as a high conversion gain sensor.

[0427] In some embodiments, the first sensor may also be referred to as an LCG pathway, and the second sensor may also be referred to as an HCG pathway.

[0428] The photographing interface may be the photographing interface 720 shown in FIG. 8B .

[0429] The shooting interface may also be the video recording interface 810 shown in FIG8C .

[0430] The shooting interface may also be the video recording interface 810 shown in FIG8E .

[0431] S1202. When the ambient brightness is less than a first illuminance value, the electronic device processes a first original image output by the image sensor into a first image according to a first sensitivity through a second sensor, and processes the first original image into a second image according to a second sensitivity through a second sensor, where the first sensitivity is less than the second sensitivity.

[0432] S1203: The electronic device synthesizes the first image and the second image into a third image, and displays the third image on the shooting interface.

[0433] In some embodiments, the first image may be referred to as a first HCG image.

[0434] In some embodiments, the second image may be referred to as a second HCG image.

[0435] Illustratively, the first sensitivity may be 400*2AG*DG as shown in Table 1.

[0436] Illustratively, the second sensitivity may be 400*32AG*DG as shown in Table 1.

[0437] Illustratively, the first sensitivity may also be a plurality of sensitivities shown in Table 2 for obtaining the first HCG image.

[0438] Illustratively, the second sensitivity may also be a plurality of those shown in Table 2 for obtaining the second HCG image.

[0439] S1204. When the ambient brightness is between the first illuminance value and the second illuminance value, the electronic device processes the second original image output by the image sensor into a fourth image according to the third sensitivity through the first sensor, and processes the second original image into a fifth image according to the fourth sensitivity through the second sensor, where the third sensitivity is smaller than the fourth sensitivity, and the fourth sensitivity is smaller than the second sensitivity.

[0440] S1205: The electronic device combines the fourth image and the fifth image into a sixth image, and displays the sixth image on the shooting interface.

[0441] The first illuminance value and the second illuminance value may be a specific value or a range.

[0442] For example, the first illuminance value may be 200 Lux to 0 Lux.

[0443] The second illumination value may be 2000 Lux to 200 Lux.

[0444] Here, 200 Lux can be the ambient brightness of a low-brightness scene or the ambient brightness of a medium-brightness scene, and this application does not limit this.

[0445] In some embodiments, the fourth image may be referred to as an LCG image.

[0446] In some embodiments, the fifth image may be referred to as an HCG image.

[0447] Illustratively, the third sensitivity may be 100*2AG*DG as shown in Table 3.

[0448] Illustratively, the fourth sensitivity may be 400*8AG*DG as shown in Table 3.

[0449] Exemplarily, the third sensitivity may also be a plurality of sensitivities for obtaining an LCG image as shown in Table 4.

[0450] Illustratively, the fourth sensitivity may also be a plurality of sensitivities for obtaining the HCG image shown in Table 4.

[0451] Through this method, on the one hand, the dynamic range of the electronic device during video shooting can be improved. On the other hand, the electronic device can automatically switch between different shooting modes based on the ambient brightness. Specifically, in low-light scenes, the signal-to-noise ratio of the first sensor is low, and the signal-to-noise ratio of the second sensor is relatively high. By outputting two frames of images through the second sensor to synthesize a video frame, there will be no noise stratification problem, and the imaging effect of the video frame is also good. In medium-light scenes, the signal-to-noise ratio of the first sensor is comparable to that of the second sensor, and the signal-to-noise ratio of the first sensor is also relatively high. By outputting two frames of images through the first sensor and the second sensor to synthesize a video frame, the imaging effect of the video frame is also relatively good.

[0452] In one possible implementation, the method also includes: when the ambient brightness is greater than or equal to the second illuminance value, the electronic device controls the image sensor to output the third original image with the first exposure time, and controls the image sensor to output the fourth original image with the second exposure time; the electronic device processes the third original image into a seventh image according to the third sensitivity through the first sensor, and processes the third original image into an eighth image according to the fourth sensitivity through the second sensor; the electronic device processes the fourth original image into a ninth image based on a merging method; the electronic device synthesizes the seventh image, the eighth image and the ninth image into a tenth image, and displays the tenth image on the shooting interface.

[0453] In some embodiments, the seventh image may be referred to as an LCG image.

[0454] In some embodiments, the eighth image may be referred to as an HCG image.

[0455] In some embodiments, the ninth image may be referred to as a short exposure preview video frame or a short exposure recording video frame.

[0456] In this way, in bright scenes, long-exposure frames (the seventh and eighth images) and a short-exposure frame (the ninth image) are output. The seventh, eighth, and ninth images are then fused to create a single image, the tenth image. This solves the problem of overexposed images caused by overflowing the full well capacity of a single pixel in bright scenes. The overexposed image can be restored using the tenth image.

[0457] In one possible implementation, the shooting interface also includes a recording end button; the method also includes: the electronic device receives a first operation of the user on the recording end button; in response to the first operation, the electronic device saves a first video, the first video includes the third image and the sixth image, or the first video includes the third image, the sixth image and the tenth image.

[0458] Exemplarily, the recording end button may be the recording end control 814 shown in FIG8E .

[0459] In this way, the method can be applied to video shooting scenarios, and videos can be shot based on the method.

[0460] In a possible implementation, the shooting interface further includes a video recording start button.

[0461] Exemplarily, the video recording start button may be the video recording start control 812 shown in FIG8C .

[0462] In this way, the method can be applied to video shooting preview scenarios. In the video shooting preview interface, the electronic device first works in the low-dynamic mode. When it is determined that the dynamic range of the environment is greater than the dynamic range of the image sensor, the electronic device can switch to a high-dynamic mode, such as a low-brightness high-dynamic mode, a medium-brightness high-dynamic mode, or a high-brightness high-dynamic mode. Then, after the electronic device enters the video shooting interface, the electronic device can directly shoot and save the video in the low-brightness high-dynamic mode, the medium-brightness high-dynamic mode, or the high-brightness high-dynamic mode.

[0463] In a possible implementation, the method further includes: when the ambient brightness is between the first illuminance value and the second illuminance value, the brighter the ambient brightness is, the smaller the sensitivity of the first sensor and the sensitivity of the second sensor are.

[0464] In this way, in low-light scenes, as the environment brightens, the sensitivity of the second sensor that obtains the first image and the sensitivity of the second sensor that obtains the second image can be gradually reduced. As the environment darkens, the sensitivity of the second sensor that obtains the first image and the sensitivity of the second sensor that obtains the second image can be gradually increased. In this way, the video frames captured by the electronic device also gradually brighten or darken, which can improve the video shooting effect.

[0465] For details, please refer to the description in the embodiment of Table 2.

[0466] In a possible implementation, the method further includes: when the ambient brightness is between the first illuminance value and the second illuminance value, the brighter the ambient brightness is, the smaller the sensitivity of the first sensor and the sensitivity of the second sensor are.

[0467] In this way, in a medium-bright scene, as the environment becomes brighter, the sensitivity of the first sensor and the second sensor can be gradually reduced. As the environment becomes darker, the sensitivity of the first sensor and the second sensor can be gradually increased. In this way, the video frames captured by the electronic device also gradually brighten or darken, which can improve the video shooting effect.

[0468] For details, please refer to the description in the embodiment of Table 4.

[0469] In a possible implementation, exposure durations of the first sensor and the second sensor are both the first exposure duration.

[0470] In this way, during the mode switching process, the exposure time of the first sensor and the second sensor remains unchanged, and only the sensitivity is changed, thereby avoiding flickering between two adjacent video frames and affecting the visual effect.

[0471] For example, the first exposure duration may be 10 ms.

[0472] In a possible implementation, a ratio between the sensitivity of the second sensor that obtains the first image and the sensitivity of the second sensor that obtains the second image remains unchanged.

[0473] In this way, in a low-light scene, as the ambient brightness becomes brighter or darker, the sensitivity of the second sensor that obtains the first image and the sensitivity of the second sensor that obtains the second image increase or decrease proportionally, avoiding differences in dynamic range.

[0474] Exemplarily, the ratio between the sensitivity of the second sensor for obtaining the first image and the sensitivity of the second sensor for obtaining the second image is 1:16.

[0475] In a possible implementation, the ratio between the sensitivity of the first sensor and the sensitivity of the second sensor remains unchanged.

[0476] In this way, in a medium-brightness scene, as the ambient brightness becomes brighter or darker, the sensitivity of the first sensor and the second sensor increases or decreases proportionally, avoiding a difference in dynamic range.

[0477] Exemplarily, the ratio between the sensitivity of the first sensor and the sensitivity of the second sensor is 1:16.

[0478] Thus, before and after the sensor mode is switched, the ratio between the sensitivity of the first sensor and the sensitivity of the second sensor is 1:16. The ratio between the sensitivity of the second sensor that obtains the first image and the sensitivity of the second sensor that obtains the second image is also 1:16.

[0479] In a possible implementation, the method further includes: when the ambient brightness is greater than or equal to the second illumination value, the brighter the ambient brightness is, the longer the first exposure time remains, and the shorter the second exposure time becomes.

[0480] In this way, in a bright scene, as the environment becomes brighter, the exposure time for the ninth image can be gradually reduced. As the environment becomes darker, the exposure time for the ninth image can be gradually increased. In this way, the video frames captured by the electronic device also gradually brighten or darken, which can improve the video shooting effect.

[0481] For details, please refer to the description in the embodiment of Table 5.

[0482] In a possible implementation manner, the method further includes: when the ambient brightness is equal to the second illuminance value, the first exposure duration is equal to the second exposure duration.

[0483] Exemplarily, the second illuminance value may be 2000 Lux to 2600 Lux.

[0484] The first exposure time and the second exposure time are 10ms.

[0485] In one possible implementation, the method includes: the electronic device adjusts the sensitivity of the first sensor by adjusting the analog gain and / or digital gain of the first sensor; the electronic device adjusts the sensitivity of the second sensor by adjusting the analog gain and / or digital gain of the second sensor.

[0486] In one possible implementation, the first sensor also includes a first capacitor, and the electronic device adjusts the sensitivity of the first sensor by adjusting the analog gain and / or digital gain of the first sensor. Specifically, when the first capacitor is a LOFIC capacitor, the electronic device adjusts the sensitivity of the first sensor by adjusting the digital gain of the first sensor; when the first capacitor is not a LOFIC capacitor, the electronic device adjusts the sensitivity of the first sensor by adjusting the analog gain and / or digital gain of the first sensor.

[0487] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in this application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive).

[0488] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0489] In short, the above description is only an embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made based on the disclosure of the present invention should be included in the scope of protection of the present invention.

[0490] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A video shooting method, characterized in that: The electronic device includes a camera module, the camera module includes an image sensor and a dual conversion gain sensor, the dual conversion gain sensor includes a first sensor and a second sensor, the conversion gain of the first sensor is less than the conversion gain of the second sensor, and the method includes: The electronic device displays a shooting interface, where the shooting interface is used to display the image captured by the camera; When the ambient brightness is less than a first illumination value, the electronic device processes a first original image output by the image sensor into a first image through the second sensor according to a first sensitivity, and processes the first original image into a second image through the second sensor according to a second sensitivity, wherein the first sensitivity is less than the second sensitivity; The electronic device combines the first image and the second image into a third image, and displays the third image on the shooting interface; When the ambient brightness is between the first illumination value and the second illumination value, the electronic device processes the second original image output by the image sensor into a fourth image through the first sensor according to the third sensitivity, and processes the second original image into a fifth image through the second sensor according to the fourth sensitivity, wherein the third sensitivity is less than the fourth sensitivity, and the fourth sensitivity is less than the second sensitivity; The electronic device synthesizes the fourth image and the fifth image into a sixth image, and displays the sixth image on the shooting interface.

2. The method according to claim 1, characterized in that The method further comprises: When the ambient brightness is greater than or equal to the second illumination value, the electronic device controls the image sensor to output a third original image with the first exposure duration, and controls the image sensor to output a fourth original image with the second exposure duration; The electronic device processes the third original image into a seventh image by using the first sensor according to the third sensitivity, and processes the third original image into an eighth image by using the second sensor according to the fourth sensitivity; The electronic device processes the fourth original image into a ninth image based on a merging manner; The electronic device synthesizes the seventh image, the eighth image and the ninth image into a tenth image, and displays the tenth image on the shooting interface.

3. The method according to claim 1 or 2, characterized in that: The shooting interface also includes a video recording end button; the method also includes: The electronic device receives a first operation of the user on the video recording end button; In response to the first operation, the electronic device saves a first video, the first video including the third image and the sixth image, or the first video including the third image, the sixth image, and the tenth image.

4. The method according to claim 1 or 2, characterized in that: The shooting interface also includes a video recording start button.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the ambient brightness is less than the first illumination value, the brighter the ambient brightness is, the smaller the sensitivity of the second sensor for obtaining the first image and the sensitivity of the second sensor for obtaining the second image are.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the ambient brightness is between the first illuminance value and the second illuminance value, the brighter the ambient brightness is, the smaller the sensitivity of the first sensor and the sensitivity of the second sensor are.

7. The method according to claim 5 or 6, characterized in that: The exposure durations of the first sensor and the second sensor are both first exposure durations.

8. The method according to claim 5, characterized in that The ratio between the sensitivity of the second sensor for obtaining the first image and the sensitivity of the second sensor for obtaining the second image remains unchanged.

9. The method according to claim 6, characterized in that The ratio between the sensitivity of the first sensor and the sensitivity of the second sensor remains unchanged.

10. The method according to claim 2, characterized in that The method further comprises: When the ambient brightness is greater than or equal to the second illumination value, the brighter the ambient brightness is, the first exposure time remains unchanged and the second exposure time becomes shorter.

11. The method according to claim 2 or 10, characterized in that: The method further comprises: When the ambient brightness is equal to the second illumination value, the first exposure duration is equal to the second exposure duration.

12. The method according to any one of claims 1 to 11, characterized in that: The method comprises: The electronic device adjusts the sensitivity of the first sensor by adjusting the analog gain and / or digital gain of the first sensor; The electronic device adjusts the sensitivity of the second sensor by adjusting the analog gain and / or the digital gain of the second sensor.

13. The method according to claim 12, wherein the first sensor further comprises a first capacitor, characterized in that: The electronic device adjusts the sensitivity of the first sensor by adjusting the analog gain and / or the digital gain of the first sensor, specifically including: In the case where the first capacitor is a LOFIC capacitor, the electronic device adjusts the sensitivity of the first sensor by adjusting the digital gain of the first sensor; In the case that the first capacitor is not a LOFIC capacitor, the electronic device adjusts the sensitivity of the first sensor by adjusting the analog gain and / or digital gain of the first sensor.

14. An electronic device, characterized in that: The electronic device includes a processor and a memory; the memory is coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, and the processor calls the computer instructions to execute the method described in any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that: Used to store computer instructions, when the computer instructions are executed on an electronic device, the server executes the method described in any one of claims 1 to 13.

16. A computer program product, characterized in that When the computer program product is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 13.

Images