Photographing method, electronic device and storage medium

Abstract

Provided in the embodiments of the present application are a photographing method, an electronic device and a storage medium. The method can be applied to an electronic device that supports bokeh capturing, wherein the electronic device is, for example, a smart phone or a tablet computer. The bokeh capturing refers to performing depth calculation and portrait blurring processing on a captured image, and optionally further performing dynamic range fusion processing. The method may comprise: in response to a detected user operation acting on a bokeh capturing control, starting bokeh capturing and calling a camera apparatus to collect preview stream data of a photographing scene; determining statistical information of the photographing scene on the basis of the preview stream data; in response to the statistical information satisfying a mode switching condition, switching the bokeh capturing processing mode from a first processing mode to a second processing mode; and in response to a bokeh capturing instruction for the photographing scene, on the basis of the second processing mode, processing an image to be processed, so as to obtain a bokeh-captured image of the photographing scene. By using the embodiments of the present application, the power consumption of an electronic device can be reduced in a bokeh capturing scenario.

Description

Shooting method, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on January 5, 2024, with application number 202410026720.9 and application name “A shooting method, electronic device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of electronic devices, and in particular to a shooting method, electronic device, and storage medium. Background Art

[0003] With the advancement of photography technology, electronic devices with camera functions now offer a wide variety of shooting methods, such as portraits, night scenes, selfies, videos, and snapshots. Snapshots refer to the ability of electronic devices with camera functions to quickly capture natural, vivid, and expressive images of a subject without the subject's knowledge. Examples include capturing a momentary image of a person, a traffic violation, or a spectacular goal in a soccer match.

[0004] Compared with other shooting methods, snapshot shooting will increase the power consumption of electronic devices to a certain extent. Therefore, how to save the power consumption of electronic devices in snapshot shooting is a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The embodiments of the present application provide a shooting method, an electronic device, and a storage medium, which can save power consumption of the electronic device in a blurred snapshot scenario.

[0006] In a first aspect, an embodiment of the present application provides a shooting method, which can be applied to an electronic device, and the electronic device can support blurred snapshot. Blurred snapshot means performing depth calculation and portrait blur processing on the captured image, and optionally, performing dynamic range fusion processing. Blurred snapshot can improve the clarity of the subject (such as a portrait) compared to ordinary snapshot. The method may include: detecting a user operation acting on a blurred snapshot control in the user interface of a camera application, and in response to the user operation, starting blurred snapshot and calling a camera device to collect preview stream data of the shooting scene; based on the preview stream data, determining statistical information of the shooting scene; in response to the statistical information meeting the mode switching condition, switching the blurred snapshot processing mode from the first processing mode to the second processing mode; in response to the blurred snapshot instruction for the shooting scene, calling the camera device to shoot and obtain the image to be processed of the shooting scene; based on the second processing mode, processing the image to be processed to obtain a blurred snapshot image of the shooting scene; and displaying a thumbnail of the blurred snapshot image of the shooting scene in the thumbnail display area of ​​the user interface of the camera application.

[0007] By implementing the shooting method provided in the first aspect, in a blurred snapshot scenario, when the statistical information of the shooting scene meets the mode switching conditions, the blurred snapshot processing mode can be switched from the first processing mode to the second processing mode, and then the captured image to be processed can be processed based on the second processing mode, thereby saving power consumption of the electronic device in the blurred snapshot scenario.

[0008] In conjunction with the method provided in the first aspect, in some embodiments, the blur capture processing mode includes a high dynamic range mode and a depth calculation mode, and the high dynamic range mode in the second processing mode is different from the high dynamic range mode in the first processing mode, and / or the depth calculation mode in the second processing mode is different from the depth calculation mode in the first processing mode. This allows the electronic device to adopt different high dynamic range modes and / or different depth calculation modes in different shooting scenarios, thereby helping to save power consumption of the electronic device.

[0009] The HDR mode refers to the sensor's HDR mode, which can be divided into interleaved HDR mode and dual-gain HDR mode. The depth calculation mode can be divided into binocular depth calculation mode and monocular depth calculation mode.

[0010] In combination with the method provided in the first aspect, in some embodiments, the high dynamic range mode in the above-mentioned first processing mode is an interlaced high dynamic range mode, and the depth calculation mode in the first processing mode is a binocular depth calculation mode. The power consumption in the interlaced high dynamic range mode is greater than that in the dual-gain high dynamic range mode, and the power consumption in the binocular depth calculation mode is greater than that in the monocular depth calculation mode. That is, the power consumption of the electronic device using the first processing mode is greater, which will accelerate the consumption of power. Switching from the first processing mode to the second processing mode can slow down the consumption of power, thereby saving the power consumption of the electronic device.

[0011] In conjunction with the method provided in the first aspect, in some embodiments, the statistical information includes scene brightness statistics and dynamic range statistics, and the statistical information satisfies a mode switching condition including the scene brightness statistics being less than a scene brightness threshold and the dynamic range statistics being less than a dynamic range threshold. In this case, the high dynamic range mode in the blur capture processing mode is switched from the interleaved high dynamic range mode to the dual-gain high dynamic range mode to save power consumption of the electronic device.

[0012] It is understood that the high dynamic range mode in the second processing mode is the dual-gain high dynamic range mode, and the depth calculation mode in the second processing mode can be a binocular depth calculation mode (i.e., the same as the depth calculation mode in the first processing mode) or a monocular depth calculation mode (i.e., switching from the binocular depth calculation mode to the monocular depth calculation mode). Switching from the binocular depth calculation mode to the monocular depth calculation mode can be triggered by the statistical value of the medium depth of field in the subject outline layer.

[0013] Among them, the scene brightness threshold and dynamic range threshold can be understood as pre-calibrated thresholds, which can be set in the electronic device when it leaves the factory, or can be set based on the numerical value input by the user. The specific numerical value is not limited in the embodiments of this application.

[0014] In conjunction with the method provided in the first aspect, in some embodiments, the statistical information includes flicker intensity detection statistics, and the statistical information meeting the mode switching condition includes the flicker intensity detection statistics being less than a flicker intensity threshold. In this case, the high dynamic range mode in the blur capture processing mode is switched from the interleaved high dynamic range mode to the dual-gain high dynamic range mode to save power consumption of the electronic device.

[0015] It is understood that the high dynamic range mode in the second processing mode is the dual-gain high dynamic range mode, and the depth calculation mode in the second processing mode can be a binocular depth calculation mode (i.e., the same as the depth calculation mode in the first processing mode) or a monocular depth calculation mode (i.e., switching from the binocular depth calculation mode to the monocular depth calculation mode). Switching from the binocular depth calculation mode to the monocular depth calculation mode can be triggered by the statistical value of the medium depth of field in the subject outline layer.

[0016] Among them, the flicker intensity threshold can be understood as a pre-calibrated threshold, which can be set in the electronic device when it leaves the factory, or it can be set based on a numerical value input by the user. The specific numerical value is not limited in the embodiments of this application.

[0017] In conjunction with the method provided in the first aspect, in some embodiments, the electronic device includes a primary sensor, and switching the high dynamic range mode in the blur capture processing mode from the interleaved high dynamic range mode to the dual-gain high dynamic range mode includes: controlling the operating mode of the primary sensor to switch from the interleaved high dynamic range mode to the dual-gain high dynamic range mode. Switching the operating mode of the primary sensor to the dual-gain high dynamic range mode saves power consumption of the primary sensor, thereby saving power consumption of the electronic device.

[0018] Optionally, the electronic device further includes an auxiliary sensor. If the depth calculation mode in the second processing mode is a monocular depth calculation mode, the operating mode of the auxiliary sensor can be controlled to switch from an outflow operating mode to a non-outflow waiting mode to save power consumption of the auxiliary sensor, thereby saving power consumption of the electronic device. If the depth calculation mode in the second processing mode is a binocular depth calculation mode, the operating mode of the auxiliary sensor can be controlled to the outflow operating mode.

[0019] In conjunction with the method provided in the first aspect, in some embodiments, the statistical information includes a statistical value of the medium depth of field in the subject outline layer, and the statistical information satisfies a mode switching condition including the statistical value of the medium depth of field in the subject outline layer being less than a depth of field threshold. In this case, the depth calculation mode in the defocused capture processing mode is switched from binocular depth calculation mode to monocular depth calculation mode to reduce depth calculation power consumption, thereby saving power consumption of the electronic device.

[0020] It is understandable that the depth calculation mode in the second processing mode is switched to the monocular depth calculation mode, and the high dynamic range mode in the second processing mode can be an interleaved high dynamic range mode (i.e., the same as the high dynamic range mode in the first processing mode) or a dual-gain high dynamic range mode (i.e., switching from the interleaved high dynamic range mode to the dual-gain high dynamic range mode). Switching from the interleaved high dynamic range mode to the dual-gain high dynamic range mode can be triggered by scene brightness statistics and dynamic range statistics, or by flicker intensity detection statistics, or by scene brightness statistics, dynamic range statistics, and flicker intensity detection statistics.

[0021] Among them, the depth of field threshold can be understood as a pre-calibrated threshold, which can be set in the electronic device when it leaves the factory, or can be set based on a numerical value input by the user. The specific numerical value is not limited in the embodiments of this application.

[0022] In conjunction with the method provided in the first aspect, in some embodiments, the electronic device includes a binocular depth calculation path and a monocular depth calculation path; switching the depth calculation mode in the defocused snapshot processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: controlling the depth calculation path in the defocused snapshot processing mode to switch from the binocular depth calculation path to the monocular depth calculation path. Switching from the binocular depth calculation path to the monocular depth calculation path can save power consumption of the electronic device.

[0023] Among them, the binocular depth calculation path is used to execute the binocular depth calculation mode, and the monocular depth calculation path is used to execute the monocular depth calculation mode.

[0024] In combination with the method provided in the first aspect, in some embodiments, the above-mentioned statistical information includes flicker intensity detection statistics and statistics of the medium depth of field in the subject contour layer, and the above-mentioned statistical information satisfies the mode switching conditions including that the flicker intensity detection statistics are less than the flicker intensity threshold, and that the medium depth of field statistics in the subject contour layer are less than the depth of field threshold. In this case, the high dynamic range mode in the blur capture processing mode is switched from the interlaced high dynamic range mode to the dual-gain high dynamic range mode, and the depth calculation mode in the blur capture processing mode is switched from the binocular depth calculation mode to the monocular depth calculation mode. The working mode of the main sensor is switched to the dual-gain high dynamic range mode to save the power consumption of the main sensor, and the depth calculation mode is switched from the binocular depth calculation mode to the monocular depth calculation mode to reduce the depth calculation power consumption, thereby saving the power consumption of the electronic device.

[0025] In combination with the method provided in the first aspect, in some embodiments, the above-mentioned statistical information includes scene brightness statistics, dynamic range statistics, and statistics of the depth of field in the subject contour layer, and the above-mentioned statistical information satisfies the mode switching conditions including that the scene brightness statistics are less than the scene brightness threshold, and the dynamic range statistics are less than the dynamic range threshold, and the depth of field statistics in the subject contour layer are less than the depth of field threshold. In this case, the high dynamic range mode in the blur capture processing mode is switched from the interlaced high dynamic range mode to the dual-gain high dynamic range mode, and the depth calculation mode in the blur capture processing mode is switched from the binocular depth calculation mode to the monocular depth calculation mode. The working mode of the main sensor is switched to the dual-gain high dynamic range mode to save the power consumption of the main sensor, and the depth calculation mode is switched from the binocular depth calculation mode to the monocular depth calculation mode to reduce the depth calculation power consumption, thereby saving the power consumption of the electronic device.

[0026] In combination with the method provided in the first aspect, in some embodiments, the electronic device includes a main sensor, an auxiliary sensor, a binocular depth calculation path, and a monocular calculation path. The high dynamic range mode in the defocused capture processing mode is switched from the interwoven high dynamic range mode to the dual-gain high dynamic range mode, including: controlling the working mode of the main sensor to switch from the interwoven high dynamic range mode to the dual-gain high dynamic range mode, and controlling the working mode of the auxiliary sensor to switch from the working outflow mode to the non-outflow waiting mode. The depth calculation mode in the defocused capture processing mode is switched from the binocular depth calculation mode to the monocular depth calculation mode, including: controlling the depth calculation path of the defocused capture processing mode to switch from the binocular depth calculation path to the monocular calculation path. Thus, the power consumption of the electronic device is saved by saving the power consumption of the main sensor and the auxiliary sensor, and reducing the power consumption of the depth calculation.

[0027] In combination with the method provided in the first aspect, in some embodiments, the electronic device also includes a multi-exposure fusion path and a single exposure fusion path; before switching the depth calculation mode in the blur capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode, the exposure fusion path of the blur capture processing mode is controlled to switch from the multi-exposure fusion path to the single exposure fusion path to further save the power consumption of the electronic device.

[0028] In a second aspect, an embodiment of the present application provides an electronic device, which includes one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the method described in the first aspect and any possible implementation method of the first aspect.

[0029] In a third aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0030] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions. When the instructions are executed on an electronic device, the electronic device executes the method described in the first aspect and any possible implementation of the first aspect.

[0031] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when the computer program product is run on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0032] It is understandable that the electronic device provided in the second aspect, the chip system provided in the third aspect, the computer storage medium provided in the fourth aspect, and the computer program product provided in the fifth aspect are all used to execute the method provided in the first aspect of this application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figures 1A-1F are schematic diagrams of a set of user interfaces provided in an embodiment of the present application;

[0034] 2A-2F are schematic diagrams of a set of shooting scenes provided in an embodiment of the present application;

[0035] FIG3 is a software architecture diagram of an electronic device provided in an embodiment of the present application;

[0036] FIG4 is a flow chart of a photographing method provided in an embodiment of the present application;

[0037] 5A-5D are flowcharts of several processing modes provided in embodiments of the present application;

[0038] FIG6 is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0040] 1. Blur snapshot

[0041] Normal snapshot refers to the process in which an electronic device with a shooting function quickly captures a natural, vivid, and expressive instant image of the subject (such as a person or animal) without the subject's knowledge. Blurred snapshot refers to the process in which an electronic device with a shooting function quickly captures an instant image of the subject without the subject's knowledge, and further performs depth calculation and portrait blur processing on the instant image to obtain an instant blurred image. Optionally, dynamic range fusion processing can be performed after or before the portrait blur processing to obtain an instantaneous high-dynamic blurred image. Among them, portrait blur refers to highlighting the detailed features of the portrait in the image and blurring the features of the background in the image to improve the clarity of the portrait.

[0042] In other words, blur capture is based on ordinary snapshots, supplemented by image processing processes such as depth calculation, portrait blur, and dynamic range fusion to enhance the display quality of the instant image, for example, increasing the clarity of the subject in the instant image and improving the dynamic range of the instant image. It is understandable that for the same subject captured in the same scene at the same time, the blur capture image will have higher clarity of the subject, a wider dynamic range, and better image display quality.

[0043] Because blur capture adds a series of image processing steps to regular snapshots, it consumes power faster for electronic devices. In other words, blur capture consumes more power than regular snapshots.

[0044] 2. Sensor High Dynamic Range (HDR) mode

[0045] Dynamic range (DR), in image processing, refers to the range from the darkest to the brightest in an image. The larger the dynamic range, the richer the image details. High dynamic range (HDR) increases the image's brightness range compared to dynamic range, allowing for wider highlights and shadows, resulting in richer image details. In other words, HDR allows both bright and dark areas in an image to be brought out.

[0046] The sensor refers to an image sensor. Sensors have different precision and dynamic range. For ease of description, the present application embodiment simply refers to the image sensor as a sensor.

[0047] The HDR mode of a sensor refers to the processing mode adopted by the sensor when processing HDR frames. The HDR mode of a sensor can be mainly divided into the following modes:

[0048] (1) Stagger HDR mode

[0049] StaggerHDR mode means that after the sensor reads a row of long-exposure data from an HDR frame, it immediately performs a short-exposure on the same row. The sensor interleaves the readings of each row with different exposure times to save waiting time. In other words, in staggerHDR mode, the sensor exposes two frames of data sequentially: one with a long exposure and the other with a short exposure. This sequential exposure can also be described as interleaved exposure. A long exposure on one frame can also be described as a long frame, while a short exposure on the other can also be described as a short frame.

[0050] The exposure time and gain of the two frames (i.e., long frame and short frame) are configurable. For example, if the exposure time of the long frame is longer, the interval between the long frame exposure time and the short frame exposure time is longer, which can easily introduce ghosting problems.

[0051] (2) Dual Conversion Gain (DCG) mode

[0052] DCG mode means the sensor exposes two frames of data simultaneously, with different gains for the two frames. In DCG mode, the sensor controls the conversion circuitry within it to achieve different gains for the two frames.

[0053] The DCG mode is a dual-gain mode at the pixel level, and the dual gain includes high conversion gain (HCG) and low conversion gain (LCG).

[0054] (3) Dual Amplifier Gain (DAG) mode

[0055] DAG mode means the sensor simultaneously exposes two frames of data, with different gains for each frame. In DAG mode, the sensor controls its internal amplification circuit to achieve different gains for the two frames.

[0056] The DAG mode is a dual-gain mode at the circuit level, and the dual gain includes HCG and LCG.

[0057] DCG mode and DAG mode can be collectively referred to as DXG mode. That is, DXG mode includes DCG mode and / or DAG mode. DXG mode refers to the sensor controlling its hardware circuitry to achieve different gains between the two exposed frames. DXG mode can also be described as dual-gain mode.

[0058] 3. Depth calculation mode

[0059] Depth calculation can also be described as visual depth estimation or depth estimation. The purpose of depth estimation is to estimate the depth of the scene in the image, that is, the vertical distance from each pixel in the scene to the camera imaging plane. Distance can be divided into absolute distance and relative distance. Depth calculation modes are divided into monocular depth calculation mode and binocular depth calculation mode.

[0060] (1) Monocular depth calculation mode

[0061] Monocular depth calculation mode can also be described as monocular depth estimation mode. Monocular depth estimation mode refers to determining the depth of a scene in an image using a single image. Monocular depth calculation mode can use either absolute or relative depth estimation algorithms. These two algorithms can be implemented using networks, such as absolute depth estimation networks and relative depth estimation networks. Relative depth estimation networks learn to extract depth differences between adjacent pixels, while absolute depth estimation networks predict absolute depth values.

[0062] (2) Binocular depth calculation mode

[0063] The binocular depth calculation mode can also be described as a binocular depth estimation mode. The binocular depth estimation mode refers to taking two images as input, calculating the cost volume through disparity, and estimating the depth value of each pixel in the image. The two images are images of the same scene taken at the same time, for example, two cameras (such as the left camera and the right camera) take images of the same scene at the same time. Binocular depth calculation may include processes such as feature point matching, epipolar line correction, disparity calculation, and disparity to depth conversion. These processes involve multiple convolutional neural networks (CNNs), network modeling, feature fusion and other complex calculation processes.

[0064] Since the computational complexity and computational intensity of the binocular depth calculation mode are higher than those of the monocular depth calculation mode, electronic devices using the binocular depth calculation mode consume more power than those using the monocular depth calculation mode.

[0065] In blurry snapshots, electronic devices consume more power than in normal snapshots. Therefore, how to save power in blurry snapshots is a pressing technical issue.

[0066] To solve the above problem, an embodiment of the present application provides a photographing method that can be applied to electronic devices with image processing capabilities, such as mobile phones, tablet computers, etc.

[0067] By implementing the shooting method provided in the embodiment of the present application, after starting the blur capture, if the statistical information of the shooting scene meets the mode switching conditions, the blur capture processing mode can be switched to a more power-saving processing mode, and the display effect of the blur capture image processed in this processing mode can be guaranteed.

[0068] The electronic device 100 is not limited to mobile phones and tablet computers. It can also be a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, an in-vehicle device, a smart home device, and / or a smart city device. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device.

[0069] 1A-1F exemplarily show a set of user interfaces on an electronic device 100 . The following specifically introduces the application scenarios of the shooting method provided in the embodiment of the present application in conjunction with FIG. 1A-1F .

[0070] First, FIG1A exemplarily illustrates a user interface, i.e., a home page, displaying installed applications on an electronic device 100. As shown in FIG1A , the home page displays one or more application icons, such as a “clock” application icon, a “calendar” application icon, a “weather” application icon, and the like.

[0071] The one or more application icons include a "Camera" application (hereinafter referred to as "Camera") icon, namely, icon 111. Electronic device 100 can detect a user operation on icon 111, such as a click operation. In response to the user operation, electronic device 100 can turn on the camera, call the camera to capture an image, and display the user interface shown in FIG1B.

[0072] In another implementation, the lock screen user interface of electronic device 100 displays a "camera" icon. Electronic device 100 can detect a user operation on the "camera" icon, such as clicking and pulling up to the top of the lock screen user interface. In response to the user operation, electronic device 100 can turn on the camera, call the camera to capture an image, and display the user interface shown in Figure 1B.

[0073] FIG1B exemplarily illustrates the user interface of a "Camera" application running on electronic device 100. The user interface may include a window 121, a capture control 122, a thumbnail display area 123, and a flip control 124. The user interface may also include some switch controls, such as a settings control, a filter switch control, an AI camera switch control, a flash switch control, etc. In FIG1B , the filter switch control, the AI ​​camera switch control, and the flash switch control are all turned off as an example.

[0074] Window 121 is used to display images captured by the camera and can be understood as a display area for captured images, allowing users to input click operations to the capture control 122 or the flip control 124 based on the image displayed in window 121. In some embodiments, window 121 also includes controls such as a motion capture control 125, a focus adjustment control 126, and a portrait blur control 127. When electronic device 100 detects a user operation on a focus adjustment control, it can adjust the camera's focus in response to the user operation, for example, from 1x to 2x.

[0075] When the electronic device 100 detects a user operation acting on the motion capture control 125, it can respond to the user operation, start motion capture and output a text prompt message "Motion capture is turned on", and control the color of the portrait in the motion capture control 125 to change and the color-changed portrait to be in a running state. After starting motion capture, the electronic device detects in real time whether the movement amplitude of the subject in the shooting environment meets the capture conditions. If the capture conditions are met, the image of this moment can be captured. If the capture conditions are not met, the image collection can continue. For example, if the amplitude of the smile at the corner of the mouth of the subject in the shooting environment is greater than the amplitude threshold, the electronic device 100 can capture the image of the smile at this moment.

[0076] FIG1C exemplarily shows the user interface of the electronic device 100 after detecting a user operation on the motion capture control 125. In FIG1C , the color of the portrait in the motion capture control 125 is different from the color of the portrait in the motion capture control 125 in FIG1B . The window 121 in FIG1C displays a text prompt message that “Motion capture is on”. And the switch control bar in FIG1C displays an automatic motion capture switch control 128, which is in the on state. The automatic motion capture switch control 128 is in the on state, indicating that the electronic device 100 can automatically detect whether the motion amplitude of the subject in the shooting environment meets the capture conditions, and capture the instantaneous image when the capture conditions are met.

[0077] If the electronic device 100 detects a user operation on the automatic motion capture switch control 128, it can respond to the user operation and turn off the automatic motion capture. In this case, the person in the motion capture control 125 continues to be in a running state, but the electronic device 100 does not automatically detect and capture the person. Instead, it responds to the user operation on the shooting control 122 to perform the capture.

[0078] When motion capture is started, if the electronic device 100 detects a user operation on the portrait blur control 127, it can respond to the user operation, start blur capture and output a text prompt message "Blur capture is turned on", and control the background color change in the portrait blur control 127.

[0079] FIG1D exemplarily shows the user interface of the electronic device 100 after detecting a user operation on the motion capture control 125 and then detecting a user operation on the portrait blur control 127. In FIG1D , the color of the portrait in the portrait blur control 127 is different from the color of the portrait in the portrait blur control 127 in FIG1C . The window 121 in FIG1D displays a text prompt message that “Blur capture is on”. And the switch control bar in FIG1D displays an automatic blur capture switch control 129, which is in the on state. The automatic blur capture switch control 129 is in the on state, indicating that the electronic device 100 can automatically detect whether the motion amplitude of the subject in the shooting environment meets the capture conditions, and capture when the capture conditions are met, and perform portrait blur processing on the captured image.

[0080] If the electronic device 100 detects a user operation on the automatic blur capture switch control 129, it can respond to the user operation and turn off the automatic blur capture. At this time, the portrait in the motion capture control 125 continues to be in the running state, and the color of the portrait in the portrait blur control 127 continues to be as shown in Figure 1D. However, the electronic device 100 does not automatically detect and capture the image. Instead, it responds to the user operation on the shooting control 122 to perform blur capture.

[0081] The transition from Figure 1B to Figure 1D shows that the electronic device 100 first detects a user operation on the motion capture control 125, initiates motion capture, and then detects a user operation on the portrait blur control 127. In other words, the blur capture control includes the motion capture control 125 and the portrait blur control 127, and blur capture can be initiated in response to detecting user operations on these two controls. This is one implementation method; in another implementation method, the blur capture control is a single control, and the electronic device 100 can initiate blur capture after detecting a user operation on this control.

[0082] FIG1E exemplarily illustrates the user interface of a "Camera" application running on electronic device 100. The user interface may include a window 121, a capture control 122, a thumbnail display area 123, and a flip control 124. The user interface may also include switch controls, such as a settings control, a filter switch control, an AI camera switch control, and a flash switch control. In FIG1E , the filter switch control, the AI ​​camera switch control, and the flash switch control are all turned off as an example.

[0083] 1E shows a window 121 including a blur capture control 130. The blur capture control 130 can be understood as a control that integrates a motion capture control and a portrait blur control, and is used to enable blur capture.

[0084] When the electronic device 100 detects a user operation on the blur capture control 130, it can respond to the user operation by enabling blur capture and outputting a text prompt message stating "Blur capture is enabled." It can also control the color of the portrait in the blur capture control 130 to change, and to control the portrait to appear to be running. After blur capture is enabled, the electronic device detects in real time whether the subject's motion in the shooting environment meets the capture conditions. If the conditions are met, the instant image is captured and the portrait is blurred.

[0085] FIG1F exemplarily shows the user interface of the electronic device 100 after detecting a user operation on the blur capture control 130. In FIG1F , the color of the portrait in the blur capture control 130 is different from the color of the portrait in the blur capture control 130 in FIG1D , and the portrait in the blur capture control 130 is in a running state. The window 121 in FIG1F displays a text prompt message that “Blur capture is on”. And the switch control bar in FIG1F displays an automatic blur capture switch control 131, which is in the on state. The automatic blur capture switch control 131 is in the on state, indicating that the electronic device 100 can automatically detect whether the motion amplitude of the subject in the shooting environment meets the capture conditions, and capture when the capture conditions are met, and perform portrait blur processing on the captured image.

[0086] If the electronic device 100 detects a user operation on the automatic blur snapshot switch control 131, it can respond to the user operation and turn off the automatic blur snapshot. In this case, the portrait in the blur snapshot control 130 continues to be in the running state, but the electronic device 100 does not automatically detect and capture the portrait. Instead, it responds to the user operation on the shooting control 122 to perform blur snapshot.

[0087] The user interface shown in Figures 1B to 1F also includes a "night scene" mode, a "portrait" mode, a "photo" mode, and a "video recording" mode. These modes are used for example and do not constitute a limitation on the embodiments of the present application. For example, in actual applications, a "multi-lens video recording" mode is also included. Among them, the "video recording" mode is used to record video files, and the "multi-lens video recording" mode is used to record video files when the front camera and the rear camera are turned on at the same time. The "night scene" mode is used to capture night scene images and improve the clarity of night scene images. The "portrait" mode is mainly used to capture portrait images. In this mode, the image processing method provided by the embodiments of the present application can be used. The "photo" mode is used to capture single-frame images. In Figures 1B to 1F, the mode selected by the user is the "photo" mode.

[0088] After the electronic device 100 performs automatic blur capture or passive blur capture, a thumbnail of the blurred captured image (i.e., the image obtained by blurring the portrait of the captured instant image) can be displayed in the thumbnail display area 123. When the electronic device 100 detects a user operation acting on the thumbnail display area 123, it jumps to the user interface for browsing the blurred captured image in response to the user operation.

[0089] By adopting the shooting method provided in the embodiment of the present application, the thumbnail display area 123 can display the thumbnail of the blurred snapshot image, or save the blurred snapshot image in a gallery.

[0090] The shooting method provided in the embodiments of the present application can be applied to one or more of the following shooting scenarios.

[0091] Shooting scene 1: Non-flicker scene

[0092] The shutter is a structure used by the camera to control the effective exposure time of the photosensitive film. It can be divided into a global shutter and a rolling shutter. The global shutter is that all pixels on the entire sensor are exposed to light at the same time, and the exposure ends at the same time after the same period of time. The rolling shutter is a row scan, and each row goes through the process of reset, exposure, and data reading, and then is exposed row by row. Current sensors usually use a rolling shutter. The start exposure time and exposure time for all pixels in the same row are consistent, that is, the energy received by the pixels in the same row is the same, but the energy received by the pixels in adjacent rows is inconsistent, which will cause light and dark stripes to appear in the image. This phenomenon can be called Flicker. In other words, due to the influence of the light source frequency, alternating light and dark conditions appear in the image, which looks like flickering. The light source frequency can be, for example, 50Hz or 60Hz.

[0093] A non-flicker scene can be understood as a scene without the influence of light source frequency, a scene without light source, or a scene with flicker intensity less than a threshold.

[0094] In a non-Flicker scenario, if blur capture is turned on, the shooting method provided in the embodiment of the present application can be used to save power consumption of the electronic device.

[0095] For example, please refer to the shooting scenes shown in Figures 2A and 2B. The shooting scene shown in Figure 2A can be understood as a Flicker scene, and the shooting scene shown in Figure 2B can be understood as a non-Flicker scene. It can be understood that in a scene where the light is on and the projector is in a playback state, the electronic device can use the staggerHDR mode to shoot the scene when the blur capture is turned on; in a scene where the light is off and the projector is off, the electronic device can use the DXG mode to shoot the scene when the blur capture is turned on to save power consumption of the electronic device.

[0096] Shooting scene 2: dark light scene

[0097] A low-light scene refers to a scene with low brightness. In such a scenario, when the electronic device is enabled for blur capture, it can use the monocular depth calculation mode to capture this scene to save power. Furthermore, in such a scenario, the display effect of the blurred snapshot image processed using the monocular depth calculation mode also meets the display effect of blur capture. A low-light scene can also be understood as a scene where the monocular depth calculation mode is acceptable.

[0098] For example, please refer to the shooting scenes shown in Figures 2C and 2D. The shooting scene shown in Figure 2C can be understood as a bright light scene, for example, the indoor lights are on, or the weather is sunny during the day, etc.; the shooting scene shown in Figure 2D can be understood as a dark light scene, for example, the indoor lights are off, or it is getting dark in the evening, or the weather is gloomy during the day, etc. For shooting window scenes, the shooting scene inside the window may be a dark light scene, and the shooting scene outside the window may be a bright light scene. It can be understood that for the shooting scene shown in Figure 2C, when the electronic device is turned on for blur capture, the binocular depth calculation mode can be used to shoot the scene; for the shooting scene shown in Figure 2D, when the electronic device is turned on for blur capture, the monocular depth calculation mode can be used to shoot the scene to save power consumption of the electronic device.

[0099] Shooting scene three: non-HDR scene

[0100] Non-HDR scenes can be understood as normal DR scenes, meaning scenes with minimal differences between bright and dark areas. Alternatively, they may be scenes that require less stringent brightness range and detail. Non-HDR scenes can also be understood as scenes with acceptable dynamic range in DXG mode.

[0101] For example, please refer to the shooting scenes shown in Figures 2E and 2F. The shooting scene shown in Figure 2E can be understood as an HDR scene, and the shooting scene shown in Figure 2F can be understood as a non-HDR scene. It can be understood that for the shooting scene shown in Figure 2E, when the electronic device is enabled for blur capture, the staggerHDR mode can be used to shoot the scene; for the shooting scene shown in Figure 2F, when the electronic device is enabled for blur capture, the DXG mode can be used to shoot the scene to save power consumption of the electronic device.

[0102] The above three shooting scenarios are for example purposes only and do not limit the embodiments of the present application. For example, they can also be applied to the auxiliary sensor automatic exposure (AutoExposure, AE) scenario, the auxiliary sensor extended or shortened exposure time scenario, etc. The above three shooting scenarios can be superimposed, such as non-HDR scenes and low-light scenes, non-Flicker scenes and low-light scenes, etc.

[0103] It is understandable that for blurred snapshots, in some shooting scenarios, the electronic device can switch the HDR mode of the sensor from staggerHDR mode to DXG mode, and / or switch the depth calculation mode from binocular depth calculation mode to monocular depth calculation mode to save power consumption of the electronic device.

[0104] The following describes in detail the specific process of the electronic device 100 implementing the user interface shown in Figure 1D or Figure 1F.

[0105] First, FIG. 3 exemplarily shows the software architecture of the electronic device 100 .

[0106] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0107] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers: from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, the hardware abstraction layer (HAL), and the kernel layer.

[0108] The application layer may include a series of application packages. As shown in Figure 3, the application package may include applications such as camera, gallery, video, music, navigation, calendar, map, WLAN, etc. In an embodiment of the present application, during operation, the camera may provide a user interface as shown in Figures 1B to 1F, and the electronic device 100 may display a thumbnail of the blurred snapshot image in the thumbnail display area 123, or save the blurred snapshot image in the gallery. After starting the camera, the electronic device 100 may call the camera to capture an image. The electronic device 100 may include at least one camera.

[0109] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions. In the embodiment of the present application, the application framework layer includes a camera service framework, which may include some functions that support blur capture.

[0110] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0111] The Android Runtime consists of a core library and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system. The core library consists of two parts: one for the Java language's callable functions and the other for the Android core library.

[0112] The system library can include multiple functional modules. For example: surface manager, media library, 3D graphics processing library (such as OpenGL), etc. The surface manager is used to manage the display subsystem and provides 2D and 3D layer fusion for multiple applications. The media library supports playback and recording of various common audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing.

[0113] The Hardware Abstraction Layer (HAL) is an interface layer between the kernel layer and the hardware, and can be used to control the actions of the hardware. In an embodiment of the present application, the HAL may include a blur capture algorithm for implementing blur capture. The blur capture algorithm may include one or more of the following: a staggerHDR mode algorithm, a DXG mode algorithm, a monocular depth calculation mode algorithm, a binocular depth calculation mode algorithm, a portrait blur algorithm, etc.

[0114] Among them, the algorithm of the staggerHDR mode is used to support the staggerHDR mode. The algorithm may include one or more of the following: long exposure of the main sensor, short exposure of the main sensor, frame selection and multi-frame fusion algorithm, multi-exposure fusion algorithm, auxiliary sensor exposure, etc.

[0115] The DXG mode algorithm is used to support the DXG mode, which may include main sensor exposure, selection and multi-frame fusion algorithms, etc.

[0116] The monocular depth calculation algorithm is used to support the monocular depth calculation mode, and the binocular depth calculation algorithm is used to support the binocular depth calculation mode. The portrait blur algorithm is used to achieve portrait blur, that is, to highlight the details of the portrait and blur the features of the background.

[0117] The kernel layer is the foundation of the Android system. For example, ART relies on the kernel layer to execute low-level functions such as threading and low-level memory management. The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, sensor drivers, GPU drivers, etc. In the embodiment of the present application, the kernel layer includes a camera driver and a display driver. The camera driver is used to drive the camera to capture images, and the display driver is used to drive the display screen to display blurred captured images.

[0118] Based on the software architecture shown in Figure 3, the shooting method provided by the embodiment of the present application is described. In some embodiments, the electronic device 100 detects a user operation acting on the camera application, responds to the user operation, and starts the camera to enable the camera to capture images. Upon detecting the user operation, the camera application triggers an instruction to start the camera. The camera application calls the API interface of the application framework layer to send the instruction to the camera service framework, and the camera service framework calls the hardware abstraction layer to send the instruction to the camera driver. The camera driver can drive the startup of the camera and drive the camera to capture images. The images captured by the camera can be cached in the image buffer area.

[0119] After the electronic device 100 starts the camera, the camera service framework, in response to detecting a user operation on the blur capture control in the user interface of the camera application, starts blur capture and obtains preview stream data of the captured scene from the image buffer. The camera service framework transmits the preview stream data to the hardware abstraction layer. The hardware abstraction layer determines statistical information of the captured scene based on the preview stream data and determines whether the statistical information meets the mode switching conditions. If so, the hardware abstraction layer switches the sensor's HDR mode from staggerHDR mode to DXG mode and / or switches the depth calculation mode from binocular depth calculation mode to monocular depth calculation mode. In response to the blur capture instruction for the captured scene, the camera service framework calls the camera driver to capture and obtain a to-be-processed image of the captured scene, and transmits the to-be-processed image to the hardware abstraction layer. The hardware abstraction layer processes the to-be-processed image based on the switched mode to obtain a blur capture image of the captured scene, and transmits the blur capture image to the display driver so that the electronic device 100 displays a thumbnail of the blur capture image. Among them, the blur capture instruction can be an instruction generated by the hardware abstraction layer based on the motion detection algorithm when it detects that the motion amplitude of the subject meets the capture conditions, or it can be an instruction generated based on the user operation of clicking the shooting control 122 when the blur capture is turned on.

[0120] FIG4 exemplarily shows a flow chart of the shooting method provided by an embodiment of the present application. In conjunction with the user interfaces shown in FIG1A to FIG1F and the software architecture of the electronic device 100 shown in FIG3 , the flow chart of the shooting method provided by an embodiment of the present application will be described in detail below.

[0121] 401. The electronic device 100 detects a user operation on a blur capture control in a user interface of a camera application. In response to the user operation, the blur capture is started and a camera device is called to collect preview stream data of the shooting scene.

[0122] The user interface of the camera application in step 401 refers to the user interface of the camera application output by the electronic device 100 in response to a detected user operation on the camera application, such as the user interface shown in FIG. 1B or FIG. 1E .

[0123] In one implementation, the user interface includes a blur capture control, such as blur capture control 130 in FIG1E . The electronic device 100 detects a user operation (e.g., a click) on the blur capture control in the user interface. In response to the user operation, blur capture is initiated and a camera (e.g., one or more cameras) is called to capture preview stream data of the captured scene. After blur capture is initiated, the color of the person in the blur capture control changes, and the person appears to be running.

[0124] In another implementation, the user interface includes a motion capture control and a portrait blur control, such as the motion capture control 125 and the portrait blur control 127 in Figure 1B. These two controls constitute the blur capture control. The electronic device 100 detects a user operation acting on the motion capture control in the user interface, and in response to the user operation, starts motion capture and calls the camera device to collect preview stream data of the shooting scene. After starting the motion capture, the portrait in the motion capture control changes color and the portrait is in a running state. Afterwards, the user operation acting on the portrait blur control in the user interface is detected again, and in response to the user operation, starts blur capture and continues to collect preview stream data of the shooting scene. After starting the blur capture, the portrait in the portrait blur control changes color.

[0125] Starting the blur capture can be understood as starting the blur capture function, or turning on the blur capture function. In this way, the electronic device 100 can capture and blur the portrait based on the movement of the subject.

[0126] The capture scene refers to the scene that the camera can currently capture. The preview stream data of the capture scene refers to the preview image captured by the camera in real time before automatic or passive capture. The preview stream data can include multiple frames of preview images.

[0127] Optionally, when the electronic device 100 detects a user operation on a camera application, it outputs the user interface of the camera application and calls the camera device to collect preview stream data of the captured scene. When the electronic device 100 detects a user operation on a blur capture control, it activates blur capture and continuously collects preview stream data of the captured scene.

[0128] Optionally, when the electronic device 100 detects a user operation on a camera application, it outputs the camera application's user interface and invokes a camera device to capture images. When a user operation on a blur capture control is detected, blur capture is initiated and the camera device is invoked to capture preview stream data of the captured scene. In other words, the series of images captured by the camera device after blur capture is initiated is referred to as preview stream data of the captured scene.

[0129] 402 : The electronic device 100 determines statistical information of the shooting scene based on the preview stream data.

[0130] The electronic device 100 performs statistical analysis on the collected preview stream data to determine statistical information of the captured scene. The statistical information is used to determine whether to switch to the blur capture processing mode. The statistical information may include, but is not limited to, one or more of the following: scene brightness statistics, dynamic range statistics, flicker intensity detection statistics, medium depth of field statistics of the subject outline layer, automatic exposure (AE) information, etc.

[0131] The scene brightness statistics value refers to the average luminance value of multiple preview frames included in the preview stream data. For a preview frame, the sensor can read the scene brightness value of the image before reading the image data.

[0132] The dynamic range statistic refers to the average dynamic range of the multiple preview frames included in the preview stream data. For a preview frame, the ratio of the sum of the number of pixels in the largest bin and the smallest bin to the number of pixels in the middle bin in its scene histogram represents the dynamic range of that preview frame. Taking a 128-bin scene histogram as an example, the ratio of the sum of the number of pixels in bin 1 and bin 128 to the number of pixels in bin 64 is the dynamic range value. A larger dynamic range value indicates a higher scene dynamic range.

[0133] The flicker intensity detection statistic value refers to the average flicker intensity detection value of multiple preview image frames included in the preview stream data. For a preview image frame, the electronic device 100 can call a flicker detection algorithm to perform flicker intensity detection on it to obtain a flicker intensity detection value.

[0134] The statistical value of the medium depth of field of the object outline layer refers to the average medium depth of field of the object outline layer of multiple preview images included in the preview stream data. For each preview image frame, the electronic device 100 performs an image morphological operation (dilation) on it to obtain a first dual-camera depth map (map), and performs an image morphological operation (erosion) on the preview image frame to obtain a second dual-camera depth map (map). Based on the first and second dual-camera depth maps, a differential signal map (map) is calculated. The depth information value of the differential signal map is statistically analyzed to obtain the medium depth of field.

[0135] The automatic exposure information is used to describe information related to automatic exposure, for example, it may include one or more of whether the automatic exposure is turned on, the aperture size of the automatic exposure, the exposure time of the automatic exposure, the gain of the automatic exposure, etc.

[0136] The above statistical information is used for example only and does not limit the embodiments of the present application. For example, other information used to determine whether to switch to the blur capture mode may also be included.

[0137] 403 , in response to the statistical information satisfying the mode switching condition, the electronic device 100 switches the blur capture processing mode from the first processing mode to the second processing mode.

[0138] Among them, the mode switching conditions may include a series of thresholds, for example, one or more of the following thresholds: scene brightness threshold, dynamic range threshold, flicker intensity threshold, depth of field threshold, etc. These thresholds can be understood as pre-calibrated thresholds, which can be set in the electronic device when it leaves the factory, or can be set based on user input values. The specific values ​​are not limited in the embodiments of this application. These thresholds can also be understood as empirical thresholds, that is, some empirical thresholds summarized by R&D personnel.

[0139] The scene brightness threshold is used for comparison with the scene brightness statistics. If the scene brightness statistics are less than the scene brightness value, it can be interpreted as a low-light scene. Conversely, if the scene brightness statistics are greater than or equal to the scene brightness value, it can be interpreted as a bright-light scene. The dynamic range threshold is used for comparison with the dynamic range statistics. If the dynamic range statistics are less than the dynamic range threshold, it can be interpreted as a non-HDR scene. Conversely, if the dynamic range statistics are greater than or equal to the dynamic range threshold, it can be interpreted as an HDR scene. The flicker intensity threshold is used for comparison with the flicker intensity detection statistics. If the flicker intensity detection statistics are less than the flicker intensity threshold, it can be interpreted as a non-flicker scene. Conversely, if the flicker intensity detection statistics are greater than or equal to the flicker intensity threshold, it can be interpreted as a flicker scene. The depth of field threshold is used for comparison with the medium depth of field statistics of the object outline layer. If the medium depth of field statistics of the object outline layer are less than the depth of field threshold, it can be interpreted as a non-high-medium depth of field ratio scene. Conversely, if the medium depth of field statistics of the object outline layer are greater than or equal to the depth of field threshold, it can be interpreted as a high-medium depth of field ratio scene.

[0140] Optionally, the mode switching condition may further include one or more of an exposure time threshold, a gain threshold, and the like.

[0141] The blur capture processing mode is used to achieve blur capture and includes high dynamic range mode and depth calculation mode. High dynamic range mode refers to the sensor's HDR mode, which can be divided into interleaved high dynamic range mode and dual-gain high dynamic range mode. The interleaved high dynamic range mode can be the staggerHDR mode mentioned above, and the dual-gain high dynamic range mode can be the DXG mode mentioned above. For convenience of description, the staggerHDR mode is used to describe the interleaved high dynamic range mode, and the DXG mode is used to describe the dual-gain high dynamic range mode. The depth calculation mode can be divided into binocular depth calculation mode and monocular depth calculation mode.

[0142] The high dynamic range mode in the second processing mode is different from the high dynamic range mode in the first processing mode, and / or the depth calculation mode in the second processing mode is different from the depth calculation mode in the first processing mode. This allows the electronic device to adopt different high dynamic range modes and / or different depth calculation modes in different shooting scenes, thereby helping to save power consumption of the electronic device. In the embodiment of the present application, the high dynamic range mode in the first processing mode is the staggerHDR mode, and the depth calculation mode in the first processing mode is the binocular depth calculation mode as an example. The blurred capture processing mode is the first processing mode, which can achieve the best effect of blurred capture images, but this mode is more power-consuming.

[0143] The statistical information is different, the mode switching conditions satisfied are different, and the second processing mode is different.

[0144] In Mode 1, the statistical information includes scene brightness statistics and dynamic range statistics. The statistical information satisfies a mode switching condition when the scene brightness statistics are less than a scene brightness threshold and the dynamic range statistics are less than a dynamic range threshold. Alternatively, the statistical information satisfies the mode switching condition when the ratio of the scene brightness statistics to the dynamic range statistics is less than a certain threshold; or when the statistical information satisfies the mode switching condition when the product of the scene brightness statistics and the dynamic range statistics is less than a certain threshold; and so on. In Mode 1, the high dynamic range mode in the second processing mode is DXG mode.

[0145] For method 1, when the electronic device 100 detects that the statistical information meets the mode switching condition, it can output a low power mode enable signal to switch from the staggerHDR mode to the DXG mode. For example, the camera application framework layer of the electronic device 100 outputs the low power mode enable signal to the hardware abstraction layer.

[0146] Mode 2: The statistical information includes a flicker intensity detection statistical value, and the statistical information satisfies a mode switching condition including the flicker intensity detection statistical value being less than a flicker intensity threshold. In this mode, the high dynamic range mode in the second processing mode is the DXG mode.

[0147] For mode 2, when the electronic device 100 detects that the statistical information meets the mode switching condition, it can output a low power mode enable signal to switch from the staggerHDR mode to the DXG mode. For example, the camera application framework layer of the electronic device 100 outputs the low power mode enable signal to the hardware abstraction layer.

[0148] In mode 3, the statistical information includes a statistical value of the medium depth of field of the object outline layer, and the statistical information satisfies a mode switching condition including the statistical value of the medium depth of field of the object outline layer being less than a depth of field threshold. In mode 3, the depth calculation mode in the second processing mode is a monocular depth calculation mode.

[0149] For mode 3, when the electronic device 100 detects that the statistical information meets the mode switching condition, it can output a low-power mode enable signal to switch from the binocular depth calculation mode to the monocular depth calculation mode. For example, the camera application framework layer of the electronic device 100 outputs the low-power mode enable signal to the hardware abstraction layer.

[0150] In Mode 4, the statistical information includes scene brightness statistics, dynamic range statistics, and medium depth of field statistics for the object outline layer. The statistical information satisfies the mode switching conditions when the scene brightness statistics are less than the scene brightness threshold, the dynamic range statistics are less than the dynamic range threshold, and the medium depth of field statistics for the object outline layer are less than the depth of field threshold. In Mode 4, the high dynamic range mode in the second processing mode is DXG mode, and the depth calculation mode is monocular depth calculation mode.

[0151] In Mode 5, the statistical information includes flicker intensity detection statistics and medium depth of field statistics for the object outline layer. The statistical information satisfies the mode switching conditions when the flicker intensity detection statistics are less than the flicker intensity threshold, and the medium depth of field statistics for the object outline layer are less than the depth of field threshold. In Mode 5, the high dynamic range mode in the second processing mode is DXG mode, and the depth calculation mode is monocular depth calculation mode.

[0152] For methods 4 and 5, when the electronic device 100 detects that the statistical information meets the mode switching conditions, it can output a low-power mode enable signal to switch from staggerHDR mode to DXG mode, and from binocular depth calculation mode to monocular depth calculation mode. For example, the camera application framework layer of the electronic device 100 outputs the low-power mode enable signal to the hardware abstraction layer.

[0153] The above-mentioned methods 1 to 5 are provided for illustrative purposes only and do not limit the embodiments of the present application. Other combinations are possible. For example, if the combination of the flicker intensity detection statistics and the AE information is both less than the corresponding threshold, the depth calculation mode in the second processing mode is the monocular depth calculation mode.

[0154] For the above-mentioned method 1 and method 2, when the electronic device 100 switches the blur processing mode from the first processing mode to the second processing mode, the high dynamic range mode in the blur capture processing mode is switched from the staggerHDR mode to the DXG mode to save the power consumption of the electronic device. The high dynamic range mode in the second processing mode is the DXG mode, and the depth calculation mode in the second processing mode can be a binocular depth calculation mode (i.e., the same as the depth calculation mode in the first processing mode), or a monocular depth calculation mode (i.e., switching from the binocular depth calculation mode to the monocular depth calculation mode). Switching from the binocular depth calculation mode to the monocular depth calculation mode can be triggered by the statistical value of the medium depth of field in the contour layer of the subject. Switching the high dynamic range mode in the blur capture processing mode from the staggerHDR mode to the DXG mode can be: controlling the working mode of the main sensor to switch from the staggerHDR mode to the DXG mode.

[0155] When the primary sensor is in stagger HDR mode, it outputs images at 60 fps, twice the normal 30 fps. Each pair of frames is paired: a long-exposure image and a short-exposure image. These two frames are obtained by performing two exposures on the primary sensor.

[0156] When the primary sensor operates in DXG mode, it outputs images at 30 fps, the same frame rate as standard. Each frame has a bit width two bits wider than the standard 10-bit frame. In this mode, the primary sensor uses two conversion gain methods for the same exposure, then fuses the data within the sensor's ISP. This results in a relatively high dynamic range.

[0157] For the above-mentioned method 3, when the electronic device 100 switches the blur processing mode from the first processing mode to the second processing mode, the depth calculation mode in the blur capture processing mode is switched from the binocular depth calculation mode to the monocular depth calculation mode to reduce the power consumption of the depth calculation, thereby saving the power consumption of the electronic device. The depth calculation mode in the second processing mode is switched to the monocular depth calculation mode, and the high dynamic range mode in the second processing mode can be the staggerHDR mode (i.e., the same as the high dynamic range mode in the first processing mode) or the DXG mode (i.e., switching from the interleaved high dynamic range mode to the dual gain high dynamic range mode). Switching from the staggerHDR mode to the DXG mode can be triggered by the scene brightness statistics and the dynamic range statistics, or by the flicker intensity detection statistics, or by the scene brightness statistics, the dynamic range statistics and the flicker intensity detection statistics. Switching the depth calculation mode in the blur capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: controlling the depth calculation path of the blur capture processing mode to switch from the binocular depth calculation path to the monocular depth calculation path. Switching from a binocular depth calculation path to a monocular depth calculation path can save power consumption of electronic devices.

[0158] The binocular depth calculation pathway is used to execute the binocular depth calculation mode. The output of the multi-exposure fusion algorithm or the DXG data after multi-frame noise reduction is combined with the auxiliary sensor data for binocular depth calculation and output to the subsequent portrait blur algorithm. The monocular depth calculation pathway is used to execute the monocular depth calculation mode. The DXG data after multi-frame noise reduction is sent to the monocular depth calculation and output to the subsequent portrait blur algorithm.

[0159] For the above-mentioned method 4 and method 5, when the electronic device 100 switches the blur processing mode from the first processing mode to the second processing mode, the high dynamic range mode in the blur capture processing mode is switched from the staggerHDR mode to the DXG mode, and the depth calculation mode in the blur capture processing mode is switched from the binocular depth calculation mode to the monocular depth calculation mode. Switching the high dynamic range mode in the blur capture processing mode from the staggerHDR mode to the DXG mode includes: controlling the working mode of the main sensor from the staggerHDR mode to the DXG mode, and controlling the working mode of the auxiliary sensor from the working outflow mode to the non-outflow waiting mode. Switching the depth calculation mode in the blur capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: controlling the depth calculation path of the blur capture processing mode to switch from the binocular depth calculation path to the monocular calculation path.

[0160] When the auxiliary sensor is in the working streaming mode, the image output of the auxiliary sensor is 30fps. When the auxiliary sensor is in the non-streaming standby mode, the auxiliary sensor does not output image data but can quickly recover to the state of outputting data. This state can be called the standby state.

[0161] Before switching the depth calculation mode in the defocused capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode, the exposure fusion path of the defocused capture processing mode is controlled to switch from the multi-exposure fusion path to the single-exposure fusion path to further save the power consumption of the electronic device.

[0162] The multi-exposure fusion pathway is as follows: the long-exposure data from the primary sensor is subjected to frame selection and a multi-frame fusion algorithm to obtain multi-frame noise-reduced data. This data is then fused with the short-exposure frames from the primary sensor using a multi-exposure fusion algorithm to obtain fused frame data. This data is then combined with information from depth calculation (monocular or binocular) to perform a portrait blur algorithm to obtain a blurred snapshot image.

[0163] The single exposure path is as follows: the DXG frame data of the main sensor is subjected to frame selection and a high-bitwidth multi-frame fusion algorithm to obtain multi-frame noise reduction data. This data is then combined with the information from depth calculation (monocular or binocular) to perform a portrait blur algorithm to obtain a blurred snapshot image.

[0164] 404 , in response to the blur capture instruction for the shooting scene, the electronic device 100 calls the camera device to capture and obtain the image to be processed of the shooting scene.

[0165] Among them, the blur capture instruction for the shooting scene can be an instruction generated by the electronic device 100 based on the motion detection algorithm when it detects that the motion amplitude of the subject meets the capture conditions, or it can be an instruction generated based on the user operation of clicking the shooting control 122 when the blur capture is turned on.

[0166] When the electronic device 100 detects a blur capture instruction for a captured scene, it calls a camera to capture an image of the captured scene to be processed. Optionally, the electronic device 100 calls one camera to capture the image to be processed. Optionally, the electronic device 100 calls two cameras to capture the image to be processed.

[0167] 405 , the electronic device 100 processes the image to be processed based on the second processing mode to obtain a blurred captured image of the shooting scene.

[0168] The electronic device 100 processes the image to be processed based on the second processing mode to obtain a blurred captured image of the shooting scene.

[0169] FIG5A illustrates the process by which the electronic device 100 uses the first processing mode to obtain a blurred snapshot image. In FIG5A , the blurred snapshot mode is determined to be the first processing mode based on the preview stream data. In response to the blurred snapshot instruction, both the primary and secondary sensors generate images. The primary sensor uses the staggerHDR mode to generate two frames of data: long-exposure data and short-exposure data. The long-exposure data is processed through frame selection and a multi-frame fusion algorithm, and then multi-exposure fusion is performed with the short-exposure data. Binocular depth calculation and portrait blur processing are then performed to obtain the blurred snapshot image.

[0170] The process of the electronic device 100 adopting the second processing mode to obtain a blurred snapshot image can be seen in Figures 5B to 5D. In Figure 5B, the blurred snapshot mode is determined to be the second processing mode (DXG mode) based on the preview stream data. In response to the blurred snapshot instruction, the main sensor and the auxiliary sensor both output images. The main sensor adopts the DXG mode and outputs a frame of data. After the frame data is processed by the frame selection and multi-frame fusion algorithm, the frame data is combined with the auxiliary sensor data for binocular depth calculation, and then the portrait is blurred, thereby obtaining a blurred snapshot image. In Figure 5C, the blurred snapshot mode is determined to be the second processing mode (monocular depth calculation mode) based on the preview stream data. In response to the blurred snapshot instruction, the main sensor outputs an image. The main sensor adopts the staggerHDR mode and outputs two frames of data, long exposure data and short exposure data. After the long exposure data is processed by the frame selection and multi-frame fusion algorithm, the multi-exposure fusion processing is performed with the short exposure data, and then the monocular depth calculation and portrait blur processing are performed, thereby obtaining a blurred snapshot image. In Figure 5D, based on the preview stream data, the blur capture mode is determined to be the second processing mode (DXG mode and monocular depth calculation). In response to the blur capture instruction, the main sensor outputs a picture. The main sensor adopts the DXG mode and outputs a frame of data. After the frame data is processed by the frame selection and multi-frame fusion algorithm, a monocular depth calculation is performed, and then the portrait blur processing is performed to obtain a blurred capture image.

[0171] Comparing FIG5A with FIG5B to FIG5D , it can be seen that FIG5B to FIG5D have fewer processes, which is beneficial to saving power consumption of the electronic device.

[0172] 406 , the electronic device 100 displays a thumbnail of the blurred captured image of the shooting scene in the thumbnail display area of ​​the user interface of the camera application.

[0173] The electronic device 100 obtains the blurred snapshot image and displays a thumbnail of the blurred snapshot image in the thumbnail display area of ​​the camera application user interface, such as the thumbnail display area 123 in Figures 1D and 1F. Optionally, the electronic device 100 can save the blurred snapshot image in a gallery.

[0174] In the embodiment shown in Figure 4, in the blurred snapshot scenario, when the statistical information of the shooting scene meets the mode switching conditions, the blurred snapshot processing mode can be switched from the first processing mode to the second processing mode, and then the captured image to be processed can be processed based on the second processing mode, thereby saving power consumption of the electronic device in the blurred snapshot scenario.

[0175] The embodiment shown in FIG4 introduces the process of switching from the first processing mode to the second processing mode. The statistical information of the shooting scene may change with time. When the shooting scene changes to meet the first processing mode, the blur capture processing mode can be switched from the second processing mode to the first processing mode. For example, for the above-mentioned method 1, when the scene brightness statistical value is greater than or equal to the scene brightness threshold, and / or the dynamic range statistical value is greater than or equal to the dynamic range threshold, the high dynamic range mode can be switched from the DXG mode to the staggerHDR mode. For another example, for the above-mentioned method 2, when the flicker intensity detection statistical value is greater than or equal to the flicker intensity threshold, the high dynamic range mode can be switched from the DXG mode to the staggerHDR mode. For another example, for the above-mentioned method 3, when the statistical value of the medium depth of field in the subject contour layer is greater than or equal to the depth of field threshold, the depth calculation mode can be switched from the monocular depth calculation mode to the binocular depth calculation mode. For another example, for the above-mentioned method 4, if the scene brightness statistics are less than the scene brightness threshold, and the dynamic range statistics are less than the dynamic range threshold, but the statistics of the medium depth of field in the object contour layer are greater than or equal to the depth of field threshold, then the DXG mode can be continued to be used, and the depth calculation mode can be switched from the monocular depth calculation mode to the binocular depth calculation mode; or, if the statistics of the medium depth of field in the object contour layer are less than the depth of field threshold, but the scene brightness statistics are greater than or equal to the scene brightness threshold, and / or the dynamic range statistics are greater than or equal to the dynamic range threshold, then the monocular depth calculation mode can be continued to be used, and the high dynamic range mode can be switched from the DXG mode to the staggerHDR mode; or, if the scene brightness statistics are greater than or equal to the scene brightness threshold, and / or the dynamic range statistics are greater than or equal to the dynamic range threshold, and the statistics of the medium depth of field in the object contour layer are greater than or equal to the depth of field threshold, then the high dynamic range mode can be switched from the DXG mode to the staggerHDR mode, and the depth calculation mode can be switched from the monocular depth calculation mode to the binocular depth calculation mode.

[0176] FIG6 exemplarily shows a schematic diagram of the hardware structure of the electronic device 100 .

[0177] The electronic device 100 may include a processor 110, an external memory interface 12B, an internal memory 12A, a universal serial bus (USB) interface 13A, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio processing module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0178] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0179] The processor 110 may include one or more processing units, for example: the processor 110 may include an AP, a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices or integrated into one or more processors. In an embodiment of the present application, the processor 110 may include a blur capture algorithm.

[0180] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0181] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0182] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0183] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0184] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB port 13A. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0185] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 and provides power to the processor 110, the internal memory 12A, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0186] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0187] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0188] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0189] The modem processor includes a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a medium- or high-frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is passed to the application processor.

[0190] The application processor outputs audio signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of processor 110 and be provided in the same device as mobile communication module 150 or other functional modules.

[0191] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0192] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0193] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information. In this embodiment of the present application, display screen 194 is used to display a blurred snapshot image.

[0194] The internal memory 12A may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0195] Random access memory can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation of DDR SDRAM is generally referred to as DDR5 SDRAM), etc. Non-volatile memory can include disk storage devices and flash memory.

[0196] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0197] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0198] The external memory interface 12B can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 via the external memory interface 12B to implement data storage. For example, files such as music and videos can be stored in the external non-volatile memory.

[0199] The electronic device 100 can implement audio functions such as music playback and recording through the audio processing module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0200] The audio processing module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio processing module 170 can also be used to encode and decode audio signals. In some embodiments, the audio processing module 170 can be provided in the processor 110, or some functional modules of the audio processing module 170 can be provided in the processor 110.

[0201] Speaker 170A, also known as a "speaker," allows electronic device 100 to listen to music or make hands-free calls through speaker 170A. Receiver 170B, also known as a "handset," allows electronic device 100 to receive a call or receive a voice message by placing receiver 170B close to one's ear. Microphone 170C, also known as a "microphone," allows a user to speak by placing their mouth close to microphone 170C to input the voice signal.

[0202] The headphone jack 170D is used to connect a wired headphone and can be the USB port 13A, or a 3.5mm open mobile terminal platform (OMTP) standard port or a cellular telecommunications industry association of the USA (CTIA) standard port.

[0203] The pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor 180A can be set on the display screen 194. The gyroscope sensor 180B can be used to determine the movement posture of the electronic device 100. The air pressure sensor 180C is used to measure air pressure. The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). The distance sensor 180F is used to measure the distance. The proximity light sensor 180G can include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The ambient light sensor 180L is used to sense the brightness of the ambient light. The fingerprint sensor 180H is used to collect fingerprints. The temperature sensor 180J is used to detect the temperature. The touch sensor 180K is also called a "touch device". Touch sensor 180K can be mounted on display screen 194. Together, touch sensor 180K and display screen 194 form a touch screen, also known as a "touch screen." Touch sensor 180K is used to detect touch operations applied to or near it. Bone conduction sensor 180M can capture vibration signals.

[0204] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0205] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or touch vibration feedback. Indicator 192 can be an indicator light that can be used to indicate charging status, power level changes, messages, missed calls, notifications, etc.

[0206] SIM card interface 195 is used to connect a SIM card. A SIM card can be connected to and disconnected from electronic device 100 by inserting or removing it from SIM card interface 195. Electronic device 100 may support one or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 may support Nano SIM cards, Micro SIM cards, and SIM cards.

[0207] Terms in the description, claims and drawings of this application

[0208] "User interface (UI)" is the media interface for interaction and information exchange between applications or operating systems and users. It realizes the conversion between the internal form of information and the form acceptable to users. The user interface of an application is source code written in specific computer languages ​​such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that users can recognize, such as pictures, text, buttons and other controls. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, pictures and text. The properties and contents of controls in the interface are defined through tags or nodes, such as XML through <textview> 、 <imgview> 、 <videoview>Nodes such as <head> and <body> are used to specify the controls contained in the interface. A node corresponds to a control or attribute in the interface, and the node is presented as user-visible content after parsing and rendering. In addition, many applications, such as hybrid applications, usually also contain web pages in their interfaces. A web page, also known as a page, can be understood as a special control embedded in the application interface. A web page is a source code written in a specific computer language, such as hypertext markup language (GTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed as user-recognizable content by a browser or a web page display component with similar functions to a browser. The specific content contained in a web page is also defined by tags or nodes in the web page source code, such as GTML through 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0209] A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operations that uses graphics. It can be an icon, window, control, or other interface element displayed on the display of an electronic device. Controls can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, and other visual interface elements.

[0210] As used in the specification and appended claims of the present application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items. As used in the above embodiments, the term "when..." can be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...". Similarly, the phrase "when determining..." or "if (stated condition or event) is detected" can be interpreted to mean "if determining..." or "in response to determining..." or "when (stated condition or event) is detected" or "in response to detecting (stated condition or event)", depending on the context.

[0211] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).

[0212] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A shooting method, characterized in that, Including: When a user operation acting on the defocus capture control is detected in the user interface of the camera application, in response to the user operation, start defocus capture and call the imaging device to collect preview stream data of the shooting scene; Based on the preview stream data, determine the statistical information of the shooting scene; In response to the statistical information satisfying the mode switching condition, switch the defocus capture processing mode from the first processing mode to the second processing mode; In response to a defocus capture instruction for the shooting scene, call the imaging device to capture a to-be-processed image of the shooting scene; Based on the second processing mode, process the to-be-processed image to obtain a defocus capture image of the shooting scene; In the thumbnail display area in the user interface of the camera application, display a thumbnail of the defocus capture image of the shooting scene.

2. The method according to claim 1, characterized in that, The defocus capture processing mode includes a high dynamic range mode and a depth calculation mode. The high dynamic range mode in the second processing mode is different from the high dynamic range mode in the first processing mode, and / or the depth calculation mode in the second processing mode is different from the depth calculation mode in the first processing mode.

3. The method according to claim 1, wherein The high dynamic range mode in the first processing mode is an interleaved high dynamic range mode, and the depth calculation mode in the first processing mode is a binocular depth calculation mode.

4. The method according to any one of claims 1 to 3, characterized in that The statistical information includes a scene brightness statistical value and a dynamic range statistical value. The statistical information satisfying the mode switching condition includes that the scene brightness statistical value is less than a scene brightness threshold, and the dynamic range statistical value is less than a dynamic range threshold; The switching of the defocus capture processing mode from the first processing mode to the second processing mode includes: Switch the high dynamic range mode in the defocus capture processing mode from the interleaved high dynamic range mode to the dual-gain high dynamic range mode.

5. The method according to any one of claims 1 to 3, characterized in that, The statistical information includes a flicker intensity detection statistical value. The statistical information satisfying the mode switching condition includes that the flicker intensity detection statistical value is less than a flicker intensity threshold; The switching of the defocus capture processing mode from the first processing mode to the second processing mode includes: Switch the high dynamic range mode in the defocus capture processing mode from the interleaved high dynamic range mode to the dual-gain high dynamic range mode.

6. The method according to claim 4 or 5, characterized in that, The method is applied to an electronic device, and the electronic device includes a main path sensor; The switching of the high dynamic range mode in the defocus capture processing mode from the interleaved high dynamic range mode to the dual-gain high dynamic range mode includes: Control the working mode of the main path sensor to switch from the interleaved high dynamic range mode to the dual-gain high dynamic range mode.

7. The method according to any one of claims 1 to 3, characterized in that The statistical information includes a statistical value of medium depth of field in the captured object contour layer. The statistical information satisfying the mode switching condition includes that the statistical value of medium depth of field in the captured object contour layer is less than a depth of field threshold; The switching of the defocus capture processing mode from the first processing mode to the second processing mode includes: Switch the depth calculation mode in the defocus capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode.

8. The method according to claim 7, wherein The method is applied to an electronic device, and the electronic device includes a binocular depth calculation path and a monocular depth calculation path; The switching of the depth calculation mode in the defocus capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: Controlling the depth calculation path of the defocus capture processing mode to switch from the binocular depth calculation path to the monocular depth calculation path.

9. The method according to any one of claims 1 to 3, characterized in that The statistical information includes the statistical value of the flicker intensity detection and the statistical value of the medium depth of field in the captured object contour layer. The satisfaction of the mode switching condition by the statistical information includes that the statistical value of the flicker intensity detection is less than the flicker intensity threshold, and the statistical value of the medium depth of field in the captured object contour layer is less than the depth of field threshold; The switching of the defocus capture processing mode from the first processing mode to the second processing mode includes: Switching the high dynamic range mode in the defocus capture processing mode from the interleaved high dynamic range mode to the dual gain high dynamic range mode, and switching the depth calculation mode in the defocus capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode.

10. The method according to claim 9, characterized in that The method is applied to an electronic device, and the electronic device includes a main path sensor, a secondary path sensor, a binocular depth calculation path, and a monocular calculation path; The switching of the high dynamic range mode in the defocus capture processing mode from the interleaved high dynamic range mode to the dual gain high dynamic range mode includes: Controlling the working mode of the main path sensor to switch from the interleaved high dynamic range mode to the dual gain high dynamic range mode, and controlling the working mode of the secondary path sensor to switch from the working out-flow mode to the non-out-flow waiting mode; The switching of the depth calculation mode in the defocus capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: Controlling the depth calculation path of the defocus capture processing mode to switch from the binocular depth calculation path to the monocular calculation path.

11. The method according to claim 10, wherein The electronic device further includes a multi-exposure fusion path and a single-exposure fusion path; Before the switching of the depth calculation mode in the defocus capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode, it further includes: Controlling the exposure fusion path of the defocus capture processing mode to switch from the multi-exposure fusion path to the single-exposure fusion path.

12. An electronic device, characterized in that, Comprising one or more processors and one or more memories; wherein, the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the method according to any one of claims 1-11 is executed.

13. A chip system, characterized in that, The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processors are used to call computer instructions to cause the electronic device to execute the method according to any one of claims 1-11.

14. A computer-readable storage medium, comprising instructions, characterized in that, When the instructions run on the electronic device, the method according to any one of claims 1-11 is executed.

Images