Shooting method, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a shooting method, electronic equipment and a storage medium, which can be applied to electronic equipment supporting blurring snapshot, such as a smart phone, a tablet computer and the like. And blurring snapshot is carried out, namely, depth calculation and portrait blurring processing are carried out on the snapshot image, and optionally, dynamic range fusion processing is carried out. The method comprises the following steps: in response to a detected user operation acting on a blurring snapshot control, starting blurring snapshot and calling a camera device to collect preview stream data of a shooting scene; determining statistical information of the shooting scene based on the preview stream data; switching the blurring snapshot processing mode from the first processing mode to a second processing mode in response to the condition that the statistical information meets the mode switching condition; and processing the to-be-processed image based on the second processing mode in response to a blurring snapshot instruction for the shooting scene to obtain a blurring snapshot image of the shooting scene. By adopting the embodiment of the invention, the power consumption of the electronic equipment can be saved in the blurred snapshot scene.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of electronic devices, and in particular, to a shooting method, electronic device and storage medium. Background Art

[0002] With the development of photography technology, electronic devices with photography functions can provide a variety of photography methods, such as portraits, night scenes, selfies, videos, snapshots, etc. Among them, snapshot refers to the electronic device with photography function quickly capturing the natural, vivid, and expressive instant image of the subject without the subject's knowledge. For example, capturing the wonderful instant image of someone at a certain moment, capturing the instant image of a traffic violation, or capturing the instant image of a wonderful goal in a football game, etc.

[0003] Compared with other shooting methods, snapshot will increase the power consumption of electronic devices to a certain extent. Therefore, for snapshot, how to save the power consumption of electronic devices is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a shooting method, an electronic device and a storage medium, which can save the power consumption of the electronic device in a blurred snapshot scenario.

[0005] In the first aspect, the embodiment of the present application provides a shooting method, which can be applied to an electronic device, and the electronic device can support blur capture. Blur capture means performing depth calculation and portrait blur processing on the captured image, and optionally, dynamic range fusion processing is also performed. Blur capture can improve the clarity of the subject (such as a portrait) compared with ordinary capture. The method may include: detecting a user operation acting on a blur capture control in the user interface of a camera application, and in response to the user operation, starting blur capture and calling a camera device to collect preview stream data of the shooting scene; based on the preview stream data, determining statistical information of the shooting scene; in response to the statistical information satisfying the mode switching condition, switching the blur capture processing mode from the first processing mode to the second processing mode; in response to the blur capture instruction of the shooting scene, calling the camera device to capture and obtain the image to be processed of the shooting scene; based on the second processing mode, processing the image to be processed to obtain a blurred captured image of the shooting scene; in the thumbnail display area in the user interface of the camera application, displaying a thumbnail of the blurred captured image of the shooting scene.

[0006] By implementing the shooting method provided in the first aspect, in a blurred snapshot scenario, when the statistical information of the shooting scene meets the mode switching conditions, the blurred snapshot processing mode can be switched from the first processing mode to the second processing mode, and then the captured image to be processed can be processed based on the second processing mode, thereby saving power consumption of the electronic device in the blurred snapshot scenario.

[0007] In combination with the method provided in the first aspect, in some embodiments, the above-mentioned defocus snapshot processing mode includes a high dynamic range mode and a depth calculation mode. The high dynamic range mode in the second processing mode is different from the high dynamic range mode in the first processing mode, and / or the depth calculation mode in the second processing mode is different from the depth calculation mode in the first processing mode. So that the electronic device can adopt different high dynamic range modes and / or different depth calculation modes in different shooting scenarios, which helps to save the power consumption of the electronic device.

[0008] Among them, the high dynamic range mode refers to the high dynamic range mode of the sensor, which can be divided into an interleaved high dynamic range mode and a dual-gain high dynamic range mode. The depth calculation mode can be divided into a binocular depth calculation mode and a monocular depth calculation mode.

[0009] In combination with the method provided in the first aspect, in some embodiments, the high dynamic range mode in the above-mentioned first processing mode is an interleaved high dynamic range mode, and the depth calculation mode in the first processing mode is a binocular depth calculation mode. The power consumption in the interleaved high dynamic range mode is more than that in the dual-gain high dynamic range mode, and the power consumption in the binocular depth calculation mode is more than that in the monocular depth calculation mode. That is, the power consumption of the electronic device adopting the first processing mode is relatively large, which will accelerate the power consumption speed. Switching from the first processing mode to the second processing mode can slow down the power consumption speed, thereby saving the power consumption of the electronic device.

[0010] In combination with the method provided in the first aspect, in some embodiments, the above-mentioned statistical information includes a scene brightness statistical value and a dynamic range statistical value. The satisfaction of the mode switching condition by the above-mentioned statistical information includes that the scene brightness statistical value is less than the scene brightness threshold, and the dynamic range statistical value is less than the dynamic range threshold. In this case, the high dynamic range mode in the defocus snapshot processing mode is switched from the interleaved high dynamic range mode to the dual-gain high dynamic range mode to save the power consumption of the electronic device.

[0011] It can be understood that the high dynamic range mode in the second processing mode is a dual-gain high dynamic range mode. The depth calculation mode in the second processing mode can be a binocular depth calculation mode (that is, the same as the depth calculation mode in the first processing mode), or a monocular depth calculation mode (that is, switched from the binocular depth calculation mode to the monocular depth calculation mode). Switching from the binocular depth calculation mode to the monocular depth calculation mode can be triggered by the statistical value of the medium depth of field in the contour layer of the object to be photographed.

[0012] Among them, the scene brightness threshold and the dynamic range threshold can be understood as pre-calibrated thresholds, which can be set in the electronic device when it leaves the factory, or can be set based on the values input by the user. The specific values are not limited in the embodiments of the present application.

[0013] In combination with the method provided in the first aspect, in some embodiments, the statistical information includes a flicker intensity detection statistic, and the statistical information satisfies a mode switching condition including that the flicker intensity detection statistic is less than a flicker intensity threshold. In this case, the high dynamic range mode in the blur capture processing mode is switched from the interlaced high dynamic range mode to the dual gain high dynamic range mode to save power consumption of the electronic device.

[0014] It can be understood that the high dynamic range mode in the second processing mode is a dual-gain high dynamic range mode, and the depth calculation mode in the second processing mode can be a binocular depth calculation mode (i.e., the same as the depth calculation mode in the first processing mode) or a monocular depth calculation mode (i.e., switching from the binocular depth calculation mode to the monocular depth calculation mode). Switching from the binocular depth calculation mode to the monocular depth calculation mode can be triggered by the statistical value of the medium depth of field in the contour layer of the subject.

[0015] Among them, the flicker intensity threshold can be understood as a pre-calibrated threshold, which can be set in the electronic device when it leaves the factory, or it can be set based on a numerical value input by the user. The specific numerical value is not limited in the embodiments of the present application.

[0016] In combination with the method provided in the first aspect, in some embodiments, the electronic device includes a main sensor, and the high dynamic range mode in the blur capture processing mode is switched from the interlaced high dynamic range mode to the dual-gain high dynamic range mode, including: controlling the working mode of the main sensor to switch from the interlaced high dynamic range mode to the dual-gain high dynamic range mode. The working mode of the main sensor is switched to the dual-gain high dynamic range mode to save power consumption of the main sensor, thereby saving power consumption of the electronic device.

[0017] Optionally, the electronic device further includes an auxiliary sensor. If the depth calculation mode in the second processing mode is a monocular depth calculation mode, the working mode of the auxiliary sensor can be controlled to switch from an outflow working mode to a non-outflow waiting mode to save power consumption of the auxiliary sensor, thereby saving power consumption of the electronic device. If the depth calculation mode in the second processing mode is a binocular depth calculation mode, the working mode of the auxiliary sensor can be controlled to be an outflow working mode.

[0018] In combination with the method provided in the first aspect, in some embodiments, the statistical information includes a statistical value of the medium depth of field in the subject contour layer, and the statistical information satisfies the mode switching condition including that the statistical value of the medium depth of field in the subject contour layer is less than the depth of field threshold. In this case, the depth calculation mode in the blur capture processing mode is switched from the binocular depth calculation mode to the monocular depth calculation mode to reduce the power consumption of the depth calculation, thereby saving the power consumption of the electronic device.

[0019] It can be understood that the depth calculation mode in the second processing mode is switched to the monocular depth calculation mode. The high dynamic range mode in the second processing mode can be the interleaved high dynamic range mode (i.e., the same as the high dynamic range mode in the first processing mode), or the dual-gain high dynamic range mode (i.e., switched from the interleaved high dynamic range mode to the dual-gain high dynamic range mode). The switch from the interleaved high dynamic range mode to the dual-gain high dynamic range mode can be triggered by the scene brightness statistical value and the dynamic range statistical value, or by the flicker intensity detection statistical value, or by the scene brightness statistical value, the dynamic range statistical value, and the flicker intensity detection statistical value.

[0020] Among them, the depth of field threshold can be understood as a pre-calibrated threshold, which can be set in the electronic device at the time of factory shipment of the electronic device, or can be set based on the value input by the user. The specific value is not limited in the embodiments of the present application.

[0021] Combined with the method provided in the first aspect, in some embodiments, the electronic device includes a binocular depth calculation path and a monocular depth calculation path; switching the depth calculation mode in the virtual capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: controlling the depth calculation path of the virtual capture processing mode to switch from the binocular depth calculation path to the monocular depth calculation path. Switching from the binocular depth calculation path to the monocular depth calculation path can save the power consumption of the electronic device.

[0022] Among them, the binocular depth calculation path is used to execute the binocular depth calculation mode, and the monocular depth calculation path is used to execute the monocular depth calculation mode.

[0023] Combined with the method provided in the first aspect, in some embodiments, the above statistical information includes the flicker intensity detection statistical value and the statistical value of the medium depth of field in the captured object contour layer. The above statistical information satisfying the mode switching condition includes that the flicker intensity detection statistical value is less than the flicker intensity threshold, and the statistical value of the medium depth of field in the captured object contour layer is less than the depth of field threshold. In this case, the high dynamic range mode in the virtual capture processing mode is switched from the interleaved high dynamic range mode to the dual-gain high dynamic range mode, and the depth calculation mode in the virtual capture processing mode is switched from the binocular depth calculation mode to the monocular depth calculation mode. The working mode of the main path sensor is switched to the dual-gain high dynamic range mode to save the power consumption of the main path sensor, and the depth calculation mode is switched from the binocular depth calculation mode to the monocular depth calculation mode to reduce the depth calculation power consumption, thereby saving the power consumption of the electronic device.

[0024] Combined with the method provided in the first aspect, in some embodiments, the above statistical information includes the statistical value of scene brightness, the statistical value of dynamic range, and the statistical value of medium depth of field in the captured object contour layer. The satisfaction of the mode switching condition by the above statistical information includes that the statistical value of scene brightness is less than the scene brightness threshold, the statistical value of dynamic range is less than the dynamic range threshold, and the statistical value of medium depth of field in the captured object contour layer is less than the depth of field threshold. In this case, the high dynamic range mode in the defocus capture processing mode is switched from the interleaved high dynamic range mode to the dual-gain high dynamic range mode, and the depth calculation mode in the defocus capture processing mode is switched from the binocular depth calculation mode to the monocular depth calculation mode. The working mode of the main path sensor is switched to the dual-gain high dynamic range mode to save the power consumption of the main path sensor, and the depth calculation mode is switched from the binocular depth calculation mode to the monocular depth calculation mode to reduce the depth calculation power consumption, thereby saving the power consumption of the electronic device.

[0025] Combined with the method provided in the first aspect, in some embodiments, the electronic device includes a main path sensor, a secondary path sensor, a binocular depth calculation path, and a monocular calculation path. Switching the high dynamic range mode in the defocus capture processing mode from the interleaved high dynamic range mode to the dual-gain high dynamic range mode includes: controlling the working mode of the main path sensor to switch from the interleaved high dynamic range mode to the dual-gain high dynamic range mode, and controlling the working mode of the secondary path sensor to switch from the working out-flow mode to the non-out-flow waiting mode. Switching the depth calculation mode in the defocus capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: controlling the depth calculation path of the defocus capture processing mode to switch from the binocular depth calculation path to the monocular calculation path. Thus, the power consumption of the electronic device is saved by saving the power consumption of the main path sensor and the secondary path sensor and reducing the depth calculation power consumption.

[0026] Combined with the method provided in the first aspect, in some embodiments, the electronic device further includes a multi-exposure fusion path and a single-exposure fusion path; before switching the depth calculation mode in the defocus capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode, controlling the exposure fusion path of the defocus capture processing mode to switch from the multi-exposure fusion path to the single-exposure fusion path to further save the power consumption of the electronic device.

[0027] In a second aspect, an embodiment of the present application provides an electronic device, which includes one or more processors and one or more memories; wherein, the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device is caused to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0028] In a third aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the processors are configured to call computer instructions to cause the electronic device to execute the methods described in the first aspect and any possible implementation manner of the first aspect.

[0029] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions, which when running on an electronic device, cause the electronic device to execute the methods described in the first aspect and any possible implementation manner of the first aspect.

[0030] In a fifth aspect, an embodiment of the present application provides a computer program product containing instructions, which when running on an electronic device, cause the electronic device to execute the methods described in the first aspect and any possible implementation manner of the first aspect.

[0031] It can be understood that the electronic device provided in the second aspect, the chip system provided in the third aspect, the computer storage medium provided in the fourth aspect, and the computer program product provided in the fifth aspect are all used to execute the method provided in the first aspect of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figures 1A - 1F are schematic diagrams of a set of user interfaces provided by an embodiment of the present application;

[0033] Figures 2A - 2F are schematic diagrams of a set of shooting scenes provided by an embodiment of the present application;

[0034] Figure 3 is a software architecture diagram of an electronic device provided by an embodiment of the present application;

[0035] Figure 4 is a flowchart of a shooting method provided by an embodiment of the present application;

[0036] Figures 5A - 5D are flowcharts of several processing modes provided by an embodiment of the present application;

[0037] Figure 6 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application.

[0039] 1. Blur capture

[0040] Ordinary capture means that, without the knowledge of the object to be photographed (such as a person or an animal), an electronic device with a photographing function quickly captures a natural, vivid and expressive instantaneous image of the object to be photographed. Blur capture means that, without the knowledge of the object to be photographed, an electronic device with a photographing function quickly captures an instantaneous image of the object to be photographed, and further performs depth calculation and portrait blurring processing on the instantaneous image to obtain an instantaneous blurred image. Optionally, dynamic range fusion processing can also be performed after or before the portrait blurring processing to obtain an instantaneous high-dynamic-range blurred image. Among them, portrait blurring means highlighting the detailed features of the portrait in the image and blurring the features of the background in the image to improve the clarity of the portrait.

[0041] That is to say, blur capture is based on ordinary capture and adds image processing processes such as depth calculation, portrait blurring, and dynamic range fusion to improve the display effect of the instantaneous image, such as improving the clarity of the object to be photographed in the instantaneous image and improving the dynamic range of the instantaneous image. It can be understood that for the same object to be photographed in the same shooting scene at the same time, in the image obtained by blur capture, the clarity of the object to be photographed is higher, the dynamic range is higher, and the image display effect is better.

[0042] Since blur capture adds a series of image processing processes on the basis of ordinary capture, for the electronic device, the speed of power consumption of blur capture is faster. That is to say, blur capture consumes more power than ordinary capture.

[0043] 2. High Dynamic Range (HDR) mode of the sensor

[0044] Dynamic Range (DR), in the field of image processing, refers to the range from "darkest" to "brightest" contained in an image. The larger the dynamic range, the richer the details of the image. Compared with the dynamic range, the high dynamic range improves the brightness range of the image, enables the range of highlights and shadows in the image to be wider, and makes the image details more plump. That is to say, HDR can make both the bright and dark parts in the image be displayed.

[0045] The sensor refers to an image sensor. The accuracy of the sensor is different, and the dynamic range is also different. For the convenience of description, in the embodiments of the present application, the image sensor is simply referred to as the sensor.

[0046] The HDR mode of the sensor refers to the processing mode adopted by the sensor when processing HDR frames. The HDR mode of the sensor can be mainly divided into the following modes:

[0047] (1) Stagger HDR mode

[0048] The staggered HDR mode means that after the sensor reads a row of long-exposure data of an HDR frame, it immediately performs a short exposure on this row. The sensor reads the data of each row with different exposure times in an interleaved manner to save waiting time. That is to say, in the staggered HDR mode, the sensor exposes two frames of data successively, one frame is long-exposure data and the other frame is short-exposure data. Successive exposure can also be described as interleaved exposure. One frame being long-exposure data can also be described as one frame being a long frame, and the other frame being short-exposure data can also be described as the other frame being a short frame.

[0049] The exposure times of the two frames (i.e., the long frame and the short frame) can be configured, and the gains of the two frames can also be configured. For example, when the exposure time of the long frame is long and the interval between the exposure time of the long frame and the exposure time of the short frame is long, it is easy to introduce ghosting problems.

[0050] (2) Dual Conversion Gain (DCG) mode

[0051] The DCG mode means that the sensor exposes two frames of data simultaneously, and the gains of these two frames are different. In the DCG mode, the sensor controls the conversion circuit inside it to make the gains of these two frames different.

[0052] The DCG mode is a pixel-level dual-gain mode, and the dual gains include High Conversion Gain (HCG) and Low Conversion Gain (LCG).

[0053] (3) Dual Amplifier Gain (DAG) mode

[0054] The DAG mode means that the sensor exposes two frames of data simultaneously, and the gains of these two frames are different. In the DAG mode, the sensor controls the amplifier circuit inside it to make the gains of these two frames different.

[0055] The DAG mode is a circuit-level dual-gain mode, and the dual gains include HCG and LCG.

[0056] The DCG mode and the DAG mode can be collectively referred to as the DXG mode. That is to say, the DXG mode includes the DCG mode and / or the DAG mode. Then the DXG mode means that the sensor controls its hardware circuit to make the gains of the two exposed frames of data different. The DXG mode can also be described as a dual-gain mode.

[0057] 3. Depth calculation mode

[0058] Depth calculation can also be described as visual depth estimation or depth estimation. The purpose of depth estimation is to estimate the depth of the scene in the image, that is, the vertical distance from each pixel point in the scene to the camera imaging plane. The distance can be divided into absolute distance and relative distance. The depth calculation mode can be divided into monocular depth calculation mode and binocular depth calculation mode.

[0059] (1) Monocular depth calculation mode

[0060] The monocular depth calculation mode can also be described as the monocular depth estimation mode. The monocular depth estimation mode refers to obtaining the depth of the scene in the image through a single image. The monocular depth calculation mode can adopt an absolute depth estimation algorithm or a relative depth estimation algorithm. These two algorithms can be implemented through networks respectively, such as an absolute depth estimation network, a relative depth estimation network, etc. Among them, the relative depth estimation network learns to extract the depth difference information between adjacent pixels, and the absolute depth estimation network is used to predict the absolute depth value.

[0061] (2) Binocular depth calculation mode

[0062] The binocular depth calculation mode can also be described as the binocular depth estimation mode. The binocular depth estimation mode refers to taking two images as input, and estimating the depth value of each pixel point in the image after calculating the cost volume through disparity. These two images are images of the same scene taken at the same time, such as images of the same scene taken by two cameras (such as a left camera and a right camera) at the same time. Binocular depth calculation can include processes such as feature point matching, epipolar rectification, disparity calculation, and disparity to depth conversion. These processes involve complex calculation processes such as multiple Convolutional Neural Networks (CNNs), network modeling, and feature fusion.

[0063] Since the calculation complexity and calculation intensity of the binocular depth calculation mode are both higher than those of the monocular depth calculation mode, the electronic device consumes more power when adopting the binocular depth calculation mode than when adopting the monocular depth calculation mode.

[0064] In the virtual capture scene, the power consumption speed of the electronic device is higher than that in the ordinary capture scene. Then how to save the power consumption in the virtual capture scene is a technical problem to be solved urgently.

[0065] To solve the above problems, an embodiment of the present application provides a shooting method. This method can be applied to an electronic device with this image processing ability. The above-mentioned electronic device (i.e., electronic device 100) is, for example, a mobile phone, a tablet computer, etc.

[0066] When implementing the shooting method provided in the embodiments of the present application, after activating the virtualization capture, if the statistical information of the shooting scene meets the mode switching condition, the virtualization capture processing mode can be switched to a more power-saving processing mode, and the display effect of the virtualization capture image processed in this processing mode is ensured.

[0067] Not limited to mobile phones and tablet computers, the electronic device 100 can also be a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, as well as a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device, and / or a smart city device. The embodiments of the present application do not impose special restrictions on the specific type of this electronic device.

[0068] Figures 1A - 1F Exemplarily shows a set of user interfaces on the electronic device 100. Next, in conjunction with Figures 1A - 1F Specifically introduce the application scenarios of implementing the shooting method provided in the embodiments of the present application.

[0069] First, Figure 1A Exemplarily shows the user interface on the electronic device 100 that displays the installed application programs, that is, the home page. As Figure 1A shown, one or more application program icons are displayed on the home page, such as the "Clock" application program icon, the "Calendar" application program icon, the "Weather" application program icon, and so on.

[0070] One or more of the above application program icons include the "Camera" application program (hereinafter simply referred to as "Camera") icon, that is, icon 111. The electronic device 100 can detect a user operation acting on icon 111. This user operation is, for example, a click operation. In response to this user operation, the electronic device 100 can turn on the camera, call the camera to capture an image, and display Figure 1B the shown user interface.

[0071] In another implementation, the lock screen user interface of the electronic device 100 displays a "Camera" icon. The electronic device 100 can detect a user operation acting on the "Camera" icon. This user operation can be, for example, an operation of clicking and pulling up to the top of the lock screen user interface. In response to this user operation, the electronic device 100 can turn on the camera, call the camera to capture an image, and display Figure 1BThe user interface shown.

[0072] Figure 1B Exemplarily shown is the user interface when the "Camera" application is running on the electronic device 100. The user interface may include a window 121, shooting controls 122, a thumbnail display area 123, and an inversion control 124. The user interface may also include some switch controls, such as a setting control, a filter switch control, an AI shooting switch control, a flash switch control, etc. Figure 1B In it, the filter switch control, the AI shooting switch control, and the flash switch control are all taken as examples of being turned off.

[0073] Among them, the window 121 is used to display the image captured by the camera, which can be understood as the display area for the captured image, so that the user can input a click operation to the shooting control 122 or the inversion control 124 according to the picture displayed in the window 121. In some embodiments, the window 121 also includes some controls, such as a motion capture control 125, a focal length adjustment control 126, and a portrait blurring control 127, etc. When the electronic device 100 detects a user operation acting on the focal length adjustment control, it can respond to the user operation and adjust the focal length of the camera, for example, from 1x to 2x.

[0074] When the electronic device 100 detects a user operation acting on the motion capture control 125, it can respond to the user operation, start motion capture and output a text prompt message of "Motion capture has been enabled", and control the color change of the portrait in the motion capture control 125 and the colored portrait to be in a running state. After starting motion capture, the electronic device 100 continuously detects whether the movement amplitude of the object to be photographed in the shooting environment meets the capture condition. If the capture condition is met, it can capture the instantaneous image at this moment. If the capture condition is not met, it can continue to collect images. Exemplarily, if the amplitude of the smile at the corner of the mouth of the object to be photographed in the shooting environment is greater than the amplitude threshold, then the electronic device 100 can capture the image of this smiling moment.

[0075] Figure 1C Exemplarily shown is the user interface of the electronic device 100 after detecting a user operation acting on the motion capture control 125. Figure 1C In it, the color of the portrait in the motion capture control 125 is different from Figure 1B the color of the portrait in the motion capture control 125 in Figure 1C The window 121 in Figure 1C displays a text prompt message of "Motion capture has been enabled". And

[0076] If the electronic device 100 detects a user operation on the automatic motion capture switch control 128, it can respond to the user operation and turn off the automatic motion capture. At this time, the person in the motion capture control 125 continues to be in a running state, but the electronic device 100 does not automatically detect and capture. Instead, it responds to the user operation on the capture control 122 and performs the capture.

[0077] In the case of starting motion capture, if the electronic device 100 detects a user operation on the portrait blur control 127, it can respond to the user operation, start the blur capture, output a text prompt message of "Blur capture has been enabled", and control the background color change in the portrait blur control 127.

[0078] Figure 1D Exemplarily shows the user interface of the electronic device 100 after detecting a user operation on the motion capture control 125 and then detecting a user operation on the portrait blur control 127. Figure 1D In it, the color of the person in the portrait blur control 127 is the same as Figure 1C the color of the person in the portrait blur control 127 in Figure 1D The window 121 in shows a text prompt message of "Blur capture has been enabled". And Figure 1D The switch control bar in shows an automatic blur capture switch control 129, which is in the on state. The automatic blur capture switch control 129 being in the on state means that the electronic device 100 can automatically detect whether the movement amplitude of the object to be photographed in the shooting environment meets the capture condition, capture when the capture condition is met, and perform portrait blur processing on the captured image.

[0079] If the electronic device 100 detects a user operation on the automatic blur capture switch control 129, it can respond to the user operation and turn off the automatic blur capture. At this time, the person in the motion capture control 125 continues to be in a running state, and the color of the person in the portrait blur control 127 continues to be as Figure 1D shown, but the electronic device 100 does not automatically detect and capture. Instead, it responds to the user operation on the capture control 122 and performs the blur capture.

[0080] Figures 1B to 1DThe change process is that the electronic device 100 first detects a user operation on the motion capture control 125 and starts motion capture, and then detects a user operation on the portrait blurring control 127. That is to say, the blurring capture control includes the motion capture control 125 and the portrait blurring control 127. In response to detecting user operations on these two controls respectively, blurring capture can be started. This is one implementation method. In another implementation method, the blurring capture control is just one control. After the electronic device 100 detects a user operation on this control, blurring capture can be started.

[0081] Figure 1E Exemplarily shows the user interface when the "Camera" application is running on the electronic device 100. The user interface may include a window 121, a shooting control 122, a thumbnail display area 123, and a reverse control 124. The user interface may also include some switch controls, such as a setting control, a filter switch control, an AI camera switch control, a flash switch control, etc. Figure 1E In it, the filter switch control, the AI camera switch control, and the flash switch control are all shown as being turned off.

[0082] Figure 1E The window 121 shown includes a blurring capture control 130. The blurring capture control 130 can be understood as a control integrating the motion capture control and the portrait blurring control, and this control is used to turn on blurring capture.

[0083] When the electronic device 100 detects a user operation on the blurring capture control 130, it can respond to this user operation, start blurring capture and output a text prompt message of "Blurring capture has been turned on", and control the color change of the portrait in the blurring capture control 130 and the running state of the colored portrait. After starting blurring capture, the electronic device continuously detects whether the movement amplitude of the object to be photographed in the shooting environment meets the capture condition. If the capture condition is met, this instantaneous image can be captured, and the instantaneous image is subjected to portrait blurring processing.

[0084] Figure 1F Exemplarily shows the user interface of the electronic device 100 after detecting a user operation on the blurring capture control 130. Figure 1F In it, the color of the portrait in the blurring capture control 130 is different from Figure 1D the color of the portrait in the blurring capture control 130 in, and the portrait in the blurring capture control 130 is in a running state. Figure 1F The window 121 in shows the text prompt message of "Blurring capture has been turned on". And Figure 1FThe switch control bar in [the above description] displays an automatic defocusing capture switch control 131, which is in the on state. The fact that the automatic defocusing capture switch control 131 is in the on state means that the electronic device 100 can automatically detect whether the movement amplitude of the object to be photographed in the shooting environment meets the capture condition, and capture when the capture condition is met, and perform portrait defocusing processing on the captured image.

[0085] If the electronic device 100 detects a user operation on the automatic defocusing capture switch control 131, it can respond to this user operation and turn off the automatic defocusing capture. At this time, the portrait in the defocusing capture control 130 continues to be in a running state, but the electronic device 100 does not automatically detect and capture, but responds to a user operation on the shooting control 122 and performs defocusing capture.

[0086] Such as Figures 1B to 1F The user interface shown also includes "night scene" mode, "portrait" mode, "photo" mode, "video recording" mode. These modes are for example purposes and do not constitute a limitation to the embodiments of this application. For example, in actual applications, there may also be a "multi-camera video recording" mode, etc. Among them, the "video recording" mode is used to record video files, and the "multi-camera video recording" mode is used to record video files when the front camera and the rear camera are both turned on. The "night scene" mode is used to shoot night scene images and improve the clarity of night scene images. The "portrait" mode is mainly used to shoot portrait images, and in this mode, the image processing method provided by the embodiments of this application can be adopted. The "photo" mode is used to shoot single-frame images. Figures 1B to 1F In [the above situation], the mode selected by the user is the "photo" mode.

[0087] After the electronic device 100 performs automatic defocusing capture or passive defocusing capture, a thumbnail of the defocusing capture image (that is, the image obtained by performing portrait defocusing processing on the captured instantaneous image) can be displayed in the thumbnail display area 123. When the electronic device 100 detects a user operation on the thumbnail display area 123, it responds to this user operation and jumps to the user interface for browsing the defocusing capture image.

[0088] By using the shooting method provided by the embodiments of this application, the thumbnail display area 123 can display the thumbnail of the defocusing capture image, or save the defocusing capture image in the gallery.

[0089] The shooting method provided by the embodiments of this application can be applied to one or more of the following shooting scenarios.

[0090] Shooting scenario one: Non-flicker scenario

[0091] A shutter is a structure in a camera used to control the effective exposure time of the photosensitive film, which can be divided into a global shutter and a rolling shutter. In a global shutter, all pixels on the entire sensor are exposed simultaneously and end exposure at the same time after the same period. In a rolling shutter, it is a row-by-row scan, where each row undergoes the processes of reset, exposure, and data reading, and then is exposed row by row. Currently, rolling shutter is commonly used in sensors. For all pixels in the same row, the start exposure time and exposure time are the same, that is, the energy received by the pixels in the same row is the same. However, the energy received by pixels in adjacent rows is inconsistent, which can lead to bright and dark stripes in the image. This phenomenon can be called Flicker. That is to say, due to the influence of the light source frequency, bright and dark alternating conditions appear in the image, looking like it is flickering. The light source frequency can be, for example, 50Hz or 60Hz, etc.

[0092] A non-Flicker scenario can be understood as a scenario where there is no influence of the light source frequency, or a scenario where there is no light source, or a scenario where the Flicker intensity is less than a threshold, etc.

[0093] In a non-Flicker scenario, if virtual capture is enabled, then the shooting method provided in the embodiments of the present application can be adopted to save the power consumption of the electronic device.

[0094] Exemplarily, reference can be made to Figure 2A and Figure 2B the shooting scenarios shown. Figure 2A The shooting scenario shown can be understood as a Flicker scenario, Figure 2B The shooting scenario shown can be understood as a non-Flicker scenario. It can be understood that in a scenario where the light is on and the projector is in the playback state, when virtual capture is enabled on the electronic device, the stagger HDR mode can be adopted to shoot this scenario; in a scenario where the light is off and the projector is off, when virtual capture is enabled on the electronic device, the DXG mode can be adopted to shoot this scenario to save the power consumption of the electronic device.

[0095] Shooting Scenario Two: Low-Light Scenario

[0096] A low-light scenario refers to a scenario where the scene brightness of the shooting scenario is relatively low. In such a scenario, when virtual capture is enabled on the electronic device, the monocular depth calculation mode can be adopted to shoot this scenario to save the power consumption of the electronic device. And in such a scenario, the display effect of the virtual capture image processed by the monocular depth calculation mode also meets the display effect of virtual capture. A low-light scenario can also be understood as a scenario acceptable to the monocular depth calculation mode.

[0097] Exemplarily, reference can be made to Figure 2C and Figure 2D the shooting scenes shown Figure 2C The shooting scenes shown can be understood as bright light scenes. For example, the indoor lights are on, or the weather is sunny during the day; Figure 2D The shooting scenes shown can be understood as low light scenes. For example, the indoor lights are off, or it is getting dark in the evening, or the weather is gloomy during the day. For the shooting of window scenes, the shooting scene inside the window may be a low light scene, and the shooting scene outside the window may be a bright light scene. It can be understood that for Figure 2C the shooting scenes shown, when the virtual capture is turned on, the electronic device can use the binocular depth calculation mode to shoot this scene; for Figure 2D the shooting scenes shown, when the virtual capture is turned on, the electronic device can use the monocular depth calculation mode to shoot this scene to save the power consumption of the electronic device.

[0098] Shooting scene three: non-HDR scene

[0099] The non-HDR scene can be understood as a normal DR scene, that is, a scene with little difference between light and dark. Or, for scenes with less demanding requirements for the brightness range and detail presentation of the image. The non-HDR scene can also be understood as a scene with an acceptable dynamic range in the DXG mode.

[0100] Exemplarily, reference can be made to Figure 2E and Figure 2F the shooting scenes shown Figure 2E The shooting scenes shown can be understood as HDR scenes, Figure 2F the shooting scenes shown can be understood as non-HDR scenes. It can be understood that for Figure 2E the shooting scenes shown, when the virtual capture is turned on, the electronic device can use the stagger HDR mode to shoot this scene; for Figure 2F the shooting scenes shown, when the virtual capture is turned on, the electronic device can use the DXG mode to shoot this scene to save the power consumption of the electronic device.

[0101] The above three shooting scenes are for example and do not constitute a limitation to the embodiments of the present application. For example, it can also be applied to the auxiliary road sensor auto exposure (AE) scene, the scene where the auxiliary road sensor extends or shortens the exposure time, etc. The above three shooting scenes can be superimposed. For example, non-HDR scene and low light scene, non-Flicker scene and low light scene, etc.

[0102] It can be understood that for virtualization capture, in some shooting scenarios, the electronic device can switch the HDR mode of the sensor from the stagger HDR mode to the DXG mode, and / or switch the depth calculation mode from the binocular depth calculation mode to the monocular depth calculation mode to save the power consumption of the electronic device.

[0103] The following specifically introduces the specific process of the electronic device 100 to implement Figure 1D or Figure 1F the user interface shown.

[0104] First, Figure 3 An exemplary software architecture of the electronic device 100 is shown.

[0105] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present invention, the Android system with a layered architecture is taken as an example to exemplarily illustrate the software structure of the electronic device 100.

[0106] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely the application layer, the application framework (Framework) layer, the Android runtime and system libraries, the Hardware Abstraction Layer (HAL), and the kernel layer.

[0107] The application layer may include a series of application packages. As Figure 3 shown, the application packages may include applications such as a camera, a gallery, a video, music, a navigation, a calendar, a map, and a WLAN. In the embodiments of the present application, during the operation of the camera, it can provide a user interface as Figures 1B to 1F shown. The electronic device 100 can display the thumbnail of the virtualization capture image in the thumbnail display area 123, or save the virtualization capture image in the gallery. After starting the camera, the electronic device 100 can call the camera to capture an image. The electronic device 100 may include at least one camera.

[0108] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions. In the embodiments of the present application, the application framework layer includes a camera service framework, and the camera service framework may contain some functions that support virtualization capture.

[0109] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files in the application layer and the application framework layer as binary files. The virtual machine is used to manage the object lifecycle, stack management, thread management, security and exception management, and garbage collection and other functions.

[0110] Android Runtime includes the core libraries and the virtual machine. Android runtime is responsible for the scheduling and management of the Android system. The core libraries contain two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core libraries of Android.

[0111] The system libraries can include multiple functional modules. For example: Surface Manager, Media Libraries, 3D graphics processing libraries (such as OpenGL), etc. The Surface Manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The Media Libraries support the playback and recording of multiple common audio and video formats, as well as static image files, etc. The Media Libraries can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc.

[0112] The Hardware Abstract Layer (HAL) is an interface layer located between the kernel layer and the hardware and can be used to control the actions of the hardware. In the embodiments of the present application, the hardware abstract layer may include a defocus capture algorithm for implementing defocus capture. The defocus capture algorithm may include one or more of the algorithms of the stagger HDR mode, the DXG mode, the monocular depth calculation mode, the binocular depth calculation mode, the portrait defocus algorithm, etc.

[0113] Among them, the algorithm of the stagger HDR mode is used to support the stagger HDR mode, and the algorithm may include one or more of the main road sensor long exposure, the main road sensor short exposure, the frame selection and multi-frame fusion algorithm, the multi-exposure fusion algorithm, the auxiliary road sensor exposure, etc.

[0114] The algorithm of the DXG mode is used to support the DXG mode, and the algorithm may include the main road sensor exposure, and the selection and multi-frame fusion algorithm, etc.

[0115] The algorithm of the monocular depth calculation mode is used to support the monocular depth calculation mode, and the algorithm of the binocular depth calculation mode is used to support the binocular depth calculation mode. The portrait defocus algorithm is used to achieve portrait defocus, that is, to highlight the detailed features of the portrait while blurring the features of the background.

[0116] The kernel layer is the foundation of the Android system. For example, ART relies on the kernel layer to perform underlying functions such as threading and low-level memory management. The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, a sensor driver, a GPU driver, etc. In the embodiments of the present application, the kernel layer includes a camera driver and a display driver. The camera driver is used to drive the camera to capture images, and the display driver is used to drive the display screen to display the defocused capture images.

[0117] Based on Figure 3 The software architecture shown above, the shooting method provided by the embodiments of the present application will be described. In some embodiments, the electronic device 100 detects a user operation on the camera application, and in response to the user operation, starts the camera so that the camera captures an image. Detecting this user operation, the camera application will trigger an instruction to start the camera. The camera application calls the API interface of the application framework layer to send this instruction to the camera service framework, and the camera service framework calls the hardware abstraction layer to send this instruction to the camera driver. The camera driver can drive the startup of the camera and drive the camera to capture an image. The image captured by the camera can be cached in the image buffer.

[0118] After the electronic device 100 starts the camera, the camera service framework, in response to detecting a user operation on the defocused capture control in the user interface of the camera application, starts defocused capture and obtains the preview stream data of the shooting scene from the image buffer. The camera service framework transmits the preview stream data to the hardware abstraction layer. The hardware abstraction layer determines the statistical information of the shooting scene based on the preview stream data, and judges whether the statistical information meets the mode switching condition. If it meets, it switches the HDR mode of the sensor from the stagger HDR mode to the DXG mode, and / or switches the depth calculation mode from the binocular depth calculation mode to the monocular depth calculation mode. The camera service framework, in response to the defocused capture instruction for the shooting scene, calls the camera driver to capture the to-be-processed image of the shooting scene, and transmits the to-be-processed image to the hardware abstraction layer. The hardware abstraction layer processes the to-be-processed image based on the switched mode to obtain the defocused capture image of the shooting scene, and transmits the defocused capture image to the display driver so that the electronic device 100 displays the thumbnail of the defocused capture image. Among them, the defocused capture instruction can be an instruction generated by the hardware abstraction layer based on the motion detection algorithm when the motion amplitude of the object to be photographed meets the capture condition, or an instruction generated based on the user operation of clicking the shooting control 122 in the case of enabling defocused capture.

[0119] Figure 4 Exemplarily shows the flowchart of the shooting method provided by the embodiments of the present application. Combining Figures 1A to 1F the user interface shown above and Figure 3The software architecture of the electronic device 100 shown below will specifically introduce the process of the shooting method provided in the embodiments of the present application.

[0120] 401. The electronic device 100 detects a user operation acting on the virtualization capture control on the user interface of the camera application. In response to this user operation, it starts the virtualization capture and calls the imaging device to collect the preview stream data of the shooting scene.

[0121] The user interface of the camera application in step 401 refers to the user interface of the camera application output by the electronic device 100 in response to the detected user operation acting on the camera application. For example Figure 1B or Figure 1E the user interface shown.

[0122] In one implementation, the user interface includes a virtualization capture control, such as Figure 1E the virtualization capture control 130 in. The electronic device 100 detects a user operation (such as a click operation) acting on the virtualization capture control in this user interface. In response to this user operation, it starts the virtualization capture and calls the imaging device (such as one or more cameras) to collect the preview stream data of the shooting scene. After starting the virtualization capture, the portrait in the virtualization capture control changes color and the portrait is in a running state.

[0123] In another implementation, the user interface includes a motion capture control and a portrait virtualization control, such as Figure 1B the motion capture control 125 and the portrait virtualization control 127 in. These two controls constitute the virtualization capture control. The electronic device 100 detects a user operation acting on the motion capture control in this user interface. In response to this user operation, it starts the motion capture and calls the imaging device to collect the preview stream data of the shooting scene. After starting the motion capture, the portrait in the motion capture control changes color and the portrait is in a running state. Then, it detects a user operation acting on the portrait virtualization control in this user interface. In response to this user operation, it starts the virtualization capture and continues to collect the preview stream data of the shooting scene. After starting the virtualization capture, the portrait in the portrait virtualization control changes color.

[0124] Starting the virtualization capture can be understood as starting the virtualization capture function or enabling the virtualization capture function. In this way, the electronic device 100 can perform capture and portrait virtualization based on the movement of the object to be photographed.

[0125] Among them, the shooting scene refers to the scene that the camera can currently capture the picture. The preview stream data of the shooting scene refers to the preview picture that the camera captures in real time before automatic capture or passive capture. The preview stream data may include multiple frames of preview images.

[0126] Optionally, when the electronic device 100 detects a user operation on the camera application, it outputs the user interface of the camera application and calls the imaging device to collect preview stream data of the shooting scene. When it detects a user operation on the defocus capture control, it starts defocus capture and continuously collects preview stream data of the shooting scene.

[0127] Optionally, when the electronic device 100 detects a user operation on the camera application, it outputs the user interface of the camera application and calls the imaging device to capture an image. When it detects a user operation on the defocus capture control, it starts defocus capture and calls the imaging device to collect preview stream data of the shooting scene. That is to say, after defocus capture is started, a series of images captured by the imaging device are referred to as preview stream data of the shooting scene.

[0128] 402. The electronic device 100 determines statistical information of the shooting scene based on the preview stream data.

[0129] The electronic device 100 performs statistical analysis on the collected preview stream data to determine statistical information of the shooting scene. The statistical information is used to determine whether to switch the defocus capture processing mode. The statistical information may include, but is not limited to, one or more of the following: scene brightness statistical value, dynamic range statistical value, flicker intensity detection statistical value, statistical value of medium depth of field in the subject contour layer, automatic exposure (AE) information, etc.

[0130] The scene brightness statistical value refers to the average value of the scene brightness values (luminance value) of multiple preview images included in the preview stream data. For a single preview image, before the sensor reads the image data, it can read the scene brightness value of the image.

[0131] The dynamic range statistical value refers to the average value of the dynamic range values of multiple preview images included in the preview stream data. For a single preview image, the ratio of the sum of the number of pixels in the largest bin and the smallest bin in its scene histogram to the number of pixels in the middle bin can represent the dynamic range value of the frame preview image. Taking the scene histogram with 128 bins as an example, the ratio between the sum of the number of pixels in bin1 and bin128 and the number of pixels in bin64 can be used as the dynamic range value. The larger the dynamic range value, the higher the scene dynamic range.

[0132] The flicker intensity detection statistical value refers to the average value of the flicker intensity detection values of multiple preview images included in the preview stream data. For a single preview image, the electronic device 100 can call a flicker detection algorithm to perform flicker intensity detection on it to obtain the flicker intensity detection value.

[0133] The statistical value of medium depth of field in the subject contour layer refers to the average value of the medium depth of field in the subject contour layer of multiple preview images included in the preview stream data. For a single preview image, the electronic device 100 performs an image morphological operation (dilation) on it to obtain a first dual-camera depth map, and performs an image morphological operation (erosion) on the preview image to obtain a second dual-camera depth map. A differential signal map is calculated based on the first dual-camera depth map and the second dual-camera depth map. On the differential signal map, the numerical values of the depth information are statistically calculated to obtain the medium depth of field.

[0134] The automatic exposure information is used to describe information related to automatic exposure, and may include, for example, one or more of whether automatic exposure is enabled, the aperture size of automatic exposure, the exposure time of automatic exposure, the gain of automatic exposure, etc.

[0135] The above several statistical information are for illustration purposes and do not constitute a limitation on the embodiments of the present application. For example, other information used to determine whether to switch to the defocus capture mode may also be included.

[0136] 403. In response to the statistical information satisfying the mode switching condition, the electronic device 100 switches the defocus capture processing mode from the first processing mode to the second processing mode.

[0137] Among them, the mode switching condition may include a series of thresholds, such as one or more of the following thresholds: scene brightness threshold, dynamic range threshold, flicker intensity threshold, depth of field threshold, etc. These thresholds can be understood as pre-calibrated thresholds, which can be set in the electronic device when it leaves the factory, or can be set based on the values input by the user. The specific values are not limited in the embodiments of the present application. These thresholds can also be understood as empirical thresholds, that is, some empirical thresholds summarized by R & D personnel.

[0138] Among them, the scene brightness threshold is used to compare with the scene brightness statistical value. If the scene brightness statistical value is less than the scene brightness value, it can be understood as a low-light scene. On the contrary, if the scene brightness statistical value is greater than or equal to the scene brightness value, it can be understood as a bright-light scene. The dynamic range threshold is used to compare with the dynamic range statistical value. If the dynamic range statistical value is less than the dynamic range threshold, it can be understood as a non-HDR scene. On the contrary, if the dynamic range statistical value is greater than or equal to the dynamic range threshold, it can be understood as an HDR scene. The flicker intensity threshold is used to compare with the flicker intensity detection statistical value. If the flicker intensity detection statistical value is less than the flicker intensity threshold, it can be understood as a non-Flicker scene. On the contrary, if the flicker intensity detection statistical value is greater than or equal to the flicker intensity threshold, it can be understood as a Flicker scene. The depth of field threshold is used to compare with the statistical value of the medium depth of field in the object contour layer. If the statistical value of the medium depth of field in the object contour layer is less than the depth of field threshold, it can be understood as a non-high-medium depth of field ratio scene. On the contrary, if the statistical value of the medium depth of field in the object contour layer is greater than or equal to the depth of field threshold, it can be understood as a high-medium depth of field ratio scene.

[0139] Optionally, the mode switching condition may further include one or more of an exposure time threshold, a gain threshold, etc.

[0140] The virtualization capture processing mode is used to implement virtualization capture, including a high dynamic range mode and a depth calculation mode. The high dynamic range mode refers to the HDR mode of the sensor, which can be divided into an interleaved high dynamic range mode and a dual-gain high dynamic range mode. The interleaved high dynamic range mode can be the above-mentioned stagger HDR mode, and the dual-gain high dynamic range mode can be the above-mentioned DXG mode. For the convenience of description, the interleaved high dynamic range mode is described by the stagger HDR mode below, and the dual-gain high dynamic range mode is described by the DXG mode. The depth calculation mode can be divided into a binocular depth calculation mode and a monocular depth calculation mode.

[0141] The high dynamic range mode in the second processing mode is different from the high dynamic range mode in the first processing mode, and / or the depth calculation mode in the second processing mode is different from the depth calculation mode in the first processing mode. So that the electronic device can adopt different high dynamic range modes and / or different depth calculation modes in different shooting scenes, which helps to save the power consumption of the electronic device. In the embodiment of the present application, the high dynamic range mode in the first processing mode is the stagger HDR mode, and the depth calculation mode in the first processing mode is the binocular depth calculation mode as an example. The virtualization capture processing mode is the first processing mode, which can make the effect of the virtualization capture image the best, but this mode is more power-consuming.

[0142] The statistical information is different, the satisfied mode switching conditions are different, and the second processing mode is different.

[0143] Mode 1, the statistical information includes the scene brightness statistical value and the dynamic range statistical value. The statistical information meeting the mode switching condition includes that the scene brightness statistical value is less than the scene brightness threshold and the dynamic range statistical value is less than the dynamic range threshold. Or, the statistical information meeting the mode switching condition includes that the ratio between the scene brightness statistical value and the dynamic range statistical value is less than a certain threshold; or, the statistical information meeting the mode switching condition includes that the product of the scene brightness statistical value and the dynamic range statistical value is less than a certain threshold; and so on. In Mode 1, the high dynamic range mode in the second processing mode is the DXG mode.

[0144] For Mode 1, when the electronic device 100 detects that the statistical information meets the mode switching condition, it can output a low-power mode enable signal for switching from the stagger HDR mode to the DXG mode. For example, the camera application framework layer of the electronic device 100 outputs this low-power mode enable signal to the hardware abstraction layer.

[0145] Mode 2, the statistical information includes the detected flicker intensity statistical value. The statistical information meeting the mode switching condition includes that the detected flicker intensity statistical value is less than the flicker intensity threshold. In this mode, the high dynamic range mode in the second processing mode is the DXG mode.

[0146] For Mode 2, when the electronic device 100 detects that the statistical information meets the mode switching condition, it can output a low-power mode enable signal for switching from the stagger HDR mode to the DXG mode. For example, the camera application framework layer of the electronic device 100 outputs this low-power mode enable signal to the hardware abstraction layer.

[0147] Mode 3, the statistical information includes the statistical value of the medium depth of field in the object contour layer. The statistical information meeting the mode switching condition includes that the statistical value of the medium depth of field in the object contour layer is less than the depth of field threshold. In Mode 3, the depth calculation mode in the second processing mode is the monocular depth calculation mode.

[0148] For Mode 3, when the electronic device 100 detects that the statistical information meets the mode switching condition, it can output a low-power mode enable signal for switching from the binocular depth calculation mode to the monocular depth calculation mode. For example, the camera application framework layer of the electronic device 100 outputs this low-power mode enable signal to the hardware abstraction layer.

[0149] Mode 4, the statistical information includes the scene brightness statistical value, the dynamic range statistical value, and the statistical value of the medium depth of field in the object contour layer. The statistical information meeting the mode switching condition includes that the scene brightness statistical value is less than the scene brightness threshold, the dynamic range statistical value is less than the dynamic range threshold, and the statistical value of the medium depth of field in the object contour layer is less than the depth of field threshold. In Mode 4, the high dynamic range mode in the second processing mode is the DXG mode, and the depth calculation mode is the monocular depth calculation mode.

[0150] In Mode 5, the statistical information includes the statistical value of the flicker intensity detection and the statistical value of the medium depth of field in the object contour layer. The condition for the statistical information to meet the mode switching condition is that the statistical value of the flicker intensity detection is less than the flicker intensity threshold, and the statistical value of the medium depth of field in the object contour layer is less than the depth of field threshold. In Mode 5, the high dynamic range mode in the second processing mode is the DXG mode, and the depth calculation mode is the monocular depth calculation mode.

[0151] For Modes 4 and 5, when the electronic device 100 detects that the statistical information meets the mode switching condition, it can output a low-power mode enable signal for switching from the stagger HDR mode to the DXG mode and from the binocular depth calculation mode to the monocular depth calculation mode. For example, the camera application framework layer of the electronic device 100 outputs this low-power mode enable signal to the hardware abstraction layer.

[0152] The above Modes 1 to 5 are for illustration and do not limit the embodiments of the present application. For example, there may be other combinations. For example, a combination of the flicker intensity detection statistical value and the AE information, both of which are less than the corresponding thresholds, and the depth calculation mode in the second processing mode is the monocular depth calculation mode.

[0153] For the above Modes 1 and 2, when the electronic device 100 switches the defocus processing mode from the first processing mode to the second processing mode, it switches the high dynamic range mode in the defocus capture processing mode from the stagger HDR mode to the DXG mode to save the power consumption of the electronic device. The high dynamic range mode in the second processing mode is the DXG mode. The depth calculation mode in the second processing mode can be the binocular depth calculation mode (i.e., the same as the depth calculation mode in the first processing mode) or the monocular depth calculation mode (i.e., switching from the binocular depth calculation mode to the monocular depth calculation mode). The switching from the binocular depth calculation mode to the monocular depth calculation mode can be triggered by the statistical value of the medium depth of field in the object contour layer. Switching the high dynamic range mode in the defocus capture processing mode from the stagger HDR mode to the DXG mode can be: controlling the working mode of the main path sensor to switch from the stagger HDR mode to the DXG mode.

[0154] Among them, for the working mode of the main path sensor being the stagger HDR mode, the main path sensor outputs images at 60fps, and its frame rate is twice that of the ordinary 30fps. When outputting frames, every two frames are paired, and these two frames are long exposure data and short exposure data respectively. These two frames of data are obtained by the main path sensor performing two exposures.

[0155] For the main road sensor operating in the DXG mode, the main road sensor outputs images at 30fps, and its frame rate is the same as the normal frame rate. The bit width of each frame of data is 2 bits more than the standard 10 bits. The data in this mode is obtained by the main road sensor through two Conversion Gain readings for the same exposure inside the main road sensor and then fused by the ISP inside the main road sensor, with a relatively high dynamic range.

[0156] For the above method 3, when the electronic device 100 switches the virtualization processing mode from the first processing mode to the second processing mode, it switches the depth calculation mode in the virtualization capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode to reduce the depth calculation power consumption, thereby saving the power consumption of the electronic device. When the depth calculation mode in the second processing mode is switched to the monocular depth calculation mode, the high dynamic range mode in the second processing mode can be the stagger HDR mode (i.e., the same as the high dynamic range mode in the first processing mode), or the DXG mode (i.e., switching from the interleaved high dynamic range mode to the dual-gain high dynamic range mode). The switch from the stagger HDR mode to the DXG mode can be triggered by the scene brightness statistical value and the dynamic range statistical value, or by the flicker intensity detection statistical value, or by the scene brightness statistical value, the dynamic range statistical value, and the flicker intensity detection statistical value. Switching the depth calculation mode in the virtualization capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: controlling the depth calculation path of the virtualization capture processing mode to switch from the binocular depth calculation path to the monocular depth calculation path. Switching from the binocular depth calculation path to the monocular depth calculation path can save the power consumption of the electronic device.

[0157] Among them, the binocular depth calculation path is used to execute the binocular depth calculation mode. The output of the multi-exposure fusion algorithm or the data frame obtained by multi-frame denoising of the DXG data is used for binocular depth calculation with the auxiliary road sensor data and then output to the subsequent portrait virtualization algorithm. The monocular depth calculation path is used to execute the monocular depth calculation mode. The DXG data after multi-frame denoising is sent for monocular depth calculation and then output to the subsequent portrait virtualization algorithm.

[0158] For the above-mentioned method 4 and method 5, when the electronic device 100 switches the blur processing mode from the first processing mode to the second processing mode, the high dynamic range mode in the blur capture processing mode is switched from the stagger HDR mode to the DXG mode, and the depth calculation mode in the blur capture processing mode is switched from the binocular depth calculation mode to the monocular depth calculation mode. Switching the high dynamic range mode in the blur capture processing mode from the stagger HDR mode to the DXG mode includes: controlling the working mode of the main sensor to switch from the stagger HDR mode to the DXG mode, and controlling the working mode of the auxiliary sensor to switch from the working outflow mode to the non-outflow waiting mode. Switching the depth calculation mode in the blur capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: controlling the depth calculation path of the blur capture processing mode to switch from the binocular depth calculation path to the monocular calculation path.

[0159] When the auxiliary sensor is in the working streaming mode, the auxiliary sensor outputs images at 30fps. When the auxiliary sensor is in the non-streaming waiting mode, the auxiliary sensor does not output image data, but can quickly recover to the state of outputting data, which can be called the stand by state.

[0160] Before switching the depth calculation mode in the blur capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode, the exposure fusion path of the blur capture processing mode is controlled to switch from the multi-exposure fusion path to the single exposure fusion path to further save the power consumption of the electronic device.

[0161] Among them, the multi-exposure fusion path is as follows: the long-exposure data of the main sensor is subjected to frame selection and multi-frame fusion algorithm to obtain multi-frame noise-reduced data, and this data is fused with the short-exposure frame of the main sensor using a multi-exposure fusion algorithm to obtain fused frame data, and then combined with the information of depth calculation (monocular or binocular) to perform a portrait blur algorithm to obtain a blurred captured image.

[0162] The single exposure path is as follows: the DXG frame data of the main sensor is processed through frame selection and high-bit-width multi-frame fusion algorithm to obtain multi-frame noise-reduced data, and then combined with the information of depth calculation (monocular or binocular) to perform a portrait blur algorithm to obtain a blurred snapshot image.

[0163] 404 , in response to a blur capture instruction for a shooting scene, the electronic device 100 calls a camera to capture an image to be processed of the shooting scene.

[0164] Among them, the blurring capture instruction for the shooting scene can be an instruction generated by the electronic device 100 based on a motion detection algorithm when it detects that the motion amplitude of the object to be photographed meets the capture condition, or it can be an instruction generated based on a user operation of clicking the shooting control 122 when blurring capture is enabled.

[0165] When the electronic device 100 detects a blurring capture instruction for the shooting scene, it calls the imaging device to capture a to-be-processed image of the shooting scene. Optionally, the electronic device 100 calls one camera to capture the to-be-processed image. Optionally, the electronic device 100 calls two cameras to capture the to-be-processed image.

[0166] 405. The electronic device 100 processes the to-be-processed image based on the second processing mode to obtain a blurring capture image of the shooting scene.

[0167] The electronic device 100 processes the to-be-processed image based on the second processing mode to obtain a blurring capture image of the shooting scene.

[0168] The process of the electronic device 100 obtaining the blurring capture image using the first processing mode can be seen in Figure 5A as shown. Figure 5A In it, based on the preview stream data, the blurring capture mode is determined to be the first processing mode. In response to the blurring capture instruction, both the main road sensor and the auxiliary road sensor output images. The main road sensor uses the stagger HDR mode to output two frames of data, the long exposure data and the short exposure data. The long exposure data is processed by the frame selection and multi-frame fusion algorithm and then undergoes multi-exposure fusion processing with the short exposure data, and then binocular depth calculation and portrait blurring processing are performed to obtain the blurring capture image.

[0169] The process of the electronic device 100 obtaining the blurring capture image using the second processing mode can be seen in Figures 5B to 5D as shown. Figure 5B In it, based on the preview stream data, the blurring capture mode is determined to be the second processing mode (DXG mode). In response to the blurring capture instruction, both the main road sensor and the auxiliary road sensor output images. The main road sensor uses the DXG mode to output one frame of data. This frame of data is processed by the frame selection and multi-frame fusion algorithm and then undergoes binocular depth calculation with the auxiliary road sensor data, and then portrait blurring processing is performed to obtain the blurring capture image. Figure 5C In it, based on the preview stream data, the blurring capture mode is determined to be the second processing mode (monocular depth calculation mode). In response to the blurring capture instruction, the main road sensor outputs an image. The main road sensor uses the stagger HDR mode to output two frames of data, the long exposure data and the short exposure data. The long exposure data is processed by the frame selection and multi-frame fusion algorithm and then undergoes multi-exposure fusion processing with the short exposure data, and then monocular depth calculation and portrait blurring processing are performed to obtain the blurring capture image. Figure 5DAmong them, based on the preview stream data, the defocus capture mode is determined to be the second processing mode (DXG mode and monocular depth calculation). In response to the defocus capture instruction, the main road sensor outputs an image. The main road sensor adopts the DXG mode to output one frame of data. After this frame of data is processed by the frame selection and multi-frame fusion algorithm, monocular depth calculation is performed, and then portrait defocus processing is carried out to obtain the defocus capture image.

[0170] Compare Figure 5A and Figures 5B - 5D It can be seen that Figures 5B - 5D The process is less, which is beneficial to saving the power consumption of the electronic device.

[0171] 406. The electronic device 100 displays a thumbnail of the defocus capture image of the shooting scene in the thumbnail display area in the user interface of the camera application.

[0172] After the electronic device 100 obtains the defocus capture image, in the thumbnail display area in the user interface of the camera application, such as Figure 1D and Figure 1F the thumbnail display area 123 in, the thumbnail of the defocus capture image is displayed. Optionally, the electronic device 100 can save the defocus capture image in the gallery.

[0173] In Figure 4 the embodiment shown, in the defocus capture scene, when the statistical information of the shooting scene meets the mode switching condition, the defocus capture processing mode can be switched from the first processing mode to the second processing mode, and then the image to be processed captured can be processed based on the second processing mode, so that the power consumption of the electronic device can be saved in the defocus capture scene.

[0174] Figure 4The process of switching from the first processing mode to the second processing mode is introduced in the illustrated embodiment. The statistical information of the shooting scene may change over time. When the shooting scene changes to meet the requirements of the first processing mode, the defocus capture processing mode can be switched from the second processing mode to the first processing mode. For example, for the above-mentioned method 1, when the scene brightness statistical value is greater than or equal to the scene brightness threshold, and / or the dynamic range statistical value is greater than or equal to the dynamic range threshold, the high dynamic range mode can be switched from the DXG mode to the stagger HDR mode. For another example, for the above-mentioned method 2, when the flicker intensity detection statistical value is greater than or equal to the flicker intensity threshold, the high dynamic range mode can be switched from the DXG mode to the stagger HDR mode. For yet another example, for the above-mentioned method 3, when the statistical value of the medium depth of field in the object contour layer is greater than or equal to the depth of field threshold, the depth calculation mode can be switched from the monocular depth calculation mode to the binocular depth calculation mode. For yet another example, for the above-mentioned method 4, when the scene brightness statistical value is less than the scene brightness threshold, and the dynamic range statistical value is less than the dynamic range threshold, but the statistical value of the medium depth of field in the object contour layer is greater than or equal to the depth of field threshold, then the DXG mode can continue to be used, and the depth calculation mode can be switched from the monocular depth calculation mode to the binocular depth calculation mode; or, when the statistical value of the medium depth of field in the object contour layer is less than the depth of field threshold, but the scene brightness statistical value is greater than or equal to the scene brightness threshold, and / or the dynamic range statistical value is greater than or equal to the dynamic range threshold, then the monocular depth calculation mode can continue to be used, and the high dynamic range mode can be switched from the DXG mode to the stagger HDR mode; or, when the scene brightness statistical value is greater than or equal to the scene brightness threshold, and / or the dynamic range statistical value is greater than or equal to the dynamic range threshold, and the statistical value of the medium depth of field in the object contour layer is greater than or equal to the depth of field threshold, then the high dynamic range mode can be switched from the DXG mode to the stagger HDR mode, and the depth calculation mode can be switched from the monocular depth calculation mode to the binocular depth calculation mode.

[0175] Figure 6 An exemplary schematic diagram of the hardware structure of the electronic device 100 is shown.

[0176] The electronic device 100 may include a processor 110, an external memory interface 12B, an internal memory 12A, a universal serial bus (USB) interface 13A, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio processing module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0177] It can be understood that the structure schematically shown in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0178] The processor 110 may include one or more processing units. For example, the processor 110 may include an AP, a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. In the embodiments of the present application, the processor 110 may include a virtual capture algorithm.

[0179] The controller may generate operation control signals according to the instruction operation code and the timing signal to complete the control of fetching instructions and executing instructions.

[0180] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly retrieved from the memory. This avoids repeated accesses and reduces the waiting time of the processor 110, thus improving the efficiency of the system.

[0181] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0182] It can be understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are only illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0183] The charging management module 140 is configured to receive a charging input from a charger. The charger may be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 may receive the charging input from the wired charger through the USB interface 13A. In some embodiments of wireless charging, the charging management module 140 may receive the wireless charging input through the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 may also supply power to the electronic device through the power management module 141.

[0184] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives the inputs from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 12A, the display screen 194, the camera 193, the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as the battery capacity, the number of battery cycles, and the battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be disposed in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can also be disposed in the same device.

[0185] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0186] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example: the antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0187] The mobile communication module 150 can provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves through the antenna 1 and radiate it out. In some embodiments, at least some functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be disposed in the same device.

[0188] The modulation and demodulation processor can include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor.

[0189] The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be disposed in the same device as the mobile communication module 150 or other functional modules.

[0190] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive signals to be sent from the processor 110, frequency-modulate them, amplify them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0191] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, such that electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0192] Electronic device 100 implements a display function through a GPU, display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, and is connected to display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or change display information. In the embodiments of the present application, display screen 194 is used to display a defocused capture image.

[0193] Internal memory 12A may include one or more Random Access Memories (RAM) and one or more Non-Volatile Memories (NVM).

[0194] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation of DDR SDRAM is generally referred to as DDR5 SDRAM), etc. Non-volatile memory may include disk storage devices, flash memory.

[0195] The random access memory can be directly read and written by the processor 110, and can be used to store the operating system or executable programs (such as machine instructions) of other running programs, and can also be used to store data of users and application programs, etc.

[0196] The non-volatile memory can also store executable programs and data of users and application programs, etc., and can be pre-loaded into the random access memory for direct reading and writing by the processor 110.

[0197] The external memory interface 12B can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 12B to implement the data storage function. For example, files such as music and videos are saved in the external non-volatile memory.

[0198] The electronic device 100 can implement audio functions through the audio processing module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor, etc. For example, music playback, recording, etc.

[0199] The audio processing module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal. The audio processing module 170 can also be used to encode and decode audio signals. In some embodiments, the audio processing module 170 can be disposed in the processor 110, or some functional modules of the audio processing module 170 can be disposed in the processor 110.

[0200] The speaker 170A, also known as the "loudspeaker". The electronic device 100 can listen to music or hands-free calls through the speaker 170A. The receiver 170B, also known as the "earpiece". When the electronic device 100 answers a call or a voice message, the voice can be received by placing the receiver 170B close to the human ear. The microphone 170C, also known as the "microphone", "transmitter". When making a call or sending a voice message, the user can speak by bringing the mouth close to the microphone 170C to input the sound signal into the microphone 170C.

[0201] The headphone jack 170D is used to connect a wired headphone. The headphone jack 170D can be a USB interface 13A, or a 3.5 mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0202] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. The barometric pressure sensor 180C is used to measure the barometric pressure. The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). The distance sensor 180F is used to measure the distance. The proximity light sensor 180G can include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The ambient light sensor 180L is used to sense the ambient light brightness. The fingerprint sensor 180H is used to collect fingerprints. The temperature sensor 180J is used to detect the temperature. The touch sensor 180K, also known as the "touch device". The touch sensor 180K can be disposed on the display screen 194. The touch screen, also known as the "touch screen", is composed of the touch sensor 180K and the display screen 194. The touch sensor 180K is used to detect the touch operation acting thereon or nearby. The bone conduction sensor 180M can obtain the vibration signal.

[0203] The keys 190 include a power-on key, volume keys, etc. The keys 190 can be mechanical keys. Or they can be touch keys. The electronic device 100 can receive key inputs to generate key signal inputs related to the user settings and function controls of the electronic device 100.

[0204] The motor 191 can generate vibration prompts. The motor 191 can be used for vibration prompts for incoming calls and can also be used for touch vibration feedback. The indicator 192 can be an indicator light and can be used to indicate the charging status, battery level change, and can also be used to indicate messages, missed calls, notifications, etc.

[0205] The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation from the electronic device 100. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0206] In the description, claims, and drawings of this application, the term "user interface (UI)" is a media interface for interaction and information exchange between an application or an operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface of an application is source code written in specific computer languages such as Java and Extensible Markup Language (XML). The interface source code is parsed and rendered on the terminal device and finally presented as content recognizable by the user, such as controls like pictures, text, buttons, etc. A control (also known as a widget) is a basic element of the user interface. Typical controls include a toolbar, a menu bar, a text box, a button, a scrollbar, pictures, and text. The attributes and content of the controls in the interface are defined through tags or nodes. For example, XML passes through <textview> 、 <imgview> 、 <videoview>Nodes such as these are used to define the controls included in the interface. One node corresponds to one control or property in the interface, and after being parsed and rendered, the node presents as visible content to the user. In addition, in the interfaces of many applications, such as hybrid applications, there are usually also web pages. A web page, also known as a page, can be understood as a special control embedded in the application interface. A web page is source code written in a specific computer language, such as hyper text markup language (HTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed as recognizable content to the user by a browser or a web page display component similar to the browser function. The specific content included in the web page is also defined by tags or nodes in the web page source code. For example, HTML uses 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0207] A commonly used form of the user interface is the graphical user interface (GUI), which refers to the user interface related to computer operations presented in a graphical manner. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include visible interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, Widgets, etc.

[0208] As used in the description and appended claims of this application, the singular forms "a", "an", "the", "above-mentioned", "said", and "this" are intended to include the plural forms as well, unless clearly indicated otherwise in the context. It should also be understood that the term "and / or" used in this application refers to and includes any and all possible combinations of one or more of the listed items. As used in the above embodiments, depending on the context, the term "when..." can be interpreted to mean "if...", or "after...", or "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if detecting (the stated condition or event)" can be interpreted to mean "if determining...", or "in response to determining...", or "when detecting (the stated condition or event)", or "in response to detecting (the stated condition or event)".

[0209] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, fiber optic, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive), etc.

[0210] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by relevant hardware instructed by a computer program. This program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The aforementioned storage medium includes: various media such as ROM or random access memory RAM, magnetic disks, or optical discs that can store program codes.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A shooting method, characterized in that, Including: When a user operation on a defocus capture control is detected in the user interface of the camera application, in response to the user operation, defocus capture is started and the camera device is called to collect preview stream data of the shooting scene; Based on the preview stream data, determine the statistical information of the shooting scene; In response to the statistical information satisfying the mode switching condition, switch the defocus capture processing mode from the first processing mode to the second processing mode; In response to a defocus capture instruction for the shooting scene, call the camera device to capture a to-be-processed image of the shooting scene; Based on the second processing mode, process the to-be-processed image to obtain a defocus capture image of the shooting scene; In the thumbnail display area in the user interface of the camera application, display a thumbnail of the defocus capture image of the shooting scene.

2. The method according to claim 1, characterized in that, The defocus capture processing mode includes a high dynamic range mode and a depth calculation mode. The high dynamic range mode in the second processing mode is different from the high dynamic range mode in the first processing mode, and / or the depth calculation mode in the second processing mode is different from the depth calculation mode in the first processing mode.

3. The method according to claim 1, wherein The high dynamic range mode in the first processing mode is an interleaved high dynamic range mode, and the depth calculation mode in the first processing mode is a binocular depth calculation mode.

4. The method according to any one of claims 1 to 3, characterized in that, The statistical information includes a scene brightness statistical value and a dynamic range statistical value. The statistical information satisfying the mode switching condition includes that the scene brightness statistical value is less than a scene brightness threshold, and the dynamic range statistical value is less than a dynamic range threshold; The switching of the defocus capture processing mode from the first processing mode to the second processing mode includes: Switch the high dynamic range mode in the defocus capture processing mode from the interleaved high dynamic range mode to the dual-gain high dynamic range mode.

5. The method according to any one of claims 1 to 3, characterized in that The statistical information includes a flicker intensity detection statistical value. The statistical information satisfying the mode switching condition includes that the flicker intensity detection statistical value is less than a flicker intensity threshold; The switching of the defocus capture processing mode from the first processing mode to the second processing mode includes: Switch the high dynamic range mode in the defocus capture processing mode from the interleaved high dynamic range mode to the dual-gain high dynamic range mode.

6. The method according to claim 4 or 5, characterized in that The method is applied to an electronic device, and the electronic device includes a main path sensor; The switching of the high dynamic range mode in the defocus capture processing mode from the interleaved high dynamic range mode to the dual-gain high dynamic range mode includes: Control the working mode of the main path sensor to switch from the interleaved high dynamic range mode to the dual-gain high dynamic range mode.

7. The method according to any one of claims 1 to 3, characterized in that, The statistical information includes a statistical value of medium depth of field in the captured object contour layer. The statistical information satisfying the mode switching condition includes that the statistical value of medium depth of field in the captured object contour layer is less than a depth of field threshold; The switching of the defocus capture processing mode from the first processing mode to the second processing mode includes: Switch the depth calculation mode in the defocus capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode.

8. The method according to claim 7, wherein The method is applied to an electronic device, and the electronic device includes a binocular depth calculation path and a monocular depth calculation path; The switching of the depth calculation mode in the virtualized capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: Controlling the depth calculation path of the virtualized capture processing mode to switch from the binocular depth calculation path to the monocular depth calculation path.

9. The method according to any one of claims 1 to 3, characterized in that, The statistical information includes the statistical value of the flicker intensity detection and the statistical value of the medium depth of field in the captured object contour layer. The satisfaction of the mode switching condition by the statistical information includes that the statistical value of the flicker intensity detection is less than the flicker intensity threshold, and the statistical value of the medium depth of field in the captured object contour layer is less than the depth of field threshold; The switching of the virtualized capture processing mode from the first processing mode to the second processing mode includes: Switching the high dynamic range mode in the virtualized capture processing mode from the interleaved high dynamic range mode to the dual gain high dynamic range mode, and switching the depth calculation mode in the virtualized capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode.

10. The method according to claim 9, characterized in that, The method is applied to an electronic device, and the electronic device includes a main path sensor, an auxiliary path sensor, a binocular depth calculation path, and a monocular calculation path; The switching of the high dynamic range mode in the virtualized capture processing mode from the interleaved high dynamic range mode to the dual gain high dynamic range mode includes: Controlling the working mode of the main path sensor to switch from the interleaved high dynamic range mode to the dual gain high dynamic range mode, and controlling the working mode of the auxiliary path sensor to switch from the working outflow mode to the non-outflow waiting mode; The switching of the depth calculation mode in the virtualized capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: Controlling the depth calculation path of the virtualized capture processing mode to switch from the binocular depth calculation path to the monocular calculation path.

11. The method according to claim 10, wherein The electronic device further includes a multi-exposure fusion path and a single-exposure fusion path; Before the switching of the depth calculation mode in the virtualized capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode, it further includes: Controlling the exposure fusion path of the virtualized capture processing mode to switch from the multi-exposure fusion path to the single-exposure fusion path.

12. An electronic device, characterized in that, Comprising one or more processors and one or more memories; wherein, the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the method according to any one of claims 1-11 is performed.

13. A chip system, characterized in that, The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processors are used to call computer instructions to cause the electronic device to execute the method according to any one of claims 1-11.

14. A computer-readable storage medium, comprising instructions, characterized in that, When the instructions run on the electronic device, the method according to any one of claims 1-11 is performed.

Images