A photographing method, an electronic device, and a storage medium

Abstract

Embodiments of the present application provide a photographing method, an electronic device and a storage medium, which can be applied to an electronic device supporting virtualization snapshot, such as a smart phone, a tablet computer and the like. The virtualization snapshot is also called depth calculation and portrait virtualization processing on a snapshot image, and optionally, dynamic range fusion processing. The method can comprise: in response to a detected user operation on a virtualization snapshot control, starting virtualization snapshot and calling a camera to collect preview stream data of a photographing scene; determining statistical information of the photographing scene based on the preview stream data; in response to the statistical information satisfying a mode switching condition, switching a virtualization snapshot processing mode from a first processing mode to a second processing mode; and in response to a virtualization snapshot instruction of the photographing scene, processing a to-be-processed image based on the second processing mode to obtain a virtualization snapshot image of the photographing scene. By using the embodiments of the present application, the power consumption of the electronic device can be saved in the virtualization snapshot scene.

Description

Technical Field

[0001] This application relates to the field of electronic devices, and more particularly to a shooting method, electronic device, and storage medium. Background Technology

[0002] With the development of photography technology, electronic devices with shooting capabilities can provide a variety of shooting methods, such as portrait, night scene, selfie, video recording, and candid photography. Among these, candid photography refers to the rapid capture of a natural, vivid, and expressive moment by an electronic device without the subject's knowledge. Examples include capturing a brilliant moment of someone, a traffic violation, or a spectacular goal in a sports game.

[0003] Compared to other shooting methods, snapshot photography increases the power consumption of electronic devices to some extent. Therefore, how to save power consumption of electronic devices for snapshot photography is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This application provides a shooting method, electronic device, and storage medium that can save power consumption of electronic devices in bokeh snapshot scenarios.

[0005] Firstly, embodiments of this application provide a shooting method applicable to electronic devices that support bokeh snapshots. Bokeh snapshots involve performing depth calculations and portrait blurring on the captured image, and optionally, dynamic range fusion processing. Compared to ordinary snapshots, bokeh snapshots improve the clarity of the subject (e.g., a portrait). The method may include: detecting a user operation on a bokeh snapshot control in the user interface of a camera application; responding to the user operation, initiating bokeh snapshots and calling a camera device to acquire preview stream data of the shooting scene; determining statistical information of the shooting scene based on the preview stream data; switching the bokeh snapshot processing mode from a first processing mode to a second processing mode in response to the statistical information satisfying the mode switching condition; calling a camera device to capture an image of the shooting scene to be processed in response to a bokeh snapshot command for the shooting scene; processing the image to be processed based on the second processing mode to obtain a bokeh snapshot image of the shooting scene; and displaying a thumbnail of the bokeh snapshot image of the shooting scene in the thumbnail display area of ​​the user interface of the camera application.

[0006] By implementing the shooting method provided in the first aspect, in a bokeh capture scenario, when the statistical information of the shooting scenario meets the mode switching conditions, the bokeh capture processing mode can be switched from the first processing mode to the second processing mode. Then, the captured image to be processed can be processed based on the second processing mode, thereby saving the power consumption of electronic devices in a bokeh capture scenario.

[0007] In conjunction with the method provided in the first aspect, in some embodiments, the aforementioned bokeh capture processing mode includes a high dynamic range (HDR) mode and a depth calculation mode. The HDR mode in the second processing mode is different from the HDR mode in the first processing mode, and / or, the depth calculation mode in the second processing mode is different from the depth calculation mode in the first processing mode. This allows the electronic device to employ different HDR modes and / or different depth calculation modes in different shooting scenarios, thereby helping to save power consumption of the electronic device.

[0008] High dynamic range (HMR) modes refer to the sensor's high dynamic range modes, which can be divided into interleaved HMR modes and dual-gain HMR modes. Depth calculation modes can be divided into binocular depth calculation modes and monocular depth calculation modes.

[0009] In conjunction with the method provided in the first aspect, in some embodiments, the high dynamic range mode in the first processing mode is an interleaved high dynamic range mode, and the depth calculation mode in the first processing mode is a binocular depth calculation mode. The power consumption in the interleaved high dynamic range mode is higher than that in the dual-gain high dynamic range mode, and the power consumption in the binocular depth calculation mode is higher than that in the monocular depth calculation mode. In other words, the electronic device consumes more power using the first processing mode, which accelerates battery consumption. Switching from the first processing mode to the second processing mode can slow down battery consumption, thereby saving power for the electronic device.

[0010] In conjunction with the method provided in the first aspect, in some embodiments, the aforementioned statistical information includes scene brightness statistics and dynamic range statistics. The statistical information satisfies the mode switching condition if the scene brightness statistics are less than a scene brightness threshold and the dynamic range statistics are less than a dynamic range threshold. In this case, the high dynamic range mode in the bokeh capture processing mode is switched from interleaved high dynamic range mode to dual-gain high dynamic range mode to save power consumption of the electronic device.

[0011] It is understood that the high dynamic range mode in the second processing mode is a dual-gain high dynamic range mode, and the depth calculation mode in the second processing mode can be a stereo depth calculation mode (i.e., the same as the depth calculation mode in the first processing mode) or a monocular depth calculation mode (i.e., switching from stereo depth calculation mode to monocular depth calculation mode). The switch from stereo depth calculation mode to monocular depth calculation mode can be triggered by the statistical value of the equal depth of field in the subject outline layer.

[0012] The scene brightness threshold and dynamic range threshold can be understood as pre-calibrated thresholds, which can be set in the electronic device at the factory or based on user-input values. The specific values ​​are not limited in this application embodiment.

[0013] In conjunction with the method provided in the first aspect, in some embodiments, the aforementioned statistical information includes flicker intensity detection statistics, and the aforementioned statistical information satisfies the mode switching condition by the flicker intensity detection statistics being less than a flicker intensity threshold. In this case, the high dynamic range mode in the bokeh capture processing mode is switched from interleaved high dynamic range mode to dual-gain high dynamic range mode to save power consumption of the electronic device.

[0014] It is understood that the high dynamic range mode in the second processing mode is a dual-gain high dynamic range mode, and the depth calculation mode in the second processing mode can be a stereo depth calculation mode (i.e., the same as the depth calculation mode in the first processing mode) or a monocular depth calculation mode (i.e., switching from stereo depth calculation mode to monocular depth calculation mode). The switch from stereo depth calculation mode to monocular depth calculation mode can be triggered by the statistical value of the equal depth of field in the subject outline layer.

[0015] The flicker intensity threshold can be understood as a pre-calibrated threshold, which can be set in the electronic device at the factory or based on a value input by the user. The specific value is not limited in this embodiment.

[0016] In conjunction with the method provided in the first aspect, in some embodiments, the electronic device includes a main sensor. Switching the high dynamic range mode in the bokeh capture processing mode from an interleaved high dynamic range mode to a dual-gain high dynamic range mode includes: controlling the operating mode of the main sensor to switch from the interleaved high dynamic range mode to the dual-gain high dynamic range mode. Switching the operating mode of the main sensor to the dual-gain high dynamic range mode saves power consumption of the main sensor, thereby saving power consumption of the electronic device.

[0017] Optionally, the electronic device also includes an auxiliary sensor. If the depth calculation mode in the second processing mode is a monocular depth calculation mode, the operating mode of the auxiliary sensor can be controlled to switch from an outflow operating mode to a non-outflow waiting mode to save power consumption of the auxiliary sensor, thereby saving power consumption of the electronic device. If the depth calculation mode in the second processing mode is a binocular depth calculation mode, the operating mode of the auxiliary sensor can be controlled to an outflow operating mode.

[0018] In conjunction with the method provided in the first aspect, in some embodiments, the aforementioned statistical information includes statistical values ​​of the depth of field in the outline layer of the subject, and the statistical information satisfies the mode switching condition when the statistical value of the depth of field in the outline layer of the subject is less than a depth of field threshold. In this case, the depth calculation mode in the bokeh capture processing mode is switched from the binocular depth calculation mode to the monocular depth calculation mode to reduce depth calculation power consumption, thereby saving power consumption of the electronic device.

[0019] Understandably, the depth calculation mode in the second processing mode switches to the monocular depth calculation mode. The high dynamic range mode in the second processing mode can be an interleaved high dynamic range mode (i.e., the same as the high dynamic range mode in the first processing mode) or a dual-gain high dynamic range mode (i.e., switching from the interleaved high dynamic range mode to the dual-gain high dynamic range mode). The switch from the interleaved high dynamic range mode to the dual-gain high dynamic range mode can be triggered by scene brightness statistics and dynamic range statistics, or by flicker intensity detection statistics, or by scene brightness statistics, dynamic range statistics, and flicker intensity detection statistics.

[0020] The depth-of-field threshold can be understood as a pre-calibrated threshold, which can be set in the electronic device at the factory or based on a value input by the user. The specific value is not limited in this application embodiment.

[0021] In conjunction with the method provided in the first aspect, in some embodiments, the electronic device includes a binocular depth calculation path and a monocular depth calculation path; switching the depth calculation mode in the bokeh capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: controlling the depth calculation path of the bokeh capture processing mode to switch from the binocular depth calculation path to the monocular depth calculation path. Switching from the binocular depth calculation path to the monocular depth calculation path can save power consumption of the electronic device.

[0022] Among them, the binocular depth calculation path is used to execute the binocular depth calculation mode, and the monocular depth calculation path is used to execute the monocular depth calculation mode.

[0023] In conjunction with the method provided in the first aspect, in some embodiments, the aforementioned statistical information includes flicker intensity detection statistics and depth-of-field statistics in the subject outline layer. The statistical information satisfies the mode switching condition if the flicker intensity detection statistics are less than a flicker intensity threshold and the depth-of-field statistics in the subject outline layer are less than a depth-of-field threshold. In this case, the high dynamic range mode in the bokeh capture processing mode is switched from interlaced high dynamic range mode to dual-gain high dynamic range mode, and the depth calculation mode in the bokeh capture processing mode is switched from binocular depth calculation mode to monocular depth calculation mode. The main sensor's operating mode is switched to dual-gain high dynamic range mode to save power consumption, and the depth calculation mode is switched from binocular depth calculation mode to monocular depth calculation mode to reduce depth calculation power consumption, thereby saving power consumption of the electronic device.

[0024] In conjunction with the method provided in the first aspect, in some embodiments, the aforementioned statistical information includes scene brightness statistics, dynamic range statistics, and depth-of-field statistics in the subject outline layer. The aforementioned statistical information satisfies the mode switching conditions as follows: the scene brightness statistics are less than a scene brightness threshold, the dynamic range statistics are less than a dynamic range threshold, and the depth-of-field statistics in the subject outline layer are less than a depth-of-field threshold. In this case, the high dynamic range mode in the bokeh capture processing mode is switched from interlaced high dynamic range mode to dual-gain high dynamic range mode, and the depth calculation mode in the bokeh capture processing mode is switched from binocular depth calculation mode to monocular depth calculation mode. The main sensor's operating mode is switched to dual-gain high dynamic range mode to save power consumption, and the depth calculation mode is switched from binocular depth calculation mode to monocular depth calculation mode to reduce depth calculation power consumption, thereby saving power consumption of the electronic device.

[0025] In conjunction with the method provided in the first aspect, in some embodiments, the electronic device includes a main path sensor, an auxiliary path sensor, a binocular depth calculation path, and a monocular calculation path. Switching the high dynamic range mode in the bokeh capture processing mode from an interleaved high dynamic range mode to a dual-gain high dynamic range mode includes: controlling the operating mode of the main path sensor to switch from the interleaved high dynamic range mode to the dual-gain high dynamic range mode, and controlling the operating mode of the auxiliary path sensor to switch from an outflow mode to a non-outflow waiting mode. Switching the depth calculation mode in the bokeh capture processing mode from a binocular depth calculation mode to a monocular depth calculation mode includes: controlling the depth calculation path of the bokeh capture processing mode to switch from a binocular depth calculation path to a monocular calculation path. This saves power consumption of the electronic device by reducing the power consumption of the main path sensor and the auxiliary path sensor, and by reducing the power consumption of depth calculation.

[0026] In conjunction with the method provided in the first aspect, in some embodiments, the electronic device further includes a multi-exposure fusion path and a single-exposure fusion path; before switching the depth calculation mode in the bokeh capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode, the exposure fusion path of the bokeh capture processing mode is controlled to switch from the multi-exposure fusion path to the single-exposure fusion path, so as to further save the power consumption of the electronic device.

[0027] In a second aspect, embodiments of this application provide an electronic device including one or more processors and one or more memories; wherein the one or more memories are coupled to one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when the one or more processors execute the computer instructions, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0028] Thirdly, embodiments of this application provide a chip system applied to an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0029] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0030] Fifthly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0031] Understandably, the electronic device provided in the second aspect, the chip system provided in the third aspect, the computer storage medium provided in the fourth aspect, and the computer program product provided in the fifth aspect are all used to execute the method provided in the first aspect of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0032] Figures 1A-1F This is a schematic diagram of a set of user interfaces provided in an embodiment of this application;

[0033] Figures 2A-2F This is a schematic diagram of a set of shooting scenarios provided in an embodiment of this application;

[0034] Figure 3 This is a software architecture diagram of the electronic device provided in the embodiments of this application;

[0035] Figure 4 This is a flowchart illustrating a shooting method provided in an embodiment of this application;

[0036] Figures 5A-5D This is a flowchart illustrating several processing modes provided in the embodiments of this application;

[0037] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0038] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be a limitation of this application.

[0039] 1. Blur-out snapshot

[0040] Regular snapshot refers to the rapid capture of a natural, vivid, and expressive moment by an electronic device with shooting capabilities, without the subject's (e.g., a person or animal) awareness. Blur-out snapshot refers to the rapid capture of a momentary image of a subject by an electronic device with shooting capabilities, without the subject's awareness, followed by depth calculations and portrait blurring processing to obtain a blurred image. Optionally, dynamic range fusion processing can be performed before or after portrait blurring to obtain a high dynamic range blurred image. Portrait blurring refers to highlighting the details of the person in the image while blurring the background to improve the sharpness of the person.

[0041] In other words, bokeh effect capture builds upon regular snapshot capture by adding image processing steps such as depth calculation, portrait blurring, and dynamic range fusion to improve the display effect of a fleeting image. For example, it increases the sharpness of the subject in the instantaneous image and enhances its dynamic range. Understandably, for the same subject captured at the same time in the same scene, a bokeh effect capture will result in an image with higher sharpness, a wider dynamic range, and a better overall image display.

[0042] Because bokeh effect involves an additional series of image processing steps compared to regular snapshots, it consumes power much faster for electronic devices. In other words, bokeh effect consumes more power than regular snapshots.

[0043] 2. High Dynamic Range (HDR) mode of the sensor

[0044] Dynamic Range (DR), in the field of image processing, refers to the range of light and shadow in an image, from the darkest to the brightest. A higher dynamic range results in richer image detail. High Dynamic Range (HDR), compared to standard dynamic range, expands the brightness range of an image, allowing for wider highlights and shadows, resulting in richer image detail. In other words, HDR can bring out both bright and dark areas in an image.

[0045] The sensor refers to an image sensor. Sensors vary in accuracy and dynamic range. For ease of description, this application will simply refer to the image sensor as a sensor.

[0046] The HDR mode of a sensor refers to the processing mode used by the sensor when processing HDR frames. The HDR modes of a sensor can be mainly divided into the following types:

[0047] (1) Stagger HDR mode

[0048] Stagger HDR mode refers to a mode where the sensor reads a line of long exposure data for an HDR frame and then immediately performs a short exposure on that line. The sensor interleaves reading data from each line with different exposure times to save waiting time. In other words, in stagger HDR mode, the sensor exposes two frames of data sequentially: one long exposure and the other a short exposure. Sequential exposure can also be described as interleaved exposure. Similarly, one long exposure frame can be described as one long frame, and the other short exposure frame can be described as one short frame.

[0049] The exposure time of the two frames (i.e., the long frame and the short frame) is configurable, as is the gain of the two frames. For example, when the exposure time of the long frame is longer, the interval between the exposure time of the long frame and the exposure time of the short frame is longer, which can easily introduce ghosting problems.

[0050] (2) Dual Conversion Gain (DCG) mode

[0051] DCG mode refers to the sensor simultaneously exposing two frames of data with different gains. In DCG mode, the sensor controls its internal conversion circuitry to achieve these different gains for the two frames.

[0052] DCG mode is a pixel-level dual-gain mode, which includes high conversion gain (HCG) and low conversion gain (LCG).

[0053] (3) Dual Amplifier Gain (DAG) mode

[0054] DAG mode refers to a sensor simultaneously exposing two frames of data with different gains. In DAG mode, the sensor controls its internal amplification circuitry to achieve these different gains for the two frames.

[0055] DAG mode is a circuit-level dual-gain mode, and dual gain includes HCG and LCG.

[0056] DCG mode and DAG mode can be collectively referred to as DXG mode. That is, DXG mode includes DCG mode and / or DAG mode. DXG mode refers to the sensor controlling its hardware circuitry to achieve different gains for two exposed frames of data. DXG mode can also be described as a dual-gain mode.

[0057] 3. Deep computing mode

[0058] Depth computation can also be described as visual depth estimation or depth estimation. The purpose of depth estimation is to estimate the depth of a scene in an image, that is, the vertical distance from each pixel in the scene to the camera's imaging plane. Distance can be divided into absolute distance and relative distance. Depth computation modes can be divided into monocular depth computation mode and binocular depth computation mode.

[0059] (1) Monocular Depth Calculation Mode

[0060] Monocular depth calculation can also be described as monocular depth estimation, which refers to obtaining the depth of a scene in an image using a single image. Monocular depth calculation can employ either absolute depth estimation or relative depth estimation algorithms. These two algorithms can be implemented using networks, such as absolute depth estimation networks and relative depth estimation networks. The relative depth estimation network learns to extract depth difference information between adjacent pixels, while the absolute depth estimation network predicts the absolute depth value.

[0061] (2) Binocular Depth Calculation Mode

[0062] Binocular depth calculation can also be described as binocular depth estimation. Binocular depth estimation refers to using two images as input and calculating the cost volume through disparity to predict the depth value of each pixel in the images. These two images are taken simultaneously at the same time, depicting the same scene; for example, images of the same scene taken by two cameras (e.g., left and right cameras) at the same time. Binocular depth calculation may include processes such as feature point matching, epipolar correction, disparity calculation, and disparity-to-depth conversion. These processes involve complex computational processes such as multiple convolutional neural networks (CNNs), network modeling, and feature fusion.

[0063] Because the computational complexity and intensity of binocular depth computing are higher than those of monocular depth computing, electronic devices using binocular depth computing consume more power than those using monocular depth computing.

[0064] In bokeh effect capture scenarios, electronic devices consume power faster than in regular capture scenarios. Therefore, how to reduce power consumption in bokeh effect capture scenarios is a pressing technical problem that needs to be solved.

[0065] To address the aforementioned problems, this application provides a shooting method. This method can be applied to electronic devices equipped with image processing capabilities, such as mobile phones and tablet computers (i.e., electronic device 100).

[0066] By implementing the shooting method provided in this application embodiment, after starting the bokeh capture, if the statistical information of the shooting scene meets the mode switching conditions, the bokeh capture processing mode can be switched to a more power-saving processing mode, and the display effect of the bokeh capture image processed in this processing mode can be guaranteed.

[0067] Not limited to mobile phones and tablets, electronic device 100 can also be a desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, in-vehicle device, smart home device and / or smart city device. This application embodiment does not impose any special limitations on the specific type of electronic device.

[0068] Figures 1A-1F An example is shown of a set of user interfaces on an electronic device 100, which are described below in conjunction with... Figures 1A-1F This paper describes in detail the application scenarios of the shooting method provided in the embodiments of this application.

[0069] first, Figure 1A An example is shown of a user interface, or home page, on an electronic device 100 that displays installed applications. Figure 1A As shown, the main page displays one or more application icons, such as the "Clock" application icon, the "Calendar" application icon, the "Weather" application icon, and so on.

[0070] The aforementioned one or more application icons include the "Camera" application (hereinafter referred to as "Camera") icon, i.e., icon 111. Electronic device 100 can detect user actions performed on icon 111, such as a click. In response to this user action, electronic device 100 can activate the camera, capture an image, and display it. Figure 1B The user interface shown.

[0071] In another implementation, the lock screen user interface of the electronic device 100 displays a "camera" icon. The electronic device 100 can detect user actions performed on the "camera" icon, such as clicking and pulling up to the top of the lock screen user interface. In response to this user action, the electronic device 100 can activate the camera, capture an image, and display it. Figure 1BThe user interface shown.

[0072] Figure 1B An exemplary user interface is shown when an electronic device 100 runs a "camera" application. This user interface may include a window 121, a shooting control 122, a thumbnail display area 123, and a flip control 124. The user interface may also include various on / off controls, such as settings controls, filter on / off controls, AI camera on / off controls, flash on / off controls, etc. Figure 1B In the example, the filter switch control, AI camera switch control, and flash switch control are all shown as being turned off.

[0073] Window 121 is used to display the image captured by the camera, which can be understood as the display area of ​​the captured image, so that the user can input a click operation to the shooting control 122 or the flip control 124 based on the image displayed in window 121. In some embodiments, window 121 also includes some controls, such as a motion capture control 125, a focus adjustment control 126, and a portrait blur control 127. When the electronic device 100 detects a user operation on the focus adjustment control, it can adjust the focus of the camera in response to the user operation, for example, from 1x to 2x.

[0074] When the electronic device 100 detects a user operation on the motion capture control 125, it can respond to the user operation by initiating motion capture and outputting the text message "Motion capture enabled," and controlling the image in the motion capture control 125 to change color and for the colored image to be in a running state. After initiating motion capture, the electronic device continuously monitors whether the amplitude of the subject's movement in the shooting environment meets the capture conditions. If the capture conditions are met, the image of that instant can be captured; otherwise, image acquisition can continue. For example, if the amplitude of the subject's smile in the shooting environment exceeds an amplitude threshold, the electronic device 100 can capture an image of that smile.

[0075] Figure 1C An example is shown of the user interface of the electronic device 100 after detecting a user operation on the motion capture control 125. Figure 1C In the motion capture control 125, the color of the portrait is... Figure 1B The colors of the portraits in the motion capture control 125 are different. Figure 1C Window 121 displays the message "Motion capture is enabled". Figure 1C The switch control bar displays an automatic motion capture switch control 128, which is in the "on" state. When the automatic motion capture switch control 128 is in the "on" state, it indicates that the electronic device 100 can automatically detect whether the movement amplitude of the subject in the shooting environment meets the capture conditions, and capture an instantaneous image when the capture conditions are met.

[0076] If the electronic device 100 detects a user operation on the automatic motion capture switch control 128, it can respond to the user operation and turn off automatic motion capture. At this time, the image of the person in the motion capture control 125 continues to be in a running state, but the electronic device 100 does not automatically detect and capture, but instead responds to the user operation on the shooting control 122 and performs the capture.

[0077] When motion capture is activated, if the electronic device 100 detects a user operation on the portrait blur control 127, it can respond to the user operation by activating blur capture and outputting the text message "Blur capture is enabled", and controlling the background color change in the portrait blur control 127.

[0078] Figure 1D An example is shown of the user interface of the electronic device 100 after detecting a user operation on the motion capture control 125 and then a user operation on the portrait blurring control 127. Figure 1D In the middle, the color of the portrait in the portrait blurring control 127 is... Figure 1C The colors of the portraits in the portrait blurring control 127 are different. Figure 1D Window 121 displays the message "Blurring capture enabled". And... Figure 1D The switch control bar displays an automatic blur capture switch control 129, which is in the on state. When the automatic blur capture switch control 129 is in the on state, it means that the electronic device 100 can automatically detect whether the range of motion of the subject in the shooting environment meets the capture conditions, and capture the image when the capture conditions are met, and perform portrait blur processing on the captured image.

[0079] If the electronic device 100 detects a user operation on the automatic blurring capture switch control 129, it can respond to the user operation and turn off automatic blurring capture. At this time, the image in the motion capture control 125 continues to be in a running state, and the color of the image in the portrait blurring control 127 continues as shown. Figure 1D As shown, the electronic device 100 does not automatically detect and capture images, but instead performs a blurred capture in response to user operations applied to the shooting control 122.

[0080] Figures 1B to 1DThe process involves the electronic device 100 first detecting a user operation applied to the motion capture control 125, then initiating motion capture, and subsequently detecting a user operation applied to the portrait blurring control 127. In other words, the blurring capture control includes both the motion capture control 125 and the portrait blurring control 127; blurring capture can be initiated in response to the detection of user operations applied to these two controls. This is one implementation method. In another implementation method, the blurring capture control is simply a single control; the electronic device 100 can initiate blurring capture after detecting a user operation applied to this control.

[0081] Figure 1E An exemplary user interface is shown when an electronic device 100 runs a "camera" application. This user interface may include a window 121, a shooting control 122, a thumbnail display area 123, and a flip control 124. The user interface may also include various on / off controls, such as settings controls, filter on / off controls, AI camera on / off controls, flash on / off controls, etc. Figure 1E In the example, the filter switch control, AI camera switch control, and flash switch control are all shown as being turned off.

[0082] Figure 1E The window 121 shown includes a blur capture control 130. The blur capture control 130 can be understood as a control that integrates a motion capture control and a portrait blur control, and this control is used to enable blur capture.

[0083] When the electronic device 100 detects a user operation on the bokeh snapshot control 130, it can respond to the user operation by starting bokeh snapshot and outputting the text message "Bokeh snapshot is enabled." It also controls the portrait in the bokeh snapshot control 130 to change color and for the colored portrait to be in a running state. After starting bokeh snapshot, the electronic device continuously monitors whether the movement of the subject in the shooting environment meets the snapshot conditions. If the conditions are met, it captures the instantaneous image and performs portrait blurring on the instantaneous image.

[0084] Figure 1F An example is shown of the user interface of the electronic device 100 after detecting a user operation on the blurring capture control 130. Figure 1F In the middle, the color of the portrait in the blur capture control 130 and Figure 1D The colors of the portraits in the blurred capture control 130 are different, and the portraits in the blurred capture control 130 are in a running state. Figure 1F Window 121 displays the message "Blurring capture enabled". And... Figure 1FThe switch control bar displays an automatic blur capture switch control 131, which is in the on state. When the automatic blur capture switch control 131 is in the on state, it means that the electronic device 100 can automatically detect whether the range of motion of the subject in the shooting environment meets the capture conditions, and capture the image when the capture conditions are met, and perform portrait blur processing on the captured image.

[0085] If the electronic device 100 detects a user operation on the automatic blur capture switch control 131, it can respond to the user operation and turn off automatic blur capture. At this time, the image of the person in the blur capture control 130 continues to be in a running state, but the electronic device 100 does not automatically detect and capture, but instead responds to the user operation on the shooting control 122 and performs blur capture.

[0086] like Figures 1B to 1F The user interface shown also includes "Night Scene" mode, "Portrait" mode, "Photo" mode, and "Video" mode. These modes are for illustrative purposes only and do not constitute a limitation on the embodiments of this application. For example, in practical applications, it also includes "Multi-camera Video" mode, etc. The "Video" mode is used to record video files, and the "Multi-camera Video" mode is used to record video files when both the front and rear cameras are simultaneously activated. The "Night Scene" mode is used to capture night scene images, improving the clarity of night scene images. The "Portrait" mode is mainly used to capture portrait images; in this mode, the image processing method provided in the embodiments of this application can be used. The "Photo" mode is used to capture single-frame images. Figures 1B to 1F In the video, the user selected the "photo mode".

[0087] After the electronic device 100 performs automatic or passive bokeh capture, a thumbnail of the bokeh capture image (i.e., the image obtained by blurring the portrait at the moment of capture) can be displayed in the thumbnail display area 123. When the electronic device 100 detects a user operation on the thumbnail display area 123, it responds to the user operation by jumping to the user interface for browsing the bokeh capture image.

[0088] Using the shooting method provided in the embodiments of this application, the thumbnail display area 123 can display a thumbnail of the blurred capture image, or the blurred capture image can be saved in the gallery.

[0089] The shooting method provided in this application embodiment can be applied to one or more shooting scenarios as follows.

[0090] Shooting Scene 1: Flicker-free Scene

[0091] A shutter is a structure in a camera used to control the effective exposure time of the film sensor. It can be divided into global shutters and rolling shutters. A global shutter exposes all pixels on the entire sensor simultaneously, ending exposure at the same time after the same duration. A rolling shutter, on the other hand, scans line by line; each line undergoes a reset, exposure, and data readout process, and then exposure is performed line by line. Current sensors typically use rolling shutters. For all pixels in the same line, the start exposure time and exposure time are the same, meaning pixels in the same line receive the same amount of energy. However, pixels in adjacent lines receive different amounts of energy, resulting in bright and dark stripes in the image, a phenomenon known as flicker. In other words, due to the influence of the light source frequency, alternating bright and dark areas appear in the image, making it appear to flicker. The light source frequency can be, for example, 50Hz or 60Hz.

[0092] Non-Flicker scenes can be understood as scenes where there is no influence from the light source frequency, or scenes where there is no light source, or scenes where the flicker intensity is less than a threshold, etc.

[0093] In non-Flicker scenarios, if bokeh snapshot is enabled, the shooting method provided in this application embodiment can be used to save power consumption of electronic devices.

[0094] For example, see Figure 2A and Figure 2B The shooting scene shown. Figure 2A The shooting scene shown can be understood as a Flicker scene. Figure 2B The shooting scenarios shown can be understood as non-Flicker scenarios. It's understandable that in scenarios where the lights are on and the projector is playing, and the electronic device has bokeh effect enabled, the stagger HDR mode can be used to capture the scene; conversely, in scenarios where the lights are off and the projector is off, and the electronic device has bokeh effect enabled, the DXG mode can be used to save power.

[0095] Shooting Scene 2: Low-light Scene

[0096] Low-light scenes refer to shooting environments with low brightness. In such scenarios, when the electronic device's bokeh effect is enabled, a monocular depth calculation mode can be used to save power. Furthermore, the bokeh effect captured using monocular depth calculation mode in these scenes also meets the display requirements for bokeh capture. Low-light scenes can also be understood as scenarios where monocular depth calculation mode is acceptable.

[0097] For example, see Figure 2C and Figure 2D The shooting scene shown. Figure 2C The shooting scene shown can be understood as a bright scene, such as indoor lights being on, or sunny weather during the day; Figure 2D The shooting scenarios shown can be understood as low-light scenes, such as indoor lights being off, dusk approaching, or overcast skies during the day. For window scenes, the scene inside the window might be a low-light scene, while the scene outside might be a bright-light scene. This is understandable. Figure 2C In the shooting scenario shown, with the bokeh effect enabled, the electronic device can use a binocular depth calculation mode to capture the scene; for Figure 2D In the shooting scenario shown, when the bokeh effect is enabled, the electronic device can use a monocular depth calculation mode to shoot the scene, thereby saving the power consumption of the electronic device.

[0098] Shooting Scene 3: Non-HDR Scene

[0099] Non-HDR scenes can be understood as standard DR scenes, which are scenes where the difference between bright and dark areas is not significant. Alternatively, they can be scenes where the requirements for image brightness range and detail rendering are not as high. Non-HDR scenes can also be understood as scenes where the dynamic range is acceptable in DXG mode.

[0100] For example, see Figure 2E and Figure 2F The shooting scene shown. Figure 2E The shooting scene shown can be understood as an HDR scene. Figure 2F The shooting scene shown can be understood as a non-HDR scene. It is understandable that for... Figure 2E In the shooting scenario shown, with the electronic device's bokeh effect enabled, the stagger HDR mode can be used to capture this scene; for Figure 2F In the shooting scene shown, when the electronic device has the bokeh effect enabled, the DXG mode can be used to shoot the scene in order to save the power consumption of the electronic device.

[0101] The three shooting scenarios described above are for illustrative purposes only and do not constitute a limitation on the embodiments of this application. For example, they can also be applied to scenarios such as automatic exposure (AE) of the auxiliary sensor and scenarios where the exposure time of the auxiliary sensor is extended or shortened. The three shooting scenarios described above can be superimposed, for example, a non-HDR scene and a low-light scene, a non-Flicker scene and a low-light scene, etc.

[0102] Understandably, for bokeh effect capture, in some shooting scenarios, electronic devices can switch the sensor's HDR mode from stagger HDR mode to DXG mode, and / or switch the depth calculation mode from binocular depth calculation mode to monocular depth calculation mode to save power consumption.

[0103] The following details the implementation of electronic device 100. Figure 1D or Figure 1F The specific process of the user interface shown.

[0104] first, Figure 3 The software architecture of electronic device 100 is illustrated as an example.

[0105] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0106] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, the Hardware Abstraction Layer (HAL), and the kernel layer.

[0107] The application layer can include a series of application packages. For example... Figure 3 As shown, the application package may include applications such as camera, gallery, video, music, navigation, calendar, map, and WLAN. In this embodiment, the camera can provide services such as... Figures 1B to 1F As shown in the user interface, the electronic device 100 can display thumbnails of the blurred captured images in the thumbnail display area 123, and can also save the blurred captured images to the gallery. After activating the camera, the electronic device 100 can call the camera to capture images. The electronic device 100 may include at least one camera.

[0108] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions. In this embodiment, the application framework layer includes a camera service framework, which may contain functions that support bokeh effect capture.

[0109] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0110] The Android Runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that the Java language needs to call, and the other part contains the core Android libraries.

[0111] The system library can include multiple functional modules. For example, it may include a surface manager, media libraries, and 3D graphics processing libraries (e.g., OpenGL). The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media libraries support playback and recording of various common audio and video formats, as well as still image files. The media libraries support various audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing libraries are used for 3D graphics drawing, image rendering, compositing, and layer processing.

[0112] The Hardware Abstraction Layer (HAL) is an interface layer located between the kernel layer and the hardware, used to control hardware operations. In this embodiment, the HAL may include a bokeh effect capture algorithm for implementing bokeh capture. The bokeh capture algorithm may include one or more of the following: stagger HDR mode algorithm, DXG mode algorithm, monocular depth calculation mode algorithm, binocular depth calculation mode algorithm, portrait bokeh algorithm, etc.

[0113] The algorithm for the stagger HDR mode is used to support the stagger HDR mode. This algorithm may include one or more of the following: main sensor long exposure, main sensor short exposure, frame selection and multi-frame fusion algorithm, multi-exposure fusion algorithm, and auxiliary sensor exposure.

[0114] The algorithm used to support DXG mode may include main sensor exposure, as well as selection and multi-frame fusion algorithms.

[0115] Algorithms for monocular depth calculation are used to support monocular depth calculation, while algorithms for binocular depth calculation are used to support binocular depth calculation. Portrait blurring algorithms are used to achieve portrait blurring, that is, to highlight the detailed features of the portrait while blurring the background.

[0116] The kernel layer is the foundation of the Android system. For example, ART relies on the kernel layer to execute low-level functions, such as threads and low-level memory management. The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, sensor drivers, and GPU drivers. In this embodiment, the kernel layer includes a camera driver and a display driver. The camera driver is used to drive the camera to capture images, and the display driver is used to drive the display screen to display the blurred captured images.

[0117] based on Figure 3 The software architecture shown illustrates the shooting method provided in the embodiments of this application. In some embodiments, the electronic device 100 detects a user operation on the camera application and, in response to the user operation, starts the camera to capture images. Upon detecting the user operation, the camera application triggers a command to start the camera. The camera application calls the API interface of the application framework layer to send the command to the camera service framework. The camera service framework calls the hardware abstraction layer to send the command to the camera driver. The camera driver can then start the camera and drive it to capture images. The captured images can be cached in an image buffer.

[0118] After the electronic device 100 activates the camera, the camera service framework, in response to detecting user operation on the blur capture control in the user interface of the camera application, initiates blur capture and retrieves preview stream data of the shooting scene from the image buffer. The camera service framework transmits the preview stream data to the hardware abstraction layer. The hardware abstraction layer determines the statistical information of the shooting scene based on the preview stream data and checks whether the statistical information meets the mode switching conditions. If it does, it switches the sensor's HDR mode from stagger HDR mode to DXG mode, and / or switches the depth calculation mode from binocular depth calculation mode to monocular depth calculation mode. In response to the blur capture command for the shooting scene, the camera service framework calls the camera driver to capture the image to be processed of the shooting scene and transmits the image to be processed to the hardware abstraction layer. Based on the switched mode, the hardware abstraction layer processes the image to be processed to obtain the blur capture image of the shooting scene and transmits the blur capture image to the display driver so that the electronic device 100 displays a thumbnail of the blur capture image. The blurring capture command can be a command generated by the hardware abstraction layer based on the motion detection algorithm when the motion amplitude of the subject meets the capture conditions, or it can be a command generated by the user's operation based on the user clicking the shooting control 122 when blurring capture is enabled.

[0119] Figure 4 A flowchart illustrating an embodiment of the shooting method provided in this application is provided. (In conjunction with...) Figures 1A to 1F The user interface shown and Figure 3The software architecture of the electronic device 100 shown will be described in detail below, and the process of the shooting method provided in the embodiments of this application will be described in detail below.

[0120] 401, Electronic device 100 detects a user operation on the bokeh capture control in the user interface of the camera application, and in response to the user operation, initiates bokeh capture and calls the camera device to collect preview stream data of the shooting scene.

[0121] The user interface of the camera application in step 401 refers to the user interface of the camera application output by the electronic device 100 in response to detected user operations performed on the camera application, such as... Figure 1B or Figure 1E The user interface shown.

[0122] In one implementation, the user interface includes a blurring capture control, for example... Figure 1E The device 100 detects a user action (e.g., a click) on the blurring capture control in the user interface, and in response to the user action, initiates blurring capture and calls a camera device (e.g., one or more cameras) to capture preview stream data of the scene. After initiating blurring capture, the image of the person in the blurring capture control changes color and the image appears to be running.

[0123] In another implementation, the user interface includes motion capture controls and portrait blurring controls, for example... Figure 1B The device 100 includes a motion capture control 125 and a portrait blurring control 127. These two controls constitute a blurring capture control. The electronic device 100 detects a user operation on the motion capture control in the user interface, and in response to the user operation, initiates motion capture and calls a camera device to collect preview stream data of the shooting scene. After initiating motion capture, the portrait in the motion capture control changes color and the portrait is in a running state. Subsequently, the device detects another user operation on the portrait blurring control in the user interface, and in response to the user operation, initiates blurring capture and continues to collect preview stream data of the shooting scene. After initiating blurring capture, the portrait in the portrait blurring control changes color.

[0124] Activating the bokeh snapshot function can be understood as activating or enabling the bokeh snapshot function. In this way, the electronic device 100 can capture images and blur portraits based on the movement of the subject.

[0125] The shooting scene refers to the scene that the camera can currently capture. The preview stream data of the shooting scene refers to the preview image captured by the camera in real time before automatic or passive capture. The preview stream data can include multiple preview images.

[0126] Optionally, the electronic device 100 detects a user operation applied to the camera application, outputs the user interface of the camera application, and calls the camera device to collect preview stream data of the shooting scene. Upon detecting a user operation applied to the bokeh capture control, it initiates bokeh capture and continuously collects preview stream data of the shooting scene.

[0127] Optionally, the electronic device 100 detects a user operation applied to the camera application, outputs the user interface of the camera application, and invokes the camera device to capture images. Upon detecting a user operation applied to the bokeh capture control, it initiates bokeh capture and invokes the camera device to capture preview stream data of the shooting scene. In other words, the series of images captured by the camera device after initiating bokeh capture is referred to as the preview stream data of the shooting scene.

[0128] 402. Electronic device 100 determines statistical information about the shooting scene based on preview stream data.

[0129] The electronic device 100 performs statistical analysis on the acquired preview stream data to determine statistical information about the shooting scene. This statistical information is used to determine whether to switch to the bokeh capture mode. The statistical information may include, but is not limited to, one or more of the following: scene brightness statistics, dynamic range statistics, flicker intensity detection statistics, depth of field statistics in the subject outline layer, and automatic exposure (AE) information.

[0130] Scene luminance statistics refer to the average scene luminance value of multiple preview images included in the preview stream data. For a single preview image, the sensor can read the scene luminance value of the image before reading the image data.

[0131] Dynamic range statistics refer to the average dynamic range values ​​of multiple preview images included in the preview stream data. For a single preview image, the ratio of the sum of the pixels in the largest and smallest bins of its scene histogram to the pixels in the middle bins represents the dynamic range value of that preview image. Taking a 128-bin scene histogram as an example, the ratio between the sum of the pixels in bin1 and bin128 and the pixels in bin64 can be used as the dynamic range value. A larger dynamic range value indicates a higher scene dynamic range.

[0132] The flicker intensity detection statistics refer to the average flicker intensity detection values ​​of multiple preview images included in the preview stream data. For a single preview image, the electronic device 100 can invoke a flicker detection algorithm to detect its flicker intensity and obtain a flicker intensity detection value.

[0133] The statistical value of medium depth of field in the subject outline layer refers to the average value of the medium depth of field in the subject outline layer of multiple preview images included in the preview stream data. For a single preview image, the electronic device 100 performs an image morphological operation (dilation) on it to obtain a first dual-camera depth map, and performs an image morphological operation (erosion) on the same preview image to obtain a second dual-camera depth map. A differential signal map is calculated based on the first and second dual-camera depth maps. The depth information values ​​are statistically analyzed on the differential signal map to obtain the medium depth of field.

[0134] Automatic exposure information describes information related to automatic exposure, such as whether automatic exposure is enabled, the aperture size of automatic exposure, the exposure time of automatic exposure, the gain of automatic exposure, etc.

[0135] The above statistical information is used as an example and does not constitute a limitation on the embodiments of this application. For example, it may also include other information for determining whether to switch the blur capture mode.

[0136] 403, In response to the statistical information meeting the mode switching conditions, the electronic device 100 switches the blurring capture processing mode from the first processing mode to the second processing mode.

[0137] The mode switching conditions may include a series of thresholds, such as one or more of the following: scene brightness threshold, dynamic range threshold, flicker intensity threshold, depth of field threshold, etc. These thresholds can be understood as pre-calibrated thresholds, which can be set in the electronic device at the factory or based on user-input values; the specific values ​​are not limited in this embodiment. These thresholds can also be understood as empirical thresholds, i.e., some empirical thresholds summarized by researchers.

[0138] The scene brightness threshold is compared with the scene brightness statistics. A scene brightness statistics value less than the scene brightness value indicates a low-light scene, while a scene brightness statistics value greater than or equal to the scene brightness value indicates a bright scene. The dynamic range threshold is compared with the dynamic range statistics. A dynamic range statistics value less than the dynamic range threshold indicates a non-HDR scene, while a dynamic range statistics value greater than or equal to the dynamic range threshold indicates an HDR scene. The flicker intensity threshold is compared with the flicker intensity detection statistics. A flicker intensity detection statistics value less than the flicker intensity threshold indicates a non-flicker scene, while a flicker intensity detection statistics value greater than or equal to the flicker intensity threshold indicates a flicker scene. The depth of field threshold is compared with the depth of field statistics in the subject outline layer. A depth of field statistics value less than the depth of field threshold indicates a non-medium depth of field ratio scene, while a depth of field statistics value greater than or equal to the depth of field threshold indicates a medium depth of field ratio scene.

[0139] Optionally, the mode switching conditions may also include one or more of the following: exposure time threshold, gain threshold, etc.

[0140] The bokeh effect capture processing modes are used to achieve bokeh capture, including High Dynamic Range (HDR) mode and Depth Calculation mode. HDR mode refers to the sensor's HDR mode, which can be divided into Interleaved HDR mode and Dual Gain HDR mode. Interleaved HDR mode can be the aforementioned Stagger HDR mode, and Dual Gain HDR mode can be the aforementioned DXG mode. For ease of description, Stagger HDR mode will be used to describe Interleaved HDR mode, and DXG mode will be used to describe Dual Gain HDR mode. Depth Calculation mode can be divided into Binocular Depth Calculation mode and Monocular Depth Calculation mode.

[0141] The high dynamic range mode in the second processing mode is different from the high dynamic range mode in the first processing mode, and / or the depth calculation mode in the second processing mode is different from the depth calculation mode in the first processing mode. This allows the electronic device to use different high dynamic range modes and / or different depth calculation modes in different shooting scenarios, thereby helping to save power consumption. In this embodiment, the high dynamic range mode in the first processing mode is the stagger HDR mode, and the depth calculation mode in the first processing mode is the binocular depth calculation mode, as an example. Using the bokeh capture processing mode as the first processing mode can achieve the best bokeh capture image effect, but this mode consumes more power.

[0142] Different statistical information leads to different mode switching conditions, resulting in different second processing modes.

[0143] In Method 1, the statistical information includes scene brightness statistics and dynamic range statistics. The mode switching condition is met if the scene brightness statistics are less than a scene brightness threshold, and the dynamic range statistics are less than a dynamic range threshold. Alternatively, the mode switching condition can be met if the ratio between the scene brightness statistics and the dynamic range statistics is less than a certain threshold; or if the product of the scene brightness statistics and the dynamic range statistics is less than a certain threshold; and so on. In Method 1, the high dynamic range mode in the second processing mode is DXG mode.

[0144] For method 1, when the electronic device 100 detects that the statistical information meets the mode switching conditions, it can output a low-power mode enable signal to switch from stagger HDR mode to DXG mode. For example, the camera application framework layer of the electronic device 100 outputs this low-power mode enable signal to the hardware abstraction layer.

[0145] Method 2: The statistical information includes flicker intensity detection statistics. The mode switching condition is met if the flicker intensity detection statistics are less than a flicker intensity threshold. In this method, the high dynamic range mode in the second processing mode is DXG mode.

[0146] For method 2, when the electronic device 100 detects that the statistical information meets the mode switching conditions, it can output a low-power mode enable signal to switch from stagger HDR mode to DXG mode. For example, the camera application framework layer of the electronic device 100 outputs this low-power mode enable signal to the hardware abstraction layer.

[0147] In Method 3, the statistical information includes the statistical values ​​of the depth of field in the subject outline layer. The mode switching condition is met if the statistical values ​​of the depth of field in the subject outline layer are less than the depth of field threshold. In Method 3, the depth calculation mode in the second processing mode is the monocular depth calculation mode.

[0148] For method 3, when the electronic device 100 detects that the statistical information meets the mode switching conditions, it can output a low-power mode enable signal to switch from the binocular depth calculation mode to the monocular depth calculation mode. For example, the camera application framework layer of the electronic device 100 outputs this low-power mode enable signal to the hardware abstraction layer.

[0149] Method 4 uses statistical information including scene brightness statistics, dynamic range statistics, and depth-of-field statistics in the subject outline layer. The statistical information must meet the mode switching conditions: the scene brightness statistics must be less than the scene brightness threshold, the dynamic range statistics must be less than the dynamic range threshold, and the depth-of-field statistics in the subject outline layer must be less than the depth-of-field threshold. In Method 4, the high dynamic range mode in the second processing mode is DXG mode, and the depth calculation mode is monocular depth calculation mode.

[0150] In Method 5, the statistical information includes flicker intensity detection statistics and depth-of-field statistics in the subject outline layer. The statistical information must meet the mode switching conditions: the flicker intensity detection statistics must be less than the flicker intensity threshold, and the depth-of-field statistics in the subject outline layer must be less than the depth-of-field threshold. In Method 5, the high dynamic range mode in the second processing mode is DXG mode, and the depth calculation mode is monocular depth calculation mode.

[0151] For methods 4 and 5, when the electronic device 100 detects that the statistical information meets the mode switching conditions, it can output a low-power mode enable signal to switch from stagger HDR mode to DXG mode and from binocular depth calculation mode to monocular depth calculation mode. For example, the camera application framework layer of the electronic device 100 outputs this low-power mode enable signal to the hardware abstraction layer.

[0152] The methods 1 to 5 described above are for illustrative purposes only and do not constitute a limitation on the embodiments of this application. Other combinations are possible, for example, a combination of flicker intensity detection statistics and AE information, both of which are less than the corresponding threshold, and the depth calculation mode in the second processing mode is a monocular depth calculation mode.

[0153] For methods 1 and 2 above, when the electronic device 100 switches the bokeh processing mode from the first processing mode to the second processing mode, it switches the high dynamic range mode in the bokeh capture processing mode from stagger HDR mode to DXG mode to save power consumption. The high dynamic range mode in the second processing mode is DXG mode. The depth calculation mode in the second processing mode can be a binocular depth calculation mode (i.e., the same as the depth calculation mode in the first processing mode) or a monocular depth calculation mode (i.e., switching from binocular depth calculation mode to monocular depth calculation mode). The switch from binocular depth calculation mode to monocular depth calculation mode can be triggered by statistical values ​​of equal depth of field in the subject outline layer. Switching the high dynamic range mode in the bokeh capture processing mode from stagger HDR mode to DXG mode can be achieved by controlling the main sensor's operating mode to switch from stagger HDR mode to DXG mode.

[0154] Specifically, for the main sensor operating in stagger HDR mode, the main sensor outputs images at 60fps, twice the normal 30fps. Frames are output in pairs, consisting of a long exposure and a short exposure, respectively. These two frames are obtained by the main sensor performing two exposures.

[0155] When the main sensor operates in DXG mode, the output frame rate is 30fps, the same as the normal output frame rate. The bit width of each frame is 2 bits wider than the standard 10-bit. The data in this mode is obtained by ISP fusion within the main sensor after reading the same exposure using two Conversion Gain methods, resulting in a relatively high dynamic range.

[0156] For method 3 above, when the electronic device 100 switches the bokeh processing mode from the first processing mode to the second processing mode, it switches the depth calculation mode in the bokeh capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode to reduce depth calculation power consumption, thereby saving power consumption of the electronic device. The high dynamic range mode in the second processing mode can be either stagger HDR mode (i.e., the same as the high dynamic range mode in the first processing mode) or DXG mode (i.e., switching from interlaced high dynamic range mode to dual-gain high dynamic range mode). The switch from stagger HDR mode to DXG mode can be triggered by scene brightness statistics and dynamic range statistics, or by flicker intensity detection statistics, or by scene brightness statistics, dynamic range statistics, and flicker intensity detection statistics. Switching the depth calculation mode in the bokeh capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode includes: controlling the depth calculation path of the bokeh capture processing mode to switch from the binocular depth calculation path to the monocular depth calculation path. Switching from a binocular depth computing path to a monocular depth computing path can save power consumption in electronic devices.

[0157] The binocular depth calculation path is used to execute the binocular depth calculation mode. The output of the multi-exposure fusion algorithm or the data frame after multi-frame noise reduction of DXG data is used to perform binocular depth calculation with the auxiliary sensor data, and the result is output to the subsequent portrait blurring algorithm. The monocular depth calculation path is used to execute the monocular depth calculation mode. The data after multi-frame noise reduction of DXG data is sent to the monocular depth calculation and the result is output to the subsequent portrait blurring algorithm.

[0158] For methods 4 and 5 above, when the electronic device 100 switches the bokeh processing mode from the first processing mode to the second processing mode, it switches the high dynamic range mode in the bokeh capture processing mode from stagger HDR mode to DXG mode, and switches the depth calculation mode in the bokeh capture processing mode from binocular depth calculation mode to monocular depth calculation mode. Switching the high dynamic range mode in the bokeh capture processing mode from stagger HDR mode to DXG mode includes: controlling the main sensor's operating mode to switch from stagger HDR mode to DXG mode, and controlling the auxiliary sensor's operating mode to switch from outgoing flow mode to non-outgoing flow waiting mode. Switching the depth calculation mode in the bokeh capture processing mode from binocular depth calculation mode to monocular depth calculation mode includes: controlling the depth calculation path in the bokeh capture processing mode to switch from binocular depth calculation path to monocular calculation path.

[0159] Specifically, for the auxiliary sensor operating in the outgoing output mode, the image output speed is 30fps. For the auxiliary sensor operating in the non-output waiting mode, the auxiliary sensor does not output image data, but can quickly recover to the state of output data; this state can be called standby state.

[0160] Before switching the depth calculation mode in the bokeh capture processing mode from binocular depth calculation mode to monocular depth calculation mode, the exposure fusion path of the bokeh capture processing mode is switched from multi-exposure fusion path to single-exposure fusion path to further save the power consumption of electronic devices.

[0161] The multi-exposure fusion path is as follows: the long exposure data of the main sensor is processed by frame selection and multi-frame fusion algorithm to obtain multi-frame noise-reduced data. This data is then fused with the short exposure frames of the main sensor using the multi-exposure fusion algorithm to obtain fused frame data. This data is then combined with depth calculation (monocular or binocular) information to perform a portrait blurring algorithm to obtain a blurred capture image.

[0162] The single exposure path is as follows: the DXG frame data from the main sensor is processed by frame selection and high bit width multi-frame fusion algorithm to obtain multi-frame noise-reduced data, and then combined with depth calculation (monocular or binocular) information to perform portrait blurring algorithm to obtain blurred capture image.

[0163] 404, In response to the command to blur the shooting scene, the electronic device 100 calls the camera device to capture the image of the shooting scene to be processed.

[0164] The blurring capture command for the shooting scene can be a command generated by the electronic device 100 based on the motion detection algorithm when the motion amplitude of the subject meets the capture conditions, or it can be a command generated by the user based on the user's operation of clicking the shooting control 122 when blurring capture is enabled.

[0165] When electronic device 100 detects a blurring capture command for the shooting scene, it invokes a camera device to capture an image of the scene to be processed. Optionally, electronic device 100 invokes one camera to capture the image to be processed. Optionally, electronic device 100 invokes two cameras to capture the image to be processed.

[0166] 405. Electronic device 100 processes the image to be processed based on the second processing mode to obtain a blurred snapshot image of the shooting scene.

[0167] The electronic device 100 processes the image to be processed based on the second processing mode to obtain a blurred snapshot image of the shooting scene.

[0168] The process by which electronic device 100 obtains a blurred capture image using the first processing mode can be found in [reference needed]. Figure 5A As shown. Figure 5A In the process, the bokeh capture mode is determined as the first processing mode based on the preview stream data. In response to the bokeh capture command, both the main sensor and the auxiliary sensor output images. The main sensor adopts the stagger HDR mode and outputs two frames of data, long exposure data and short exposure data. The long exposure data is processed by frame selection and multi-frame fusion algorithm and then combined with the short exposure data for multi-exposure fusion processing. Then, binocular depth calculation and portrait bokeh processing are performed to obtain the bokeh capture image.

[0169] The process by which electronic device 100 obtains a blurred capture image using the second processing mode can be found in [reference needed]. Figures 5B to 5D As shown. Figure 5B In the process, based on the preview stream data, the bokeh capture mode is determined to be the second processing mode (DXG mode). In response to the bokeh capture command, both the main sensor and the auxiliary sensor output images. The main sensor adopts DXG mode and outputs one frame of data. This frame of data is processed by frame selection and multi-frame fusion algorithm and then combined with the auxiliary sensor data to perform binocular depth calculation. Then, portrait bokeh processing is performed to obtain the bokeh capture image. Figure 5C In the process, based on the preview stream data, the bokeh capture mode is determined to be the second processing mode (monocular depth calculation mode). In response to the bokeh capture command, the main sensor outputs an image. The main sensor adopts the stagger HDR mode and outputs two frames of data, long exposure data and short exposure data. The long exposure data is processed by frame selection and multi-frame fusion algorithm and then combined with the short exposure data for multi-exposure fusion processing. Then, monocular depth calculation and portrait bokeh processing are performed to obtain the bokeh capture image. Figure 5DIn the process, based on the preview stream data, the bokeh capture mode is determined to be the second processing mode (DXG mode and monocular depth calculation). In response to the bokeh capture command, the main sensor outputs an image. The main sensor adopts DXG mode and outputs one frame of data. This frame of data is processed by frame selection and multi-frame fusion algorithm, and then monocular depth calculation is performed. Then, portrait bokeh processing is performed to obtain the bokeh capture image.

[0170] contrast Figure 5A and Figures 5B-5D It can be seen that, Figures 5B-5D The process is less, which helps save power consumption of electronic devices.

[0171] 406, The thumbnail display area of ​​the user interface of the electronic device 100 in the camera application displays a thumbnail of the blurred captured image of the shooting scene.

[0172] The electronic device 100 captures a blurred image, which is then displayed as a thumbnail in the user interface of the camera application, for example... Figure 1D and Figure 1F The thumbnail display area 123 displays a thumbnail of the blurred captured image. Optionally, the electronic device 100 can save the blurred captured image to a gallery.

[0173] exist Figure 4 In the illustrated embodiment, in a bokeh capture scenario, if the statistical information of the shooting scene meets the mode switching conditions, the bokeh capture processing mode can be switched from the first processing mode to the second processing mode. Then, the captured image to be processed can be processed based on the second processing mode, thereby saving the power consumption of electronic devices in a bokeh capture scenario.

[0174] Figure 4The illustrated embodiments describe the process of switching from a first processing mode to a second processing mode. The statistical information of the shooting scene may change over time. If the shooting scene changes to satisfy the first processing mode, the bokeh capture processing mode can be switched from the second processing mode back to the first processing mode. For example, for method 1 above, if the scene brightness statistics are greater than or equal to the scene brightness threshold, and / or the dynamic range statistics are greater than or equal to the dynamic range threshold, the high dynamic range mode can be switched from DXG mode to stagger HDR mode. As another example, for method 2 above, if the flicker intensity detection statistics are greater than or equal to the flicker intensity threshold, the high dynamic range mode can be switched from DXG mode to stagger HDR mode. As yet another example, for method 3 above, if the depth-of-field statistics in the subject outline layer are greater than or equal to the depth-of-field threshold, the depth calculation mode can be switched from monocular depth calculation mode to binocular depth calculation mode. For example, in method 4 above, if the scene brightness statistics are less than the scene brightness threshold and the dynamic range statistics are less than the dynamic range threshold, but the depth-of-field statistics in the subject outline layer are greater than or equal to the depth-of-field threshold, then the DXG mode can continue to be used, and the depth calculation mode can be switched from monocular depth calculation mode to binocular depth calculation mode. Alternatively, if the depth-of-field statistics in the subject outline layer are less than the depth-of-field threshold, but the scene brightness statistics are greater than or equal to the scene brightness threshold, and / or the dynamic range statistics are greater than or equal to the dynamic range threshold, then the monocular depth calculation mode can continue to be used, and the high dynamic range mode can be switched from DXG mode to stagger HDR mode. Alternatively, if the scene brightness statistics are greater than or equal to the scene brightness threshold, and / or the dynamic range statistics are greater than or equal to the dynamic range threshold, and the depth-of-field statistics in the subject outline layer are greater than or equal to the depth-of-field threshold, then the high dynamic range mode can be switched from DXG mode to stagger HDR mode, and the depth calculation mode can be switched from monocular depth calculation mode to binocular depth calculation mode.

[0175] Figure 6 An exemplary schematic diagram of the hardware structure of electronic device 100 is shown.

[0176] Electronic device 100 may include processor 110, external memory interface 12B, internal memory 12A, universal serial bus (USB) interface 13A, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio processing module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0177] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0178] Processor 110 may include one or more processing units, such as an access point (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors. In this embodiment, processor 110 may include a bokeh capture algorithm.

[0179] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0180] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0181] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0182] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0183] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via a USB interface 13A. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0184] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 12A, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0185] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0186] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0187] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0188] A modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor.

[0189] The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0190] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0191] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0192] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU performs mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information. In this embodiment, the display screen 194 is used to display a blurred snapshot image.

[0193] The internal memory 12A may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0194] Random access memory (RAM) can include static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), and double data rate synchronous dynamic random-access memory (DDR SDRAM, such as fifth-generation DDR SDRAM, which is generally called DDR5 SDRAM). Non-volatile memory can include disk storage devices and flash memory.

[0195] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0196] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0197] The external memory interface 12B can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 12B to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0198] Electronic device 100 can implement audio functions, such as music playback and recording, through an audio processing module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, and an application processor.

[0199] The audio processing module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio processing module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio processing module 170 may be located in the processor 110, or some functional modules of the audio processing module 170 may be located in the processor 110.

[0200] Speaker 170A, also known as a "loudspeaker". Electronic device 100 can listen to music or make hands-free calls through speaker 170A. Receiver 170B, also known as a "handpiece". When electronic device 100 answers a phone call or voice message, the receiver 170B can be brought close to the user's ear to hear the voice. Microphone 170C, also known as a "microphone" or "voice transducer". When making a phone call or sending a voice message, the user can bring their mouth close to microphone 170C to speak, inputting the sound signal into microphone 170C.

[0201] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 13A interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, or a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0202] A pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be disposed on a display screen 194. A gyroscope sensor 180B may be used to determine the motion posture of the electronic device 100. A barometric pressure sensor 180C is used to measure barometric pressure. A magnetic sensor 180D includes a Hall effect sensor. The electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of a flip cover. An accelerometer sensor 180E can detect the magnitude of acceleration of the electronic device 100 in various directions (generally three axes). A distance sensor 180F is used to measure distance. A proximity sensor 180G may include, for example, a light-emitting diode (LED) and a photodetector, such as a photodiode. An ambient light sensor 180L is used to sense ambient light intensity. A fingerprint sensor 180H is used to collect fingerprints. A temperature sensor 180J is used to detect temperature. A touch sensor 180K, also known as a "touch device". Touch sensor 180K can be placed on display screen 194. The touch sensor 180K and display screen 194 together form a touch screen, also known as a "touchscreen". Touch sensor 180K is used to detect touch operations applied to or near it. Bone conduction sensor 180M can acquire vibration signals.

[0203] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0204] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, or to indicate messages, missed calls, notifications, etc.

[0205] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and detach from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0206] The term "user interface (UI)" used in the specification, claims, and drawings of this application refers to the medium through which an application or operating system interacts and exchanges information with the user. It converts information from its internal form to a form acceptable to the user. The user interface of an application is source code written in a specific computer language such as Java or Extensible Markup Language (XML). This source code is parsed and rendered on the terminal device, ultimately presenting user-recognizable content such as images, text, and buttons. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, images, and text. The attributes and content of controls in the interface are defined using tags or nodes, such as XML tags. <textview> 、 <imgview> 、 <videoview>Nodes define the controls contained in the interface. A node corresponds to a control or property in the interface, and after parsing and rendering, the node is presented as the content visible to the user. In addition, many applications, such as hybrid applications, often contain web pages within their interfaces. A web page, also known as a webpage, can be understood as a special control embedded in the application interface. Web pages are source code written in a specific computer language, such as Hypertext Markup Language (GTML), Cascading Style Sheets (CSS), JavaScript (JS), etc. Web page source code can be loaded and displayed as user-readable content by a browser or a web page display component with browser-like functionality. The specific content contained in a webpage is also defined through tags or nodes in the webpage source code; for example, GTML uses tags or nodes to define the content. 、 、 <video> 、 <canvas>Used to define the elements and attributes of a webpage.

[0207] The most common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0208] As used in the specification and appended claims of this application, the singular expressions "a," "an," "the," "the," "the," and "this" are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the listed items. As used in the above embodiments, depending on the context, the term "when" can be interpreted as meaning "if..." or "after..." or "in response to determining..." or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining..." or "in response to determining..." or "when (the stated condition or event) is detected" or "in response to detecting (the stated condition or event)."

[0209] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0210] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A shooting method, characterized in that, include: The user interface of the camera application detects a user operation on the bokeh capture control, and in response to the user operation, initiates bokeh capture and calls the camera device to collect preview stream data of the shooting scene; Based on the preview stream data, determine the statistical information of the shooting scene; In response to the statistical information satisfying the mode switching condition, the bokeh capture processing mode is switched from the first processing mode to the second processing mode; the bokeh capture processing mode includes a high dynamic range mode and a depth calculation mode, wherein the high dynamic range mode in the first processing mode is an interlaced high dynamic range mode, and the depth calculation mode in the first processing mode is a binocular depth calculation mode; the statistical information satisfying the mode switching condition includes a scene brightness statistical value less than a scene brightness threshold and a dynamic range statistical value less than a dynamic range threshold, or a flicker intensity detection statistical value less than a flicker intensity threshold, and the switching of the bokeh capture processing mode from the first processing mode to the second processing mode includes: switching the high dynamic range mode in the bokeh capture processing mode from the interlaced high dynamic range mode to the dual-gain high dynamic range mode; the statistical information The conditions for mode switching include that the statistical value of the depth of field in the outline layer of the subject is less than the depth of field threshold. Switching the bokeh capture processing mode from the first processing mode to the second processing mode includes: switching the depth calculation mode in the bokeh capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode. The conditions for mode switching include that the statistical value of the flicker intensity detection is less than the flicker intensity threshold and the statistical value of the depth of field in the outline layer of the subject is less than the depth of field threshold. Switching the bokeh capture processing mode from the first processing mode to the second processing mode includes: switching the high dynamic range mode in the bokeh capture processing mode from the interlaced high dynamic range mode to the dual gain high dynamic range mode, and switching the depth calculation mode in the bokeh capture processing mode from the binocular depth calculation mode to the monocular depth calculation mode. In response to the blurring capture command for the shooting scene, the camera device is invoked to capture the image of the shooting scene to be processed; Based on the second processing mode, the image to be processed is processed to obtain a blurred capture image of the shooting scene; In the thumbnail display area of ​​the user interface of the camera application, a thumbnail of the blurred snapshot image of the shooting scene is displayed.

2. The method as described in claim 1, characterized in that, The method is applied to an electronic device, which includes a main circuit sensor; The step of switching the high dynamic range mode in the bokeh capture processing mode from interlaced high dynamic range mode to dual-gain high dynamic range mode includes: The operating mode of the main sensor is switched from interleaved high dynamic range mode to dual-gain high dynamic range mode.

3. The method as described in claim 1, characterized in that, The method is applied to an electronic device, which includes a binocular depth calculation path and a monocular depth calculation path. The step of switching the depth calculation mode in the bokeh capture processing mode from binocular depth calculation mode to monocular depth calculation mode includes: The depth calculation path for controlling the blurring capture processing mode is switched from the binocular depth calculation path to the monocular depth calculation path.

4. The method as described in claim 1, characterized in that, The method is applied to an electronic device, which includes a main path sensor, an auxiliary path sensor, a binocular depth calculation path, and a monocular calculation path. The step of switching the high dynamic range mode in the bokeh capture processing mode from interleaved high dynamic range mode to dual-gain high dynamic range mode, and switching the depth calculation mode in the bokeh capture processing mode from binocular depth calculation mode to monocular depth calculation mode, includes: The operating mode of the main circuit sensor is controlled to switch from interleaved high dynamic range mode to dual gain high dynamic range mode, and the operating mode of the auxiliary circuit sensor is controlled to switch from working outflow mode to non-outflow waiting mode. The depth calculation path for controlling the blurring capture processing mode is switched from the binocular depth calculation path to the monocular calculation path.

5. The method as described in claim 4, characterized in that, The electronic device also includes a multi-exposure fusion path and a single-exposure fusion path; Before switching the depth calculation mode in the bokeh capture processing mode from binocular depth calculation mode to monocular depth calculation mode, the process also includes: The exposure fusion path of the control blur capture mode is switched from the multi-exposure fusion path to the single-exposure fusion path.

6. An electronic device, characterized in that, The method includes one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the method of any one of claims 1-5 to be performed.

7. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1-6.

8. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, they cause the method described in any one of claims 1-5 to be performed.

Images