Shooting method and electronic equipment

Abstract

The invention discloses a shooting method and electronic equipment, and is applied to the technical field of terminals. The method comprises the following steps: starting a camera sensor at a first moment; at a second moment, obtaining an exposure parameter and an aperture parameter of the Nth frame; at a third moment, the exposure parameters are issued to a camera sensor; at the fourth moment, the aperture assembly adjusts the aperture based on the aperture parameters; and issuing the first calibration data to the camera sensor at a fifth moment. According to the embodiment of the invention, the calibration data can be issued after the camera sensor starts stream, the image quality is improved, and the shooting experience of a user can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular, to a method and electronic device for photographing. Background Art

[0002] With the increasing development of smart terminals, the shooting function has become an essential function of smart terminals. Users' demand and experience for shooting (taking photos and / or recording videos) of smart terminals are also increasing. When shooting, smart terminals will control the blur of the background of the picture, or the blur effect, by adjusting the aperture of the camera. The aperture is a device used to control the amount of light entering. Generally speaking, the larger the aperture, the shallower the depth of field, and accordingly, the more obvious the blur effect of the image; the smaller the aperture, the smaller the amount of light entering, the deeper the depth of field, and accordingly, the weaker the blur effect of the image.

[0003] In order to obtain better image effects, the smart terminal will use the calibration data preset in the register to calibrate the image data of the camera sensor. However, for smart terminals with variable apertures, this method is not effective and affects the image effect. Summary of the invention

[0004] In view of this, the present application provides a method for shooting, an electronic device, a computer-readable storage medium and a computer program product, which can switch calibration data after the aperture changes, improve image effects, and help improve the user's shooting experience.

[0005] In a first aspect, a method for photographing is provided, the method being applied to an electronic device, the method comprising:

[0006] At the first moment, the camera sensor is activated;

[0007] At a second moment, acquiring exposure parameters and aperture parameters of the Nth frame; wherein the first moment is before the second moment;

[0008] At a third moment, the exposure parameter is sent to the camera sensor; the third moment is a moment corresponding to the start of frame delimiter SOF of the Nth frame; the second moment is before the third moment;

[0009] At a fourth moment, the aperture component adjusts the aperture based on the aperture parameter, and the fourth moment is a moment corresponding to the end-of-frame delimiter EOF of the N+1th frame;

[0010] At a fifth moment, the first calibration data is sent to the camera sensor; the fifth moment is a moment corresponding to the SOF of the N+2th frame.

[0011] The above method can be executed by an electronic device or a chip in the electronic device. Based on the above scheme, the electronic device starts the camera sensor at the first moment, adjusts the aperture based on the aperture parameters after receiving the aperture parameters, and sends the first calibration data to the camera sensor after starting to adjust the aperture. Compared with the method of burning a set of calibration data before the camera sensor starts streaming, the embodiment of the present application supports sending calibration data after the camera sensor starts streaming, which improves the image quality and helps to improve the user's shooting experience.

[0012] In a possible implementation, the electronic device includes a plurality of apertures, and the plurality of apertures includes at least a first aperture. In other words, the embodiment of the present application is applied to an electronic device with a variable aperture.

[0013] The embodiments of the present application are applied to the scenario where the camera sensor (sensor) adopts Quad Bayer Coding (QBC) to output images, or is suitable for Quad sensor. The pixel arrangement of the image data output by the Quad sensor is different from the pixel arrangement of the image data output by the traditional sensor. For the image data output by the Quad sensor, there are brightness differences (or light sensitivity differences) between sub-pixels of the same color, so Quad Bayer Coding Sensitivity Correction (QSC) calibration is required. This is because, if QSC correction (or QSC calibration) is not performed, the image data output by the Quad sensor will have uneven brightness, resulting in poor output image effect.

[0014] It can be understood that the embodiments of the present application are applicable to scenarios where users use electronic devices to take photos or videos.

[0015] In a possible implementation, the electronic device activates a camera sensor in response to a first operation of the user.

[0016] Optionally, the first operation is an operation of opening a camera application, wherein the electronic device has a camera application installed therein.

[0017] The embodiment of the present application does not limit the specific content of the first operation. The first operation can be understood as an operation of opening the camera application in any way. The first operation includes but is not limited to any of the following forms: through voice commands, physical buttons, and UI interactive operations, etc.

[0018] Exemplarily, the first operation is an operation in which a user clicks on a camera application in an interface of the electronic device.

[0019] In a possible implementation manner, the aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture.

[0020] The aforementioned exposure parameters and aperture parameters are obtained from the parameters sent by the automatic exposure module. It should be noted that the embodiments of the present application do not limit whether the timing of obtaining the exposure parameters and the aperture parameters is the same timing, or do not limit the order of obtaining the two. For example, after adjusting the parameters, the exposure parameters and the aperture parameters can be obtained at the same time. For another example, after adjusting the parameters, the exposure parameters can be obtained first, and the aperture parameters corresponding to the exposure parameters can be obtained later. For another example, after adjusting the parameters, the aperture parameters can be obtained first, and the exposure parameters corresponding to the aperture parameters can be obtained later.

[0021] In a possible implementation, obtaining exposure parameters and aperture parameters of the Nth frame includes:

[0022] In response to the second operation, the exposure parameter and the aperture parameter are acquired.

[0023] The exposure parameters and aperture parameters may be acquired in response to a second operation, and the specific form of the second operation is not limited in the embodiment of the present application. The second operation may be understood as an operation that triggers the aperture of the electronic device to switch.

[0024] Optionally, the second operation is an operation of manually adjusting the aperture by a user, or the second operation is an operation of automatically adjusting the aperture by the electronic device.

[0025] That is, at the second moment, the electronic device can obtain the exposure parameter and the aperture parameter in response to the user's manual aperture adjustment operation. For example, the second operation is the user's manual aperture adjustment operation in the professional shooting mode. For another example, the second operation is the user's operation after switching the current normal shooting mode to the large aperture shooting mode.

[0026] Alternatively, at the second moment, the electronic device can obtain the exposure parameters and the aperture parameters based on the operation of automatically adjusting the aperture. For example, the electronic device is equipped with a variable aperture; after detecting a specific shooting scene, the electronic device automatically adjusts the variable aperture, and the electronic device obtains the exposure parameters and the aperture parameters.

[0027] In a possible implementation, within the display duration corresponding to the N+3th frame, the image corresponding to the N+3th frame is displayed based on the exposure parameter, the aperture parameter and the first calibration data. That is, after the calibration data is sent down, the exposure parameter of the image of the N+3th frame sent for display matches the aperture parameter. Since the aperture has been switched to the first aperture and the first QSC calibration data sent down has also taken effect, the image effect corresponding to the N+3th frame obtained is better than the image effect obtained using the QSC calibration data corresponding to the original aperture. That is, compared with using the QSC calibration data of the original aperture (i.e., the aperture before switching) for compensation, using the QSC calibration data corresponding to the first aperture after switching for compensation will result in better image quality.

[0028] In a possible implementation, the first calibration data is calibration data corresponding to the first aperture. Exemplarily, when the camera sensor is a Quad sensor, the first calibration data corresponding to the first aperture is QSC calibration data. In this way, after switching to the first aperture, the QSC calibration data corresponding to the first aperture can be used to compensate for the image data output by the Quad sensor, thereby improving the image quality.

[0029] In a possible implementation, the aperture parameter is used to instruct the camera sensor to switch the current aperture to a first aperture. Through the aperture parameter, the aperture assembly switches the current aperture to an aperture corresponding to the aperture parameter, such as the first aperture, to achieve aperture switching.

[0030] Since sending the first calibration data requires a certain transmission time, in order to minimize the impact on the image frame during the process of sending the calibration data, the embodiment of the present application can pre-process the calibration data in advance, that is, the first calibration data sent is the pre-processed calibration data. After the pre-processed calibration data, the corresponding data volume occupies fewer bits, thereby reducing the transmission time.

[0031] In a possible implementation, the first calibration data is calibration data obtained by preprocessing the second calibration data, wherein the number of bits corresponding to the second calibration data is greater than the number of bits corresponding to the first calibration data.

[0032] The embodiments of the present application do not limit the specific method of preprocessing. One possible implementation method is that since part of the image area is used for cropping in a high zoom ratio shooting scene, the QSC calibration data can be cropped in the corresponding area, which can ensure the data compensation effect while reducing the amount of QSC calibration data and the number of affected image frames.

[0033] In a possible implementation manner, a transmission duration of the first calibration data is less than a frame interval between the N+2th frame and the N+3th frame.

[0034] In a possible implementation, the method further includes:

[0035] During the display duration of the N+2th frame, the image corresponding to the N+1th frame is displayed.

[0036] Since the process of sending the first calibration data affects the image of the N+2 frame, the image brightness of the N+2 frame becomes abnormal, and the image of the N+2 frame has not been QSC compensated, the image of the N+2 frame needs to be discarded when it is displayed. At this time, the image corresponding to the N+1 frame can be displayed to minimize the impact caused by not displaying the image of the N+2 frame.

[0037] In a possible implementation, starting a camera sensor includes:

[0038] In response to receiving a first operation of the user, a camera sensor is activated, wherein the first operation is used to turn on the camera.

[0039] The embodiments of the present application can be applied to a preview scene with a high zoom ratio (including photo preview or video preview) and a shooting environment with a high illumination environment.

[0040] In a possible implementation, after starting the camera sensor, the method further includes:

[0041] Displaying a first preview interface, wherein the first preview interface includes a zoom ratio option;

[0042] In response to a user adjusting the zoom ratio to a first zoom ratio based on the zoom ratio option, and detecting that the current shooting environment is a high-illuminance environment, the camera sensor adopts a full-size cropping mode;

[0043] Among them, the first preview interface is a photo preview interface or a video preview interface; the first zoom ratio is greater than a preset zoom ratio; the high illumination environment includes: the ambient illumination of the current shooting environment is greater than a first illumination threshold.

[0044] The embodiments of the present application can be applied to a preview scene with a high zoom ratio (including photo preview or video preview) and a shooting scene in a high dynamic environment.

[0045] In a possible implementation, after starting the camera sensor, the method further includes:

[0046] Displaying a second preview interface, wherein the second preview interface includes a zoom ratio option;

[0047] In response to the user adjusting the zoom ratio to a second zoom ratio based on the zoom ratio option, and detecting that the current shooting environment is a high-dynamic environment, the camera sensor adopts a full-size cropping mode;

[0048] Among them, the second preview interface is a photo preview interface or a video preview interface; the second zoom ratio is greater than a preset zoom ratio; the high dynamic environment includes: the dynamic range value satisfies the dynamic range DR constraint condition.

[0049] It should be noted that in each of the above shooting scenarios, the sensor uses a full size crop output mode. Therefore, in the above scenarios, if the aperture changes, QSC calibration is required to improve the image effect.

[0050] In a second aspect, an electronic device is provided, comprising a unit for executing any one of the methods in the first aspect. The electronic device may be a terminal or a chip in a terminal. The electronic device comprises a communication unit, a display unit and a processing unit.

[0051] When the electronic device is a terminal, the processing unit may be a processor, the communication unit may be a communication interface, and the display unit may be a graphics processing module and a screen; the terminal may also include a memory for storing computer program code, and when the processor executes the computer program code stored in the memory, the terminal executes any one of the methods in the first aspect.

[0052] When the electronic device is a chip in a terminal, the processing unit may be a logic processing unit inside the chip, the communication unit may be a communication interface, a pin or a circuit, etc., and the display unit may be a graphics processing unit inside the chip; the chip may also include a memory, which may be a memory inside the chip (for example, a register, a cache, etc.) or a memory located outside the chip (for example, a read-only memory, a random access memory, etc.); the memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the chip executes any one of the methods of the first aspect.

[0053] According to a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by an electronic device, the electronic device executes any one of the methods according to the first aspect.

[0054] According to a fourth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is executed by an electronic device, the electronic device executes any one of the methods according to the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1A It is an example diagram of an application scenario of an embodiment of the present application;

[0056] Figure 1B is another example diagram of an application scenario of an embodiment of the present application;

[0057] Figure 2A It is a schematic diagram of the pixel array distribution of QBC;

[0058] Figure 2B This is a schematic diagram of the before and after QSC calibration.

[0059] Figure 3 It is a schematic diagram of a software architecture applied in an embodiment of the present application;

[0060] Figure 4 Is applied to Figure 3 A sequential interaction diagram of the software architecture;

[0061] Figure 5 This is a timing example diagram of sending QSC calibration data according to an embodiment of the present application;

[0062] Figure 6 This is a schematic diagram of the preprocessing comparison of QSC correction data;

[0063] Figure 7 is a flowchart of a method for photographing according to an embodiment of the present application;

[0064] Fig. 8A This is an example diagram of an interface for a user to adjust the aperture in an embodiment of the present application;

[0065] Figure 8B is another example diagram of an interface for a user to adjust the aperture according to an embodiment of the present application;

[0066] Fig. 9 is a schematic diagram of the structure of an electronic device applicable to the present application;

[0067] Fig.10 It is a schematic block diagram of a device for photographing according to an embodiment of the present application. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0069] In the embodiments of the present application, unless otherwise specified, “plurality” may mean two or more than two.

[0070] The embodiments of the present application are applicable to electronic devices, which may be mobile phones, smart screens, tablet computers, wearable electronic devices, vehicle-mounted electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), projectors, and the like.

[0071] The embodiments of the present application are applicable to electronic devices including a camera sensor and multiple apertures. Among them, the multiple apertures can have different characteristics. By setting multiple apertures, different apertures correspond to different amounts of light entering (i.e., the amount of light entering the interior of the fuselage). In this way, when the electronic device switches to different apertures, different depth of field effects, or blurring effects of taking photos, can be obtained based on different amounts of light entering. In other words, compared to electronic devices with a single aperture and unable to adjust the amount of light entering, the advantage of setting multiple apertures in electronic devices is that the amount of light entering can be dynamically adjusted to optimize the depth of field effect. Furthermore, although electronic devices with fixed apertures can adjust the aperture parameters, the aperture parameters are simulated by an algorithm inside the electronic device, and the effect of this method is worse than adjusting the physical aperture (i.e., switching different apertures). The following describes the benefits of setting multiple apertures in electronic devices in combination with different scenarios.

[0072] In some embodiments, during the shooting process, the electronic device can select a suitable aperture based on the distance of the target shooting object to ensure the blur effect of the photo. Different depths of field will affect the blur effect of the background of the picture, resulting in different blur effects of the picture.

[0073] In some embodiments, in low-light shooting environments (e.g., night scenes), the electronic device can ensure the amount of light entering by switching to a large aperture, thereby presenting a better preview effect in the night shooting scene, making the preview image of the target object clearer.

[0074] Here, for unified explanation, a low illumination environment can be understood as a dark light environment. A low illumination environment includes: the ambient illumination of the shooting environment is less than a preset brightness threshold. Correspondingly, a high illumination environment includes: the ambient illumination of the shooting environment is greater than or equal to the preset brightness threshold. A high illumination environment can also be understood as a high brightness environment.

[0075] Ambient illumination refers to the light intensity of the shooting environment that the user is in. The value of ambient illumination can be represented by the following indicators: lighting value (LV), lux, or lux index, etc.

[0076] LV is used to estimate the ambient brightness, and its specific calculation formula is as follows:

[0077]

[0078] Among them, Exposure is the exposure time, Aperture is the aperture size, Iso is the sensitivity, and Luma is the average value of Y of the image in the XYZ color space.

[0079] For example, the ambient illumination is represented by LV, and accordingly, the brightness threshold is the LV threshold. When the ambient illumination of the shooting environment is greater than the LV threshold, the current shooting environment is a high illumination environment.

[0080] Optionally, in some implementations, the ambient illumination of the current shooting environment may be acquired through an ambient light sensor, and then based on the acquired ambient illumination, it is determined whether the current environment is a high illumination environment.

[0081] In some embodiments, in the professional shooting mode, the electronic device adjusts the aperture accordingly in response to the user manually adjusting the aperture parameters, thereby providing the user with a richer shooting experience.

[0082] Optionally, the aperture assembly of the electronic device includes at least a first aperture and a second aperture, and the amount of light entering corresponding to the first aperture is different from the amount of light entering corresponding to the second aperture. For example, the first aperture is a large aperture, and the second aperture is a small aperture; the amount of light entering corresponding to the first aperture is greater than the amount of light entering corresponding to the second aperture. In the embodiment of the present application, each aperture has corresponding calibration data. For example, the first aperture corresponds to first calibration data, and the second aperture corresponds to second calibration data. Figure 4 The meaning of calibration data is introduced in.

[0083] The above describes different scenarios of electronic devices with multiple apertures. The following describes different shooting scenarios to which the embodiments of the present application are applicable.

[0084] In some embodiments, in the photo preview mode, for a shooting scene with a zoom ratio of 2x or more (eg, 3x) and a high-illumination environment, the sensor uses a full-size crop mode to output the image.

[0085] In other embodiments, in the photo preview mode, for a shooting scene with a zoom ratio of 2x or more (for example, 3x) and a high dynamic range, the sensor uses a full size cropped staggered mode to output the image. For the output method of the full size cropped staggered mode, please refer to the description in the related art, which will not be repeated here.

[0086] Here, for the dynamic range of the shooting scene, it can be divided into high dynamic range and low dynamic range based on preset conditions. For example, if the dynamic range value of the shooting scene meets the DR (Dynamic Range) constraint condition, it is a high dynamic range; if the dynamic range value of the shooting scene does not meet the DR constraint condition, it is a low dynamic range.

[0087] Optionally, the DR constraint condition can be determined based on a histogram of a RAW image of the shooting scene. Specifically, the dynamic range of the scene is determined based on the percentage of overexposed pixels and the percentage of underexposed pixels in the image.

[0088] It should be understood that the above division process of the high dynamic range is only an exemplary description, and the embodiments of the present application are not limited thereto. In fact, it is also possible to determine whether it is a high dynamic range scene based on other methods in the art.

[0089] For ease of understanding, the following combination Figure 1A The photo preview application scenario in the example of the present application is described. The present application embodiment does not impose any restrictions on the specific type of electronic device. The following takes the electronic device as a mobile phone as an example to describe the method for shooting in the present application embodiment.

[0090] Figure 1A : is an example diagram of an application scenario of a photo preview in a high-illuminance environment in an embodiment of the present application. Figure 1A In the mobile phone interface shown in (1), the interface can display multiple applications: application 1, application 2, ..., application 7 and camera application. In response to the user clicking the camera application, the mobile phone starts the camera application. After the camera application is running, the mobile phone interface displays the following Figure 1A The interface shown in (2). Figure 1A The interface shown in (2) can be called the camera's photo preview interface. The photo preview interface may include a viewfinder 11, a light source 17, a zoom ratio 12 (default is 1x), an album icon 13, a shooting control 14, and a camera rotation control. The brightness of the light source 17 can determine whether the current shooting environment is an illumination environment. For example, when the brightness of the light source 17 is high, so that the ambient illumination of the current shooting environment is greater than a preset brightness threshold, it can be determined that the current scene is a high illumination scene (or a highlight scene).

[0091] The mobile phone can take a photo in response to the user clicking the shooting control 14. The album icon 13 displays thumbnails of the photos. The camera rotation control can be used to switch cameras. Among them, the viewfinder 11 is used to obtain the image of the shooting preview, and the preview image can be displayed in real time.

[0092] In the photo preview scenario, the phone can support digital zoom. When using the photo function, the user can select different zoom ratios by operating on the touch screen. As an example, Figure 1A As shown in (2), the user clicks Figure 1A The zoom ratio of (2) is 12, and the interface is displayed as follows Figure 1A In the interface shown in (3), a zoom ratio selection item 15 appears (for example, the highest zoom ratio is 8x and the lowest zoom ratio is 1x). The user drags the zoom ratio 12 upward in the selection item 15 and releases the drag when the zoom ratio is 2x. The interface displays as follows: Figure 1A As shown in (4), the zoom ratio is 2x. Figure 1A The display area of ​​the light source 17 shown in (4) in the viewfinder 11 also becomes larger accordingly. Of course, after the zoom ratio is selected, the zoom ratio selection item 15 can be hidden, that is, the selected zoom ratio displayed on the interface is 2x.

[0093] Of course, the above description is based on the example that the zoom ratio selected in the photo preview scene is 2x, and the embodiment of the present application is not limited thereto. For example, the user can also adjust the zoom ratio to more than 2x.

[0094] It should be understood that Figure 1A (2)-(4) show a schematic diagram of an interface for a user to take a photo in a vertical position of a mobile phone, but the present application is not limited thereto. For example, a user can take a photo in a horizontal position of a mobile phone.

[0095] In some other embodiments, in the video preview mode, for a shooting scene with a zoom ratio of 2x or more (for example, 3x) and a high-illumination environment, the sensor uses a full-size crop mode to output the image.

[0096] Here, cropping refers to cropping the acquired image to obtain a view angle corresponding to the target zoom factor. For the description of the high illumination environment, please refer to the previous description, and for the sake of brevity, it will not be repeated here.

[0097] For ease of understanding, the following combination Figure 1B An example diagram describing a high-light environment in a video preview scenario. Figure 1B In the photo preview interface shown in (1), the mobile phone displays the video preview interface in response to the user clicking the video control, for example, Figure 1B The video preview interface 181 shown in (2) .

[0098] like Figure 1BAs shown in (2), the video preview interface 181 includes an album icon 13, a video control 16, a light source 17, and a zoom factor 12 (default is 1x). The light source 17 indicates that the current video preview scene is a highlighted scene.

[0099] Similarly, in the video preview scenario, the mobile phone can also support digital zoom. When using the video function, the user can select different zoom ratios by operating on the touch screen. As an example, Figure 1B As shown in (2), the user clicks Figure 1B The zoom ratio of (2) is 12, and the interface is displayed as follows Figure 1B In the interface 182 shown in (3), a zoom ratio selection item 15 appears (for example, the highest zoom ratio is 8x and the lowest zoom ratio is 1x). In response to the user dragging the zoom ratio 12 upward in the selection item 15 and releasing the drag button when the zoom ratio is 2x, the mobile phone displays the following: Figure 1B The interface 183 shown in (4) in the figure, i.e., the zoom ratio is selected to be 2x. Of course, after the zoom ratio is selected, the zoom ratio selection item 15 can be hidden, i.e., the selected zoom ratio displayed on the interface is 2x.

[0100] Of course, the above description is based on the example that the zoom ratio selected in the video preview scene is 2x, and the embodiment of the present application is not limited thereto. For example, the user can also adjust the zoom ratio to more than 2x.

[0101] It should be understood that Figure 1B The interface shown in (2)-(4) may be an example interface of an application scenario of an embodiment of the present application.

[0102] In other embodiments, in the video preview mode, for a shooting scene with a zoom ratio of 2x or more (eg, 3x) and a high dynamic range, the sensor uses a full size crop and staggered mode to output the image.

[0103] For the relevant description of high dynamic range, please refer to the previous description, which will not be repeated here for the sake of brevity.

[0104] The embodiments of the present application are applicable to the scenario where the sensor uses a quad bayer coding (QBC) array to output images, or in other words, to a quad sensor. The pixel arrangement of the image data output by the quad sensor is different from the pixel arrangement of the image data output by the traditional sensor. The image data output by the quad sensor is composed of 4 pixels (which can be respectively recorded as RGGB), and each pixel is composed of 4 sub-pixels. Figure 2A Describes the pixel arrangement of image data output by the Quad sensor.

[0105] like Figure 2A As shown, the image data output by the Quad sensor contains 4 color pixels, namely R, Gr, Gb, and B. Each color pixel is composed of 4 sub-pixels, namely: R0, R1, R2, and R3; Gr0, Gr1, Gr2, and Gr3; Gb0, Gb1, Gb2, and Gb3; B0, B1, B2, and B3. For each color pixel's sub-pixel (or adjacent 2*2 sub-pixels of the same color), for example, R0, R1, R2, and R3, there is a sensitivity difference between the sub-pixels of the same color, so Quad bayer coding sensitivity correction (QSC) calibration is required. The sensitivity difference will eventually lead to a difference in brightness.

[0106] This is because if QSC correction (or QSC calibration) is not performed, the image data output by the Quad sensor will have uneven brightness, resulting in poor output image quality. Among them, poor image quality can be manifested as: stripes appearing in the image, such as crosstalk stripes or other forms of stripes.

[0107] Figure 2B A comparison chart of the sensitivity difference of pixel points before and after QSC correction is shown. Figure 2B The vertical axis represents the sensitivity values ​​corresponding to different color pixels (or sub-pixels). Figure 2B As shown, for sub-pixels R0, R1, R2 and R3, the sensitivity difference is large before QSC correction is performed, while after QSC correction is performed, the sensitivity difference between sub-pixels R0, R1, R2 and R3 is reduced.

[0108] In the above four shooting scenarios, specifically including: (1) in the photo preview mode, the zoom ratio is 2x or more (for example, 3x) and the shooting scene is in a high illumination environment; (2) in the photo preview mode, the zoom ratio is 2x or more (for example, 3x) and the shooting scene is in a high dynamic range; (3) in the video preview mode, the zoom ratio is 2x or more (for example, 3x) and the shooting scene is in a high illumination environment; (4) in the video preview mode, the zoom ratio is 2x or more (for example, 3x) and the shooting scene is in a high dynamic range, the sensor uses full size crop for image output. In the above four shooting scenarios, if the aperture of the electronic device changes, the image data needs to be calibrated using calibration data to ensure the image effect. Calibration data refers to data used to compensate the collected image data.

[0109] At present, when the module of an electronic device is calibrated at the factory, the QSC calibration data will be burned into the memory of the electronic device, such as EEPROM. In this way, before the electronic device sensor starts streaming, or when the camera is initialized, the calibration data can be called to correct the image data after the sensor starts streaming. The sensor starts streaming can be understood as the sensor starting to output image data, or transmitting data stream. However, this method cannot support scenarios where the aperture changes, or does not support sensors with variable aperture characteristics. That is, when the electronic device switches from one aperture to another, the calibration data of the aperture before the switch is still used, which will result in poor correction effect, thereby affecting the image quality.

[0110] In view of this, an embodiment of the present application proposes a method for shooting. In a scenario where the sensor adopts a quadruple Bayer array QBC, it supports sending calibration data after the aperture changes or after the sensor starts to flow, that is, it can dynamically update the QSC calibration data. The image data compensated by the calibration data can effectively reduce the brightness difference between pixels, thereby reducing the stripes in the image caused by the brightness difference and improving the image effect.

[0111] The following first combines Figure 3 Describe the software system used in the embodiments of the present application.

[0112] Figure 3 It is a schematic diagram of the architecture (including software system and some hardware) of the embodiment of the present application. Figure 3 As shown, the application architecture is divided into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the application architecture can be divided into five layers, from top to bottom, namely, the application layer (Application Layer), the application framework layer (Application Framework Layer), the hardware abstraction layer (Hardware Abstraction Layer), the kernel layer (Kernel Layer) and the hardware layer (Hardware Layer).

[0113] like Figure 3 As shown, the application layer includes the camera and the gallery. It can be understood that Figure 3 The application layer shows some applications. In fact, the application layer can also include other applications (including system applications and / or third-party applications), which is not limited in this application. For example, the application layer also includes information, alarm clock, weather, stopwatch, compass, timer, flashlight, calendar, Alipay and other applications.

[0114] like Figure 3As shown, the application framework layer includes a camera access interface. For example, the camera access interface includes camera management and camera devices. The hardware abstraction layer includes an automatic exposure (AE) module, a variable aperture HAL module, and a sensor service processing module (for example, a sensor node HAL module). The AE module is used to automatically adjust the sensor exposure time to adjust the image brightness when the external ambient light conditions change.

[0115] Understandably, Figure 3 The hardware abstraction layer shown in the figure may also include other modules, but the embodiments of the present application are not limited thereto. For example, optionally, the hardware abstraction layer may also include a camera algorithm library. The camera algorithm library includes an image processing algorithm module.

[0116] The kernel layer is used to drive hardware resources. The kernel layer can include multiple driver modules. Figure 3 As shown, the core layer includes a variable aperture driver and an image sensor driver. Optionally, the core layer may also include a digital signal processor driver and a graphics processor driver.

[0117] The hardware layer includes various hardware resources. Figure 3 As shown, the hardware layer includes hardware related to the camera module, such as an image sensor and a variable aperture component.

[0118] Optionally, the camera module further includes an electrically EPROM (EEPROM) or other storage resources. The EEPROM can store QSC calibration data corresponding to the aperture.

[0119] Optionally, the hardware layer also includes other sensors (such as an ambient light sensor), an image signal processor, a digital signal processor, and a graphics processor.

[0120] It should be understood that Figure 3 The software architecture shown is only an example description, and the embodiments of the present application are not limited thereto. Figure 3 The layers in the can include more other modules.

[0121] The following combination Figure 4 Brief description of application Figure 3 The method timing interaction process when implementing software architecture. Figure 4 The sensor node HAL module and the variable aperture HAL module shown in FIG. Figure 3 The HAL layer in . Figure 4 The variable aperture drive and image sensor drive shown in FIG. Figure 3 The kernel layer in . Figure 4The kernel layer and the hardware abstraction layer in the system communicate through the middle layer (or interface, also referred to as the middle layer). Figure 4 As shown, including but not limited to the following steps:

[0122] Step 1: The sensor node HAL module sends an execution process request (ExecuteProcessRequest) to the variable aperture HAL module.

[0123] Among them, the execution process request may correspond to each image frame, or in other words, when each image frame is output, the sensornode HAL module will issue an execution process request.

[0124] Optionally, the request to execute the process includes an aperture parameter, for example, the aperture parameter is an aperture value (ie, VA code).

[0125] Step 2: The variable aperture HAL module obtains aperture parameters. The aperture parameters are used to implement aperture switching.

[0126] After receiving the execution process request, the variable aperture module obtains the aperture parameters from the execution process request. Exemplarily, the variable aperture HAL module calls the aperture parameter method function, and converts the aperture parameters into variable aperture parameter values ​​through the aperture parameter method function. For example, the aperture parameter method function is the GetVAparams method function, and the aperture value (ie, VA code) is obtained from the AE tag through the GetVAparams method function. Among them, the AE tag can be understood as the parameter issued by the AE module; the GetVAparams method function is a function for obtaining the aperture parameters. In other words, the aperture value can be obtained from the AE tag by calling the GetVAparams method function.

[0127] Step 3: The variable aperture HAL module sends the aperture parameters to the middle layer.

[0128] Step 4: The middle layer sends the aperture parameters to the variable aperture driver.

[0129] Step 5, the variable aperture driver applies the aperture parameters.

[0130] Exemplarily, the variable aperture driver applies the aperture parameter to perform aperture switching, switching from the current aperture to a first aperture corresponding to the aperture parameter.

[0131] Of course, after the variable aperture driver applies the aperture parameters, the hardware of the corresponding aperture (for example, the first aperture) will be driven to work.

[0132] Step 6-1, the variable aperture driver returns the application result of the aperture parameter to the middle layer.

[0133] In step 6-2, the middle layer passes the application result of the aperture parameter to the variable aperture HAL module.

[0134] Step 6-3, the variable aperture HAL module passes the application result of the aperture parameter to the sensor node HAL module.

[0135] Step 7: The sensor node HAL module sends the QSC calibration data to the middle layer.

[0136] Specifically, after determining that the aperture parameters have changed, the sensor node HAL module will issue a command including QSC calibration data, which corresponds to the switched aperture (for example, the QSC calibration data corresponding to the first aperture) so as to update the QSC calibration data in time to adapt to the switched aperture.

[0137] Step 8: The middle layer passes the QSC calibration data to the image sensor driver.

[0138] Step 9: The image sensor driver applies the QSC calibration data, or the QSC calibration data takes effect.

[0139] Optionally, the QSC calibration data is a sequence or matrix of brightness offset values. Taking the QSC calibration data as a matrix as an example, each element in the matrix corresponds to a pixel (specifically, it can correspond to a sub-pixel of a pixel), and the value of the element in the matrix is ​​the calibration data value corresponding to the pixel (or sub-pixel).

[0140] For each sub-pixel of a pixel, its pixel value can be compensated by using the corresponding calibration data value. Take the sub-pixel of one pixel as an example. Assuming that the pixel value of a sub-pixel of a pixel is 100, by superimposing the corresponding calibration data value on the pixel value (for example, the calibration data value corresponding to the sub-pixel of the pixel is -20), the compensated pixel value is 80. For the pixel value of each pixel, the corresponding calibration data value is superimposed to achieve the purpose of reducing the brightness difference.

[0141] It should be understood that the embodiments of the present application are described only by naming QSC calibration data (such as first calibration data and second calibration data), and the embodiments of the present application are not limited thereto. In fact, the calibration data may also have other names or titles, such as compensation data, correction data, and correction data. However, no matter how the naming is performed, the essence of the term remains unchanged, that is, the relevant explanation of the term can refer to the above description of the calibration data.

[0142] Step 10-1, the image sensor driver returns the application result of the QSC calibration data to the middle layer.

[0143] Step 10-2: The middle layer passes the application results of the QSC calibration data to the sensor node HAL module.

[0144] for Figure 4 In the process, the above steps 3 to 6-3 can be understood as the process of changing the aperture, or the process of switching the aperture. The above steps 7 to 10-2 can be understood as the process of sending QSC calibration data.

[0145] In some embodiments, steps 3 to 6-3 occur in one image frame, and steps 7 to 10-2 occur in the next image frame of the image frame. For example, steps 3 to 6-3 occur in the N+1th frame, and steps 7 to 10-2 occur in the N+2th frame.

[0146] To facilitate understanding of the timing of sending calibration data in the embodiment of the present application, the following is combined with Figure 5 The timing in is described. Figure 5 As shown, Figure 5 What is shown in is the time axis after the sensor starts streaming. T1 to T14 can be understood as the corresponding partial timestamps after the sensor starts streaming. Taking the N-1th frame as an example, the timestamp corresponding to the start-of-frame delimiter (SOF) of the N-1th frame is T1, and the timestamp corresponding to the end-of-frame delimiter (EOF) of the N-1th frame is T2, and the timestamps corresponding to other image frames are similar. The time interval between the EOF of the N-1th frame and the SOF of the Nth frame can be called Vblank, that is, the duration between T2 and T4. The frame interval between the N-1th frame and the Nth frame is the time interval between T1 and T4.

[0147] In the embodiment of the present application, the premise for sending the QSC calibration data is that the aperture changes and the sensor starts to flow. That is to say, the sending of the QSC calibration data in the embodiment of the present application includes two stages, the first stage is the stage of adjusting the aperture, and the second stage is the stage of sending the QSC calibration data. Optionally, the second stage may occur after the first stage. Or, optionally, the second stage is carried out synchronously during the process of the first stage. Generally speaking, the aperture has been switched before the QSC calibration data is sent. In short, after these two stages are completed, the image frames output by the sensor match the latest QSC calibration data and the latest aperture effect. The detailed timing of these two stages is introduced below.

[0148] Phase 1

[0149] The sensor receives the exposure parameters and aperture parameters (used to switch the aperture to the first aperture) sent by the AE module before the timestamp corresponding to the SOF of the Nth frame, that is, before T4. After obtaining the exposure parameters and aperture parameters, the sensor starts to set the exposure parameters at the SOF of the Nth frame; and the sensor starts to adjust the aperture from the EOF of the N+1th frame (that is, T8), or it can be understood as switching from the current aperture to the first aperture.

[0150] Among them, the AE module sends the exposure parameters and aperture parameters to the sensor based on the decision or the instruction sent by the camera application, and sends the aperture parameters to the aperture.

[0151] The present application embodiment does not limit the object or time to trigger the aperture change, and the electronic device may automatically switch the aperture in combination with the shooting mode or shooting scene, or the user may actively trigger the aperture switch. For example, when the user uses the professional shooting mode to shoot, the aperture is actively changed, triggering the aperture adjustment instruction.

[0152] Optionally, in the scenario where the user triggers the aperture switching, the timestamp corresponding to the user triggering the aperture switching may be: Figure 5 The time stamp may be T2-1, or a time stamp earlier than T2-1, without any specific limitation.

[0153] Optionally, the corresponding timestamp when the AE module decides the exposure parameter and the aperture parameter may be T2-2.

[0154] After receiving the above exposure parameters, from the SOF of the N+2th frame before and the SOF of the N+1th frame (for example, Figure 5 Starting from a timestamp after T7 in the figure, the sensor performs exposure based on the exposure parameters. For example, the sensor performs exposure based on the exposure parameters from T9 to T11, and outputs the exposure result at the N+2th frame; however, since the N+2th frame will be discarded in the embodiment of the present application, it can be considered that the exposure result is output at the next frame after the N+2th frame (that is, the N+3th frame). For the effectiveness mechanism of the exposure parameters, please refer to the relevant technical description, which will not be repeated here.

[0155] Since the aperture adjustment starts at the EOF (i.e., T8) of the N+1th frame, and it takes a certain amount of time to adjust the aperture, the process of adjusting the aperture will affect the image brightness of the N+2th frame. Therefore, it is necessary to discard (or skip) the image data corresponding to the N+2th frame, that is, the image corresponding to the N+2th frame is not displayed.

[0156] Optionally, the image corresponding to the N+1th frame can be displayed during the display duration of the N+2th frame. That is, since the N+2th frame is the frame output by the sensor when the aperture changes, its image brightness will be abnormal, or its exposure will be affected. In order to avoid presenting an image frame with abnormal brightness to the user, the last normal image frame before adjusting the aperture (i.e., the N+1th frame) can be displayed.

[0157] Phase II

[0158] The sensor starts writing the QSC calibration data corresponding to the first aperture at the SOF of the N+2th frame. For example, the sensor starts writing the QSC calibration data corresponding to the first aperture at the timestamp T10.

[0159] It should be noted that in order to minimize the impact of writing QSC calibration data on the image effect, it is possible to consider writing the QSC calibration data corresponding to the first aperture during the frame time to avoid affecting the image effect. The frame time here can specifically be the interval between the SOF of the N+2th frame and the SOF of the N+3th frame.

[0160] In addition, since the QSC function (or correction function) is turned off when writing the calibration data corresponding to the first aperture, the latest written QSC calibration data has not yet taken effect. From this perspective, the image quality of the N+2th frame will also be affected, so the N+2th frame needs to be discarded. As mentioned above, the image corresponding to the N+1th frame can be used for display to ensure that the preview image presented to the user is an image frame with appropriate brightness, which will not affect the display image effect.

[0161] Since writing the QSC calibration data corresponding to the first aperture will take a certain amount of time, in order to reduce the time required to write the QSC calibration data, the embodiment of the present application pre-processes the QSC calibration data to compress the number of bits corresponding to the QSC calibration data.

[0162] In some embodiments, the QSC calibration data corresponding to the first aperture is calibration data obtained by preprocessing original QSC calibration data (or second calibration data, or calibration data before preprocessing).

[0163] The embodiments of the present application do not limit the specific method of preprocessing. In some embodiments, the application scenarios applicable to the embodiments of the present application are related to the zoom ratio (for example, applied to a zoom ratio of 2x or more). Since for scenes with a zoom ratio of 2x or more, the sensor will crop the image data accordingly in combination with the zoom ratio when outputting the image, in order to minimize the impact of the QSC calibration data sent on the image frame, the QSC calibration data is preprocessed and only the QSC calibration data corresponding to the cropped image is sent. This ensures the data compensation effect while reducing the time spent on sending the QSC calibration data.

[0164] For ease of understanding, combined Figure 6 The process of image cropping when the sensor outputs an image is shown in FIG. Figure 6 As shown in the figure, the size of the cropped image is obviously smaller than the size of the full-size image. The image size corresponding to the QSC calibration data before preprocessing. For example, in full-size mode, the image size is 8192*6144, and after cropping, the image size of the cropped center area is 4096*3072. In this way, by preprocessing the QSC calibration data, only the QSC calibration data corresponding to the image of the cropped center area is sent, which reduces the time for transmitting the QSC calibration data.

[0165] Optionally, the number of bits corresponding to the QSC calibration data corresponding to the first aperture is smaller than the number of bits corresponding to the original QSC calibration data.

[0166] For example, the number of bits corresponding to the original QSC calibration data is 3536 bytes, and the corresponding transmission time is about 43 milliseconds. The number of bits corresponding to the first aperture QSC calibration data is 1008 bytes, and the corresponding transmission time is about 15 milliseconds.

[0167] When the number of bits corresponding to the QSC calibration data corresponding to the first aperture is reduced, the transmission time will be reduced accordingly. The writing process of the QSC calibration data corresponding to the first aperture occurs between the SOF of the N+2th frame and the SOF of the N+3th frame, and the image corresponding to the affected N+2th frame is discarded.

[0168] Optionally, the transmission duration of the QSC calibration data corresponding to the first aperture is less than a frame interval between the N+2th frame and the N+3th frame.

[0169] For example, the transmission duration of the QSC calibration data corresponding to the first aperture is 15 milliseconds, and the frame interval between the SOF of the N+2th frame and the SOF of the N+3th frame is 33 milliseconds. It can be seen that the transmission duration of the QSC calibration data corresponding to the first aperture is less than the duration of the frame interval, that is, the QSC calibration data corresponding to the first aperture can be sent within one image frame, so only one frame of data will be affected.

[0170] After the transmission time of the QSC calibration data corresponding to the first aperture is reduced, the number of affected image frames will also be reduced accordingly, that is, the affected image frames only include the N+2th frame, and will not affect a larger number of image frames, thereby obtaining a better shooting experience.

[0171] After the above-mentioned first and second stages, when the sensor outputs the N+3th frame, that is, starting from T13, the QSC calibration data corresponding to the first aperture and the effect corresponding to the first aperture can be effective. Moreover, in the N+3th frame, the exposure parameters match the aperture parameters. Since the aperture has been switched to the first aperture, and the latest QSC calibration data corresponding to the first aperture has also been effective, the image effect corresponding to the N+3th frame obtained is better than the image effect obtained by using the QSC calibration data corresponding to the first aperture after the switch, compared with the image effect obtained by using the QSC calibration data corresponding to the original aperture (i.e., the aperture before the switch).

[0172] It should be noted that the timing of writing the QSC calibration data in the embodiment of the present application occurs after the sensor starts to flow. Figure 5 As shown, compared with the timing of writing QSC calibration data before the flow in the related art (or although the camera is turned on, the sensor has not yet produced an image), the QSC calibration data written in the embodiment of the present application occurs after the sensor starts to flow, specifically after the aperture changes. The advantage of this is that QSC calibration data that matches the switched aperture can be sent down, or the QSC calibration data can be dynamically updated based on the change of the aperture, so that the QSC calibration data corresponds to the switched aperture, thereby achieving better correction or compensation and obtaining better image effects.

[0173] The embodiment of the present application does not specifically limit the storage space where the QSC calibration data (for example, the QSC calibration data corresponding to the first aperture) is written. Taking the QSC calibration data corresponding to the first aperture as an example, the storage space where the QSC calibration data corresponding to the first aperture is located can have the following different implementation methods.

[0174] In one implementation, the QSC calibration data corresponding to the first aperture is stored in the OTP.

[0175] Exemplarily, if the QSC calibration data corresponding to the first aperture needs to be written, the QSC calibration data corresponding to the first aperture may be written into the sensor OTP.

[0176] In another implementation, the QSC calibration data corresponding to the first aperture is stored in a non-volatile memory EEPROM.

[0177] For example, if the QSC calibration data corresponding to the first aperture needs to be written, the QSC calibration data corresponding to the first aperture can be written in the EEPROM, and then the QSC calibration data written in the first aperture can be transferred to the sensor to take effect. The advantage of doing this is that when the OTP space reserved for the sensor is relatively small, or even when no OTP space is reserved, writing the calibration data into the EEPROM can be an option. In addition, since the sensor OTP itself has a relatively small storage space and is easily damaged, in this case, in order to improve reliability, the storage of the QSC calibration data corresponding to the first aperture can be achieved by attaching an EEPROM to the camera module.

[0178] In another implementation manner, the QSC calibration data corresponding to the first aperture is stored in a system configuration file oeminfo.

[0179] Exemplarily, when the QSC calibration data corresponding to the first aperture needs to be written, the QSC calibration data corresponding to the first aperture can be stored in oeminfo. Therefore, when the sensor does not reserve OTP space and does not set EEPROM, the first calibration data can also be stored in oeminfo.

[0180] The following combination Figure 7 The process shown in the figure describes the method process for shooting in the embodiment of the present application. Figure 7 As shown, the method includes:

[0181] Step 701, at a first moment, start the camera sensor.

[0182] The first moment can be understood as the moment when the user triggers the camera. Figure 5 The moment when the sensor starts to flow.

[0183] The embodiment of the present application does not specifically limit how to start the camera sensor. Optionally, in response to a first operation of the user, the camera sensor is started at a first moment. After the electronic device detects the operation of turning on the camera, the camera can be started to collect image data.

[0184] The embodiment of the present application does not limit the way in which the user triggers the camera. The first operation is an operation for triggering the camera of the electronic device to turn on. The first operation includes but is not limited to: touch operation, key operation, voice control, etc. The embodiment of the present application does not specifically limit the specific form of the first operation.

[0185] Optionally, the first operation is an operation of opening a camera application. Figure 1A As shown in (1), the first operation is an operation in which a user clicks on a camera application to start the camera.

[0186] Step 702, at a second moment, obtaining exposure parameters and aperture parameters of the Nth frame; wherein the first moment is before the second moment.

[0187] Optionally, the aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture.

[0188] In some embodiments, the exposure parameters and aperture parameters of the Nth frame are determined by an automatic exposure module, and the exposure parameters and the aperture parameters are cached in the camera driver.

[0189] The embodiment of the present application does not limit the order of acquiring the exposure parameters and the aperture parameters. In other words, the embodiment of the present application does not limit whether the timing of acquiring the exposure parameters and the aperture parameters is the same timing.

[0190] At the second moment, obtaining the exposure parameters and aperture parameters of the Nth frame may include the following three situations: (1) obtaining the exposure parameters and aperture parameters at the same time after the parameters are adjusted; (2) obtaining the exposure parameters first, and then obtaining the aperture parameters corresponding to the exposure parameters; (3) obtaining the aperture parameters first, and then obtaining the exposure parameters corresponding to the aperture parameters.

[0191] The embodiments of the present application do not limit the triggering operation of acquiring the exposure parameters and the aperture parameters. Optionally, in some embodiments, acquiring the exposure parameters and the aperture parameters of the Nth frame includes: acquiring the exposure parameters and the aperture parameters in response to the second operation. That is, the exposure parameters and the aperture parameters may be acquired in response to the second operation.

[0192] The second operation can be understood as an operation that triggers the switching of the aperture of the electronic device. The embodiment of the present application does not limit the specific form of the second operation.

[0193] Optionally, the second operation is an operation of manually adjusting the aperture by the user. That is, at the second moment, the electronic device can obtain the exposure parameter and the aperture parameter in response to the operation of manually adjusting the aperture by the user.

[0194] Exemplarily, the second operation is an operation of manually adjusting the aperture by the user in the professional photography mode. Fig. 8AFIG. 4 shows an example of an interface for a user to adjust the aperture in professional mode. Fig. 8A As shown in (1), taking the photo preview interface 80 as an example, the interface 80 includes a professional mode. In response to the user clicking the professional mode, the interface displays Fig. 8A Interface 81 shown in (2).

[0195] Fig. 8A The interface 81 shown in (2) is a preview interface in the professional mode. In the professional mode, the relevant parameters of the camera settings can be manually adjusted by the user. Fig. 8A As shown in (2), the display area 801 in the camera interface 81 includes a plurality of camera parameter setting options. The display area 801 includes at least an aperture adjustment control 802, or an icon "A". The icon "A" can also be understood as an aperture setting button.

[0196] In response to the user clicking the aperture adjustment control 802, the mobile phone displays Fig. 8A The interface 82 shown in (3) in FIG. 8 includes at least a display area 803. The display area 803 displays controls for adjusting the aperture. In response to the user's operation of adjusting the aperture in area 803 (for example, sliding to the left or right), the mobile phone switches to the corresponding aperture. Fig. 8A In the interface shown, the second operation is an operation of the user manually adjusting the aperture in area 803 in the interface 81 .

[0197] Understandably, Fig. 8A The aperture adjustment interface shown in (3) is only an example description, and the embodiments of the present application are not limited thereto. For example, the aperture adjustment interface in the display area 803 may also be in the form of a dial, and accordingly, the user's operation of adjusting the aperture size may be an operation of adjusting the dial.

[0198] It can also be understood that other camera parameters in the professional mode are also shown in the display area 801. Optionally, Fig. 8A The area 801 shown in (2) also includes the following camera parameters: metering mode (corresponding to the icon "M"), ISO parameter (corresponding to the icon ISO, for example, the ISO value in the figure is 100), shutter speed (corresponding to the icon "S", for example, the shutter speed in the figure is 1 / 40), exposure compensation (corresponding to the icon "EV."), focus mode (corresponding to the icon "AF."), white balance parameter (corresponding to the icon "WB."). For the explanation of the camera parameters, please refer to the description in the relevant technology, which will not be repeated here.

[0199] Exemplarily, the second operation is an operation performed by the user after switching the current photographing mode to a large aperture photographing mode. Figure 8B Figure 1 shows an example of an interface for a user to switch from a normal camera mode to a large aperture mode. Figure 8B The interface 83 shown in (1) is a photo preview interface. In response to the user sliding right in the photo mode area of ​​the interface 83, the mobile phone displays Figure 8B Interface 84 shown in (2) in the figure. Interface 84 presents more options for photo modes, including at least a large aperture mode. In response to the user clicking on the large aperture mode, the mobile phone enters the large aperture shooting mode and adjusts the aperture accordingly. Figure 8B In the interface shown, the second operation is the user clicking on the large aperture mode.

[0200] It should be understood that the entrance to the professional mode or large aperture mode shown above is merely an example description, and the embodiments of the present application are not limited thereto.

[0201] based on Fig. 8A or Figure 8B As shown in the interface, the mobile phone can respond to the user's manual adjustment of the aperture and execute aperture control, so that the aperture can be adjusted based on user needs to assist the user in taking photos with better image quality.

[0202] Or, optionally, the second operation is an operation of manually adjusting the aperture of the electronic device.

[0203] At the second moment, the electronic device can obtain exposure parameters and aperture parameters based on the operation of automatically adjusting the aperture. For example, the electronic device is equipped with a variable aperture; after detecting a specific shooting scene, the electronic device will automatically adjust the aperture according to the light conditions of the shooting scene, and the electronic device will obtain exposure parameters and aperture parameters.

[0204] Step 703: at a third moment, the exposure parameter is sent to the camera sensor; the third moment is a moment before the moment corresponding to the start of frame delimiter SOF of the Nth frame; the second moment is before the third moment.

[0205] It should be noted that after the exposure parameters are sent to the camera sensor, the exposure parameters can be obtained from the SOF of the N+2th frame before the SOF of the N+1th frame (for example, Figure 5 For example, the exposure parameters are effective from a time stamp after T7 in Figure 5 The exposure parameters shown in FIG. 1 are effective from T9, that is, exposure is performed based on the exposure parameters from T9; for example, the exposure parameters are Figure 5It should be understood that the moment when the exposure parameters shown here take effect is only an exemplary description, and the embodiments of the present application are not limited thereto. In addition, the process of writing the calibration data will affect the N+2th frame, and the N+2th frame will be discarded (for example, the image of the N+1th frame is displayed during the display time of the N+2th frame), so the brightness of the N+3th frame is consistent with the brightness of the Nth frame and the brightness of the N+1th frame, and the exposure parameters match the aperture parameters in the N+3th frame. The meaning of the exposure parameters matching the aperture parameters means that the aperture has been successfully switched in the N+3th frame, the QSC calibration data corresponding to the aperture issued also matches the aperture, and the exposure parameters have also taken effect. Of course, after the aperture is successfully switched, the subsequent AE module can issue the corresponding exposure parameters in combination with the actual situation, for example, the exposure parameters corresponding to the switched aperture are issued in the N+3th frame. For the issuance mechanism and effectiveness mechanism of the exposure parameters, reference can be made to the description of the relevant technology, and for the sake of brevity, they will not be repeated here.

[0206] In some embodiments, after time T2-2 and before time T4, the camera sensor HAL obtains the aperture parameter and exposure parameter input by the AE module. The camera sensor HAL sends the exposure parameter to the sensor driver. The sensor driver can send the exposure parameter to the hardware (ie, the camera sensor) for subsequent effectiveness.

[0207] In some embodiments, the camera sensor HAL sends the aperture parameters to the aperture HAL. After acquiring the aperture parameters, the aperture HAL may send the aperture parameters to the aperture motor driver. The aperture motor driver may send the aperture parameters to the hardware (aperture component) for validation.

[0208] Exemplarily, the aperture HAL calls the aperture parameter method function, and converts the aperture parameter into a variable aperture parameter value (VA code) through the aperture parameter method function.

[0209] Step 704: At a fourth moment, the aperture component adjusts the aperture based on the aperture parameter. The fourth moment is the moment corresponding to the end-of-frame delimiter EOF of the N+1th frame.

[0210] Exemplarily, the fourth moment is Figure 5 That is, starting from the EOF of the N+1th frame, the aperture is adjusted based on the aperture parameter, that is, the current aperture is switched to the first aperture indicated by the aperture parameter according to the aperture parameter.

[0211] Understandably, adjusting the aperture also takes a certain amount of time, for example, Figure 5 The time required to adjust the aperture is shown in FIG. Adjusting the aperture affects the N+2 frame and does not affect the N+3 frame. The N+2 frame can also be called an aperture change frame.

[0212] Step 705: at a fifth moment, the first calibration data is sent to the camera sensor; the fifth moment is the moment corresponding to the SOF of the N+2th frame.

[0213] Optionally, the first calibration data is calibration data corresponding to the first aperture. Accordingly, the aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture. That is, when the electronic device switches from the current aperture to the first aperture, the first calibration data sent is calibration data corresponding to the switched first aperture, i.e., the first calibration data.

[0214] In which, within the display duration corresponding to the N+3th frame, the image corresponding to the N+3th frame is displayed based on the exposure parameter, the aperture parameter and the first calibration data. That is to say, after the calibration data is sent down, the exposure parameter and the aperture parameter of the image of the N+3th frame sent for display match each other. For related descriptions, please refer to the previous text Figure 5 The description of the above will not be repeated here.

[0215] In some embodiments, when the camera sensor of the electronic device is a Quad sensor, the first calibration data is QSC calibration data corresponding to the first aperture.

[0216] Exemplarily, the fifth moment is Figure 5 That is to say, starting from the SOF of the N+2th frame, the QSC calibration data corresponding to the first aperture is sent to the sensor.

[0217] In some embodiments, the camera sensor HAL sends the QSC calibration data corresponding to the first aperture to the sensor driver. The sensor driver sends the QSC calibration data corresponding to the first aperture to the hardware (ie, the camera sensor) for subsequent effectiveness.

[0218] In an embodiment of the present application, the electronic device supports switching the calibration data corresponding to the aperture after the sensor starts to flow. That is to say, when the aperture changes, the electronic device can switch the calibration data corresponding to the aperture accordingly, so as to perform image data compensation based on appropriate calibration data and improve image quality. Compared with the method of only burning a set of calibration data before the sensor starts to flow, the embodiment of the present application can also realize the switching of calibration data after the sensor starts to flow, thereby improving the quality of the output image.

[0219] Further, optionally, the first calibration data is calibration data obtained by preprocessing the second calibration data, wherein the number of bits corresponding to the second calibration data is greater than the number of bits corresponding to the first calibration data. As described above, by preprocessing the calibration data to reduce the number of bits corresponding to the calibration data, the transmission time when the calibration data is sent is reduced, thereby minimizing the number of affected image frames.

[0220] For example, after receiving the exposure parameters and aperture parameters before the SOF of the Nth frame, the electronic device starts adjusting the aperture based on the aperture parameters at the EOF of the N+1th frame, and sends the QSC calibration data of the adjusted aperture at the SOF of the N+2th frame, so that the exposure parameters match the aperture parameters at the N+3th frame. In this process, only the N+2th frame is affected, and no more image frames are affected.

[0221] Optionally, the transmission duration of the first calibration data is less than the frame interval between the N+2th frame and the N+3th frame. As mentioned above, the process of writing the first calibration data occurs between the SOF of the N+2th frame and the SOF of the N+3th frame to avoid affecting more image frames.

[0222] Optionally, within the display duration corresponding to the N+3th frame, the image corresponding to the N+3th frame is displayed based on the exposure parameter and the aperture parameter and the first calibration data. As described above, starting from the N+3th frame, the exposure parameter and the aperture parameter match, and the image data can be compensated accordingly based on the first calibration data, so within the display duration corresponding to the N+3th frame, the image corresponding to the N+3th frame can be displayed.

[0223] Optionally, the image corresponding to the N+1th frame is displayed within the display duration of the N+2th frame. That is, the camera sensor discards the N+2th frame when outputting the image. As mentioned above, the N+2th frame is affected during the process of writing the first calibration data, so the image of the previous frame (for example, the image corresponding to the N+1th frame) can be displayed when sending the image for display to avoid affecting the user experience.

[0224] The embodiments of the present application are applied to preview scenes (including photo preview and video preview) with a zoom ratio greater than a preset zoom ratio (for example, 2x) in a high dynamic environment or a high illumination environment. Figure 1A The scenario shown in (4) is shown in Figure 1. For example, Figure 1B The scene shown in (4).

[0225] It should be understood that Figure 1A The scene shown in or Figure 1B The scenario shown in is only an exemplary description, and the embodiments of the present application are not limited to this.

[0226] Combination of the above Figures 1A to 7 , describes in detail the method for shooting provided by the embodiment of the present application. Fig. 9 and Fig.10It should be understood that the apparatus for photographing in the embodiments of the present application can execute the method for photographing in the embodiments of the present application, that is, the specific working process of the following various products can refer to the corresponding process in the aforementioned method embodiment.

[0227] Fig. 9 A structural schematic diagram of an electronic device 1000 suitable for the present application is shown.

[0228] The electronic device 1000 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0229] Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0230] It should be noted that Fig. 9 The structure shown does not constitute a specific limitation on the electronic device 1000. In other embodiments of the present application, the electronic device 1000 may include Fig. 9 The electronic device 1000 may include more or fewer components than those shown. Fig. 9 A combination of some of the components shown, or the electronic device 1000 may include Fig. 9 Subassemblies of some of the components shown. Fig. 9 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0231] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). Different processing units may be independent devices or integrated devices.

[0232] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0233] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0234] In some embodiments, the processor 110 may include one or more interfaces. For example, the processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and a USB interface.

[0235] Fig. 9The connection relationship between the modules shown is only a schematic illustration and does not constitute a limitation on the connection relationship between the modules of the electronic device 1000. Optionally, the modules of the electronic device 1000 may also adopt a combination of multiple connection modes in the above embodiments.

[0236] The charging management module 140 is used to receive power from the charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive current from the wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive electromagnetic waves through the wireless charging coil of the electronic device 1000 (the current path is shown as a dotted line). While the charging management module 140 is charging the battery 142, it can also power the electronic device 1000 through the power management module 141.

[0237] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, and battery health status (e.g., leakage, impedance). Optionally, the power management module 141 can be set in the processor 110, or the power management module 141 and the charging management module 140 can be set in the same device.

[0238] The wireless communication function of the electronic device 1000 can be implemented through components such as the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

[0239] The electronic device 1000 can realize the display function through the GPU, the display screen 194 and the application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0240] The display screen 194 can be used to display images or videos. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a micro OLED (Micro OLED) or a quantum dot light emitting diode (QLED). In some embodiments, the electronic device 1000 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0241] The electronic device 1000 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

[0242] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can perform algorithm optimization on the noise, brightness and color of the image. The ISP can also optimize the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0243] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard red green blue (RGB), YUV or other format. In some embodiments, the electronic device 1000 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0244] In some embodiments, the electronic device 1000 includes a plurality of apertures, and the plurality of apertures includes at least a first aperture. Optionally, the camera 193 of the electronic device 1000 includes the plurality of apertures.

[0245] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 1000 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0246] Video codecs are used to compress or decompress digital videos. The electronic device 1000 may support one or more video codecs. Thus, the electronic device 1000 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0247] The electronic device 1000 can implement audio functions, such as music playing and recording, through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0248] The ambient light sensor 180L is used to sense the brightness of the ambient light. The electronic device 1000 can adaptively adjust the brightness of the display screen 194 according to the perceived brightness of the ambient light. The ambient light sensor 180L can also be used to automatically adjust the white balance during shooting.

[0249] In some embodiments, the ambient light sensor 180L can obtain the ambient illuminance of the current shooting environment. When the ambient illuminance is greater than or equal to the first illuminance threshold, it can be considered that the current shooting environment is relatively bright, and it can be considered that the current shooting environment is a high-brightness shooting environment.

[0250] The touch sensor 180K is also called a touch control device. The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, which is also called a touch control screen. The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor 180K can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 1000 and set at a different position from the display screen 194.

[0251] The key 190 includes a power key and a volume key. The key 190 can be a mechanical key or a touch key. The electronic device 1000 can receive a key input signal and implement a function related to the case input signal.

[0252] Motor 191 can generate vibration. Motor 191 can be used for incoming call reminders and can also be used for touch feedback. Motor 191 can generate different vibration feedback effects for touch operations acting on different applications. Motor 191 can also generate different vibration feedback effects for touch operations acting on different areas of the display screen 194. Different application scenarios (for example, time reminders, receiving messages, alarm clocks, and games) can correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0253] In some embodiments, the processor 110 is used to start the camera sensor at a first moment; at a second moment, obtain the exposure parameters and aperture parameters of the Nth frame; wherein the first moment is before the second moment; at a third moment, the exposure parameters are sent to the camera sensor; the third moment is the moment corresponding to the start of frame delimiter SOF of the Nth frame; the second moment is before the third moment; at a fourth moment, the aperture component adjusts the aperture based on the aperture parameters, and the fourth moment is the moment corresponding to the end of frame delimiter EOF of the N+1th frame; at a fifth moment, the first calibration data is sent to the camera sensor; the fifth moment is the moment corresponding to the SOF of the N+2th frame.

[0254] Optionally, in some embodiments, the processor 110 is further used to call the display screen 194 to display the image corresponding to the N+3 frame based on the exposure parameters, the aperture parameters and the first calibration data within the display time corresponding to the N+3 frame.

[0255] Optionally, in some embodiments, the processor 110 is further configured to call the display screen 194 to display the image corresponding to the N+1th frame within the display duration of the N+2th frame.

[0256] It can be understood that the method for photographing in the embodiment of the present application can be applied to Fig. 9 In the electronic device shown in , the specific implementation steps can refer to the introduction of the method embodiment above, which will not be repeated here.

[0257] Fig.10 FIG. 8 is a schematic block diagram of an apparatus 800 for photographing according to an embodiment of the present application. It should be understood that the apparatus 800 may perform Figures 4 to 7 The method used for shooting is shown.

[0258] like Fig.10 As shown, the device 800 includes: an input unit 810 and a processing unit 820. The device 800 may be an electronic device. Optionally, the device further includes a display unit 830.

[0259] In some embodiments, the processing unit 820 is used to start the camera sensor at a first moment; at a second moment, obtain the exposure parameters and aperture parameters of the Nth frame; wherein the first moment is before the second moment; at a third moment, the exposure parameters are sent to the camera sensor; the third moment is the moment corresponding to the start of frame delimiter SOF of the Nth frame; the second moment is before the third moment; at a fourth moment, the aperture component adjusts the aperture based on the aperture parameters, and the fourth moment is the moment corresponding to the end of frame delimiter EOF of the N+1th frame; at a fifth moment, the first calibration data is sent to the camera sensor; the fifth moment is the moment corresponding to the SOF of the N+2th frame.

[0260] In some embodiments, the processing unit 820 is configured to start a camera sensor, including: starting the camera sensor in response to a first operation of a user.

[0261] Optionally, the input unit 810 is used to detect a first operation of a user.

[0262] Optionally, the first operation is an operation of opening a camera application.

[0263] In some embodiments, the processing unit 820 is used to obtain the exposure parameters and aperture parameters of the Nth frame, including:

[0264] In response to the second operation, the exposure parameter and the aperture parameter are acquired.

[0265] Optionally, the second operation is an operation of manually adjusting the aperture by a user, or the second operation is an operation of automatically adjusting the aperture by the electronic device.

[0266] Optionally, the input unit 810 is used to detect a user's manual operation of adjusting the aperture.

[0267] In some embodiments, the display unit 830 is used to display the image corresponding to the N+3th frame based on the exposure parameter, the aperture parameter and the first calibration data within the display time corresponding to the N+3th frame.

[0268] In some embodiments, the aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture.

[0269] In some embodiments, the first calibration data is calibration data corresponding to the first aperture.

[0270] Optionally, the device 800 includes a plurality of apertures, and the plurality of apertures include at least a first aperture.

[0271] In some embodiments, the first calibration data is calibration data obtained by preprocessing the second calibration data, wherein the number of bits corresponding to the second calibration data is greater than the number of bits corresponding to the first calibration data.

[0272] In some embodiments, the transmission duration of the first calibration data is less than the frame interval between the N+2th frame and the N+3th frame.

[0273] In some embodiments, the display unit 830 is further configured to display an image corresponding to the N+1th frame within a display duration of the N+2th frame.

[0274] In some embodiments, the display unit 830 is also used to display a first preview interface, which includes a zoom ratio option; the processing unit 820 is used to respond to the user's operation of adjusting the zoom ratio to a first zoom ratio based on the zoom ratio option, and, when detecting that the current shooting environment is a high-illuminance environment, call the camera sensor to adopt a full-size cropping mode; wherein the first preview interface is a photo preview interface or a video preview interface; the first zoom ratio is greater than a preset zoom ratio; the high-illuminance environment includes: the ambient illumination of the current shooting environment is greater than a first illumination threshold.

[0275] In some embodiments, the display unit 830 is also used to display a second preview interface, which includes a zoom ratio option; the processing unit 820 is used to respond to the user's operation of adjusting the zoom ratio to a second zoom ratio based on the zoom ratio option, and, when it is detected that the current shooting environment is a high dynamic environment, call the camera sensor to adopt a full-size cropping mode; wherein the second preview interface is a photo preview interface or a video preview interface; the second zoom ratio is greater than a preset zoom ratio; the high dynamic environment includes: the dynamic range value satisfies the dynamic range DR constraint.

[0276] In a possible example, the input unit 810 and the processing unit 820 may be implemented by a processor or a processing unit. The display unit 830 may be implemented by a display screen or a display unit. It should be understood that the above-mentioned device 800 is embodied in the form of a functional unit. The term "unit" here may be implemented in the form of software and / or hardware, and the embodiments of the present application do not specifically limit this.

[0277] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two to implement the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) that executes one or more software or firmware programs and a memory, an integrated logic circuit, and / or other suitable devices that can provide the above functions. In a simple embodiment, a person skilled in the art can imagine that the device 800 can be used Fig. 9 The form shown.

[0278] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0279] The present application also provides a computer program product, which, when executed by a processor, implements the method described in any method embodiment of the present application.

[0280] The computer program product can be stored in a memory, and is finally converted into an executable target file that can be executed by a processor after preprocessing, compiling, assembling and linking.

[0281] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a computer, the method described in any method embodiment of the present application is implemented. The computer program can be a high-level language program or an executable target program.

[0282] The computer-readable storage medium may be a volatile memory or a nonvolatile memory, or may include both a volatile memory and a nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM).

[0283] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described devices and equipment and the technical effects produced can refer to the corresponding processes and technical effects in the aforementioned method embodiments, and will not be repeated here.

[0284] In several embodiments provided in the present application, the disclosed systems, devices and methods can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not performed. The device embodiments described above are merely schematic, and the division of units is only a logical function division. There may be other division methods in actual implementation, and multiple units or components may be combined or integrated into another system. In addition, the coupling between the units or the coupling between the components may be direct coupling or indirect coupling, and the above coupling includes electrical, mechanical or other forms of connection.

[0285] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0286] In addition, the terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0287] The terms (or numbers) "first", "second", ..., etc. that appear in the embodiments of the present application are only used for descriptive purposes, that is, only to distinguish different objects, such as different "calibration data", etc., and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", ..., etc. may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, "at least one (item)" refers to one or more. "Multiple" means two or more. "At least one of the following (item)" or similar expressions thereof refers to any combination of these items, including any combination of a single (item) or a plurality of (items).

[0288] For example, the meaning of the expression similar to "the item includes at least one of the following: A, B, and C" in the embodiments of the present application, unless otherwise specified, generally means that the item can be any one of the following: A; B; C; A and B; A and C; B and C; A, B and C; A and A; A, A and A; A, A and B; A, A and C, A, B and B; A, C and C; B and B, B, B and B, B, B and C, C and C; C, C and C, and other combinations of A, B and C. The above is an example of three elements, A, B and C, to illustrate the optional items of the item. When it is expressed as "the item includes at least one of the following: A, B, ..., and X", that is, when there are more elements in the expression, the items that can be applied to the item can also be obtained according to the above rules.

[0289] In short, the above is only a preferred embodiment of the technical solution of this application, and is not intended to limit the protection scope of this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application should be included in the protection scope of this application.

Claims

1. A method for photographing, It is characterized in that The method is applied to an electronic device, and the method comprises: At the first moment, the camera sensor is activated; At a second moment, acquiring exposure parameters and aperture parameters of the Nth frame; wherein the first moment is before the second moment; At a third moment, the exposure parameter is sent to the camera sensor; the third moment is a moment corresponding to the start of frame delimiter SOF of the Nth frame; the second moment is before the third moment; At a fourth moment, the aperture component adjusts the aperture based on the aperture parameter, and the fourth moment is a moment corresponding to the end-of-frame delimiter EOF of the N+1th frame; At a fifth moment, the first calibration data is sent to the camera sensor; the fifth moment is the moment corresponding to the SOF of the N+2th frame.

2. The method according to claim 1, It is characterized in that The starting the camera sensor comprises: In response to a first operation of a user, the camera sensor is activated.

3. The method according to claim 2, It is characterized in that The first operation is an operation of opening a camera application.

4. The method according to any one of claims 1 to 3, It is characterized in that The step of obtaining the exposure parameter and aperture parameter of the Nth frame includes: In response to the second operation, the exposure parameter and the aperture parameter are acquired.

5. The method according to claim 4, It is characterized in that The second operation is an operation of manually adjusting the aperture by a user, or the second operation is an operation of automatically adjusting the aperture by the electronic device.

6. The method according to any one of claims 1 to 5, It is characterized in that The method further comprises: Within the display time corresponding to the N+3th frame, the image corresponding to the N+3th frame is displayed based on the exposure parameter, the aperture parameter and the first calibration data.

7. The method according to any one of claims 1 to 6, It is characterized in that The first calibration data is calibration data corresponding to the first aperture.

8. The method according to any one of claims 1 to 7, It is characterized in that The aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture.

9. The method according to any one of claims 1 to 8, It is characterized in that The electronic device includes a plurality of apertures, and the plurality of apertures includes at least a first aperture.

10. The method according to any one of claims 1 to 9, It is characterized in that The first calibration data is calibration data obtained by preprocessing the second calibration data, wherein the number of bits corresponding to the second calibration data is greater than the number of bits corresponding to the first calibration data.

11. The method according to any one of claims 1 to 10, It is characterized in that The transmission duration of the first calibration data is less than the frame interval between the N+2th frame and the N+3th frame.

12. The method according to any one of claims 1 to 11, It is characterized in that The method further comprises: During the display duration of the N+2th frame, the image corresponding to the N+1th frame is displayed.

13. The method according to any one of claims 1 to 12, It is characterized in that After starting the camera sensor, the method further includes: Displaying a first preview interface, wherein the first preview interface includes a zoom ratio option; In response to a user adjusting the zoom ratio to a first zoom ratio based on the zoom ratio option, and detecting that the current shooting environment is a high-illuminance environment, the camera sensor adopts a full-size cropping mode; Among them, the first preview interface is a photo preview interface or a video preview interface; the first zoom ratio is greater than a preset zoom ratio; the high illumination environment includes: the ambient illumination of the current shooting environment is greater than a first illumination threshold.

14. The method according to any one of claims 1 to 12, It is characterized in that After starting the camera sensor, the method further includes: Displaying a second preview interface, wherein the second preview interface includes a zoom ratio option; In response to the user adjusting the zoom ratio to a second zoom ratio based on the zoom ratio option, and detecting that the current shooting environment is a high-dynamic environment, the camera sensor adopts a full-size cropping mode; Among them, the second preview interface is a photo preview interface or a video preview interface; the second zoom ratio is greater than a preset zoom ratio; the high dynamic environment includes: the dynamic range value satisfies the dynamic range DR constraint condition.

15. An electronic device, It is characterized in that The electronic device comprises a processor and a memory, wherein the processor and the memory are coupled, and the memory is used to store a computer program. When the computer program is executed by the processor, the electronic device executes the method according to any one of claims 1 to 14.

16. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the electronic device executes the method according to any one of claims 1 to 14.

17. A chip system, It is characterized in that The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method as described in any one of claims 1 to 14.

Images