Photographing method and electronic device

Abstract

A photographing method and an electronic device, applied to the technical field of terminals. The method comprises: starting a camera sensor at a first moment; obtaining an exposure parameter and an aperture parameter of an Nth frame at a second moment; issuing the exposure parameter to the camera sensor at a third moment; adjusting an aperture by means of an aperture component on the basis of the aperture parameter; and issuing first calibration data to the camera sensor at a fifth moment. Embodiments of the present application support issuing calibration data after the camera sensor starts streaming, improve the image output quality, and are conducive to improving the user photographing experience.

Description

Method and electronic device for photographing

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 30, 2023, with application number 202311637550.X and application name “A method and electronic device for photographing”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of image processing technology, and in particular, to a method and electronic device for photographing. Background Art

[0003] With the increasing development of smart terminals, the shooting function has become an essential function of smart terminals. Users' demand for and experience of shooting (taking photos and / or recording videos) with smart terminals are also increasing. When shooting, smart terminals will control the blur of the background, or the blur effect, by adjusting the aperture of the camera. The aperture is a device used to control the amount of light entering. Generally speaking, the larger the aperture, the shallower the depth of field, and accordingly, the more obvious the blur effect of the image; the smaller the aperture, the less light entering, the deeper the depth of field, and accordingly, the weaker the blur effect of the image.

[0004] To achieve better image quality, smart terminals use calibration data pre-set in registers to calibrate the image data of the camera sensor. However, for smart terminals with variable apertures, this method is not very effective and affects the image quality.

[0005] Summary of the Invention

[0006] In view of this, the present application provides a method, electronic device, computer-readable storage medium and computer program product for shooting, which can switch calibration data after the aperture changes, improve image effects, and help improve the user's shooting experience.

[0007] In a first aspect, a method for photographing is provided, the method being applied to an electronic device, the method comprising:

[0008] At the first moment, the camera sensor is activated;

[0009] At a second moment, acquiring exposure parameters and aperture parameters of the Nth frame; wherein the first moment is before the second moment;

[0010] At a third moment, the exposure parameter is sent to the camera sensor; the third moment is a moment corresponding to a start of frame delimiter SOF of the Nth frame; and the second moment is before the third moment;

[0011] At a fourth moment, the aperture component adjusts the aperture based on the aperture parameter, and the fourth moment is a moment corresponding to the end-of-frame delimiter EOF of the N+1th frame;

[0012] At a fifth moment, the first calibration data is sent to the camera sensor; the fifth moment is a moment corresponding to the SOF of the N+2th frame.

[0013] The above method can be executed by an electronic device or a chip within the electronic device. Based on the above solution, the electronic device activates the camera sensor at the first moment, adjusts the aperture based on the aperture parameters after receiving them, and sends the first calibration data to the camera sensor after the aperture adjustment begins. Compared to burning a set of calibration data before the camera sensor starts streaming, the embodiments of the present application support sending calibration data after the camera sensor starts streaming, improving image quality and enhancing the user's shooting experience.

[0014] In a possible implementation, the electronic device includes multiple apertures, and the multiple apertures include at least the first aperture. In other words, the embodiment of the present application is applied to an electronic device with a variable aperture.

[0015] The embodiments of the present application are applicable to scenarios where a camera sensor uses a quad bayer coding (QBC) array to output images, or in other words, are suitable for a quad sensor. The pixel arrangement of the image data output by a quad sensor differs from the pixel arrangement of the image data output by a traditional sensor. For the image data output by a quad sensor, there are brightness differences (or light sensitivity differences) between sub-pixels of the same color, so quad bayer coding sensitivity correction (QSC) calibration is required. This is because if QSC correction (or QSC calibration) is not performed, the image data output by the quad sensor will have uneven brightness, resulting in poor output image quality.

[0016] It can be understood that the embodiments of the present application are applicable to scenarios where users use electronic devices to take photos or videos.

[0017] In one possible implementation, the electronic device activates a camera sensor in response to a first operation by the user.

[0018] Optionally, the first operation is an operation of opening a camera application, wherein the electronic device has a camera application installed therein.

[0019] The embodiments of the present application do not limit the specific content of the first operation. The first operation can be understood as an operation of opening the camera application in any manner. The first operation includes but is not limited to any of the following forms: through voice commands, physical buttons, and UI interactive operations, etc.

[0020] Exemplarily, the first operation is an operation in which the user clicks on a camera application in an interface of the electronic device.

[0021] In a possible implementation, the aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture.

[0022] The aforementioned exposure parameters and aperture parameters are obtained from the parameters issued by the automatic exposure module. It should be noted that the embodiments of the present application do not limit whether the timing of obtaining the exposure parameters and the aperture parameters is the same, or in other words, do not limit the order in which the two are obtained. For example, after adjusting the parameters, the exposure parameters and the aperture parameters can be obtained at the same time. For another example, after adjusting the parameters, the exposure parameters can be obtained first, and the aperture parameters corresponding to the exposure parameters can be obtained later. For another example, after adjusting the parameters, the aperture parameters can be obtained first, and the exposure parameters corresponding to the aperture parameters can be obtained later.

[0023] In one possible implementation, obtaining exposure parameters and aperture parameters of the Nth frame includes:

[0024] In response to a second operation, the exposure parameter and the aperture parameter are acquired.

[0025] The above-mentioned exposure parameters and aperture parameters may be acquired in response to a second operation, and the embodiment of the present application does not limit the specific form of the second operation. The second operation may be understood as an operation that triggers the aperture of the electronic device to switch.

[0026] Optionally, the second operation is an operation of manually adjusting the aperture by a user, or the second operation is an operation of automatically adjusting the aperture by the electronic device.

[0027] That is, at the second moment, the electronic device can obtain exposure parameters and aperture parameters in response to the user's manual aperture adjustment operation. For example, the second operation is the user manually adjusting the aperture in professional shooting mode. For another example, the second operation is the user switching from the current normal shooting mode to the large aperture shooting mode.

[0028] Alternatively, at the second moment, the electronic device may obtain the exposure and aperture parameters based on an automatic aperture adjustment operation. For example, if the electronic device is equipped with a variable aperture, upon detecting a specific shooting scene, the electronic device may automatically adjust the variable aperture, at which point the electronic device will obtain the exposure and aperture parameters.

[0029] In one possible implementation, during the display duration corresponding to the N+3th frame, the image corresponding to the N+3th frame is displayed based on the exposure parameters, the aperture parameters, and the first calibration data. That is, after the calibration data is sent, the exposure parameters of the image of the N+3th frame displayed match the aperture parameters. Because the aperture has been switched to the first aperture and the sent first QSC calibration data has taken effect, the image quality corresponding to the N+3th frame is better than that obtained using the QSC calibration data corresponding to the original aperture. That is, compared to using the QSC calibration data of the original aperture (i.e., the aperture before the switch), using the QSC calibration data corresponding to the first aperture after the switch for compensation results in better image quality.

[0030] In one possible implementation, the first calibration data is calibration data corresponding to the first aperture. For example, when the camera sensor is a quad sensor, the first calibration data corresponding to the first aperture is QSC calibration data. Thus, after switching to the first aperture, the QSC calibration data corresponding to the first aperture can be used to compensate for image data output by the quad sensor, thereby improving image quality.

[0031] In one possible implementation, the aperture parameter is used to instruct the camera sensor to switch the current aperture to a first aperture. Based on the aperture parameter, the aperture assembly switches the current aperture to an aperture corresponding to the aperture parameter, such as the first aperture, to implement aperture switching.

[0032] Since sending the first calibration data requires a certain amount of transmission time, in order to minimize the impact on the image frames during the process of sending the calibration data, embodiments of the present application can pre-process the calibration data. That is, the first calibration data sent is the pre-processed calibration data. After pre-processing, the corresponding data volume occupies fewer bits, thereby reducing the transmission time.

[0033] In a possible implementation, the first calibration data is calibration data obtained by preprocessing the second calibration data, wherein the number of bits corresponding to the second calibration data is greater than the number of bits corresponding to the first calibration data.

[0034] The embodiments of the present application do not limit the specific method of preprocessing. One possible implementation method is that since a portion of the image area is used for cropping in high zoom ratio shooting scenarios, the QSC calibration data can be cropped in the corresponding area. This ensures the data compensation effect while reducing the amount of QSC calibration data and the number of affected image frames.

[0035] In a possible implementation, a transmission duration of the first calibration data is less than a frame interval between the N+2th frame and the N+3th frame.

[0036] In a possible implementation, the method further includes:

[0037] During the display duration of the N+2th frame, the image corresponding to the N+1th frame is displayed.

[0038] Since the process of sending the first calibration data affects the image of the N+2 frame, the brightness of the image of the N+2 frame becomes abnormal, and the image of the N+2 frame is not QSC compensated, the image of the N+2 frame needs to be discarded when it is displayed. At this time, the image corresponding to the N+1 frame can be displayed to minimize the impact caused by not displaying the image of the N+2 frame.

[0039] In a possible implementation, starting a camera sensor includes:

[0040] In response to receiving a first operation of the user, a camera sensor is activated, where the first operation is used to turn on the camera.

[0041] The embodiments of the present application can be applied to a shooting scene in a preview scene with a high zoom ratio (including photo preview or video preview) and a high illumination environment.

[0042] In a possible implementation, after starting the camera sensor, the method further includes:

[0043] Displaying a first preview interface, wherein the first preview interface includes a zoom ratio option;

[0044] In response to a user adjusting the zoom magnification to a first zoom magnification based on the zoom magnification option, and detecting that the current shooting environment is a high-illumination environment, the camera sensor adopts a full-size cropping mode;

[0045] Among them, the first preview interface is a photo preview interface or a video preview interface; the first zoom ratio is greater than a preset zoom ratio; the high illumination environment includes: the ambient illumination of the current shooting environment is greater than a first illumination threshold.

[0046] The embodiments of the present application can be applied to a preview scene with a high zoom ratio (including photo preview or video preview) and a shooting scene in a high-dynamic environment.

[0047] In a possible implementation, after starting the camera sensor, the method further includes:

[0048] Displaying a second preview interface, wherein the second preview interface includes a zoom ratio option;

[0049] In response to a user adjusting the zoom ratio to a second zoom ratio based on the zoom ratio option, and detecting that the current shooting environment is a high-dynamic environment, the camera sensor adopts a full-size cropping mode;

[0050] Among them, the second preview interface is a photo preview interface or a video preview interface; the second zoom ratio is greater than a preset zoom ratio; the high dynamic environment includes: the dynamic range value meets the dynamic range DR constraint condition.

[0051] It should be noted that in all the above shooting scenarios, the sensor uses a full-size crop output mode. Therefore, if the aperture changes in these scenarios, QSC calibration is required to improve the image quality.

[0052] In a second aspect, an electronic device is provided, comprising a unit for executing any one of the methods in the first aspect. The electronic device may be a terminal or a chip within the terminal. The electronic device includes a communication unit, a display unit, and a processing unit.

[0053] When the electronic device is a terminal, the processing unit may be a processor, the communication unit may be a communication interface, and the display unit may be a graphics processing module and a screen; the terminal may also include a memory for storing computer program code, and when the processor executes the computer program code stored in the memory, the terminal executes any one of the methods in the first aspect.

[0054] When the electronic device is a chip in a terminal, the processing unit may be a logic processing unit inside the chip, the communication unit may be a communication interface, pin or circuit, etc., and the display unit may be a graphics processing unit inside the chip; the chip may also include a memory, which may be a memory inside the chip (for example, a register, a cache, etc.) or a memory located outside the chip (for example, a read-only memory, a random access memory, etc.); the memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the chip executes any one of the methods of the first aspect.

[0055] According to a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer program code. When the computer program code is executed by an electronic device, the electronic device executes any one of the methods according to the first aspect.

[0056] In a fourth aspect, a computer program product is provided, comprising: a computer program code, wherein when the computer program code is executed by an electronic device, the electronic device executes any one of the methods in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] FIG1A is an example diagram of an application scenario of an embodiment of the present application;

[0058] FIG1B is another example diagram of an application scenario of an embodiment of the present application;

[0059] FIG2A is a schematic diagram of the pixel array distribution of QBC;

[0060] FIG2B is a schematic diagram showing a comparison before and after performing QSC correction;

[0061] FIG3 is a schematic diagram of a software architecture of an embodiment of the present application;

[0062] FIG4 is a timing interaction diagram applied to the software architecture of FIG3 ;

[0063] FIG5 is a timing diagram illustrating an example of sending QSC calibration data according to an embodiment of the present application;

[0064] FIG6 is a schematic diagram showing the comparison of preprocessing of QSC correction data;

[0065] FIG7 is a schematic flow chart of a method for photographing according to an embodiment of the present application;

[0066] FIG8A is an example diagram of an interface for adjusting the aperture by a user according to an embodiment of the present application;

[0067] FIG8B is another example diagram of an interface for adjusting the aperture by a user according to an embodiment of the present application;

[0068] FIG9 is a schematic structural diagram of an electronic device applicable to the present application;

[0069] FIG10 is a schematic block diagram of a device for photographing according to an embodiment of the present application. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0071] In the embodiments of the present application, unless otherwise specified, “a plurality of” may mean two or more.

[0072] The embodiments of the present application are applicable to electronic devices, which may be mobile phones, smart screens, tablet computers, wearable electronic devices, in-vehicle electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), projectors, and the like.

[0073] The embodiments of the present application are applicable to electronic devices including a camera sensor and multiple apertures. Among them, the multiple apertures can have different characteristics. By setting multiple apertures, different apertures correspond to different amounts of light entering (i.e., the amount of light entering the interior of the body). In this way, when the electronic device switches to different apertures, different depth of field effects, or blurring effects of taking photos, can be obtained based on different amounts of light entering. In other words, compared to electronic devices with a single aperture that cannot adjust the amount of light entering, the advantage of setting multiple apertures in electronic devices is that the amount of light entering can be dynamically adjusted to optimize the depth of field effect. Furthermore, although electronic devices with fixed apertures can adjust the aperture parameters, the aperture parameters are simulated by an algorithm inside the electronic device, and the effect of this method is worse than adjusting the physical aperture (i.e., switching different apertures). The following describes the benefits of setting multiple apertures in electronic devices in combination with different scenarios.

[0074] In some embodiments, during the shooting process, the electronic device can select an appropriate aperture based on the distance of the target object to ensure the blur effect of the photo. Different depths of field will affect the blur effect of the background of the picture, resulting in different blur effects of the picture.

[0075] In some embodiments, in low-light shooting environments (e.g., night scenes), the electronic device can ensure the amount of light entering by switching to a large aperture, thereby presenting a better preview effect in the night shooting scene, making the preview image of the target object clearer.

[0076] For the purposes of this explanation, a low-light environment can be understood as a dim environment. This includes situations where the ambient illumination of the shooting environment is less than a preset brightness threshold. Similarly, a high-light environment includes situations where the ambient illumination of the shooting environment is greater than or equal to a preset brightness threshold. A high-light environment can also be understood as a bright environment.

[0077] Ambient illumination refers to the light intensity of the user's shooting environment. Ambient illumination can be represented by the following indicators: lighting value (LV), illuminance (lux), or illuminance index (luxindex).

[0078] LV is used to estimate the ambient brightness. Its specific calculation formula is as follows:

[0079] Among them, Exposure is the exposure time, Aperture is the aperture size, Iso is the sensitivity, and Luma is the average value of Y in the XYZ color space of the image.

[0080] For example, the ambient illumination is represented by LV, and accordingly, the brightness threshold is the LV threshold. When the ambient illumination of the shooting environment is greater than the LV threshold, the current shooting environment is a high illumination environment.

[0081] Optionally, in some implementations, the ambient illumination of the current shooting environment may be acquired through an ambient light sensor, and then it is determined whether the current environment is a high illumination environment based on the acquired ambient illumination.

[0082] In some embodiments, in the professional shooting mode, the electronic device adjusts the aperture accordingly in response to the user manually adjusting the aperture parameters, thereby providing the user with a richer shooting experience.

[0083] Optionally, the aperture assembly of the electronic device includes at least a first aperture and a second aperture, wherein the amount of light entering the first aperture is different from the amount of light entering the second aperture. For example, the first aperture is a large aperture, and the second aperture is a small aperture; the amount of light entering the first aperture is greater than the amount of light entering the second aperture. In this embodiment of the present application, each aperture has corresponding calibration data. For example, the first aperture corresponds to first calibration data, and the second aperture corresponds to second calibration data. The meaning of the calibration data will be described later in Figure 4.

[0084] The above describes different scenarios for electronic devices with multiple apertures. The following describes different shooting scenarios to which the embodiments of the present application are applicable.

[0085] In some embodiments, in the photo preview mode, for a shooting scene with a zoom ratio of 2x or more (for example, 3x) and a high illumination environment, the sensor uses a full size crop mode to output the image.

[0086] In other embodiments, in photo preview mode, for scenes with a zoom ratio of 2x or greater (e.g., 3x) and a high dynamic range, the sensor uses a full-size cropped and staggered image output mode. The full-size cropped and staggered image output method can be found in the relevant art and will not be further elaborated here.

[0087] For a unified explanation, the dynamic range of a scene can be divided into high dynamic range and low dynamic range based on preset conditions. For example, if the dynamic range value of a scene satisfies the dynamic range (DR) constraint, it is considered high dynamic range; if the dynamic range value of a scene does not meet the DR constraint, it is considered low dynamic range.

[0088] Alternatively, the DR constraint may be determined based on a histogram of a RAW image of the captured scene. Specifically, the dynamic range of the scene may be determined based on the percentage of overexposed pixels and the percentage of underexposed pixels in the image.

[0089] It should be understood that the above-mentioned high dynamic range division process is only an exemplary description, and the embodiments of the present application are not limited thereto. In fact, it is also possible to determine whether it is a high dynamic range scene based on other methods in the art.

[0090] For ease of understanding, the following is an example of a photo preview application scenario in conjunction with FIG1A. The present application embodiment does not impose any restrictions on the specific type of electronic device. The following describes the method for shooting in the present application embodiment using a mobile phone as an example.

[0091] FIG1A is an example diagram of an application scenario under a high-illuminance environment photo preview in an embodiment of the present application. In the mobile phone interface shown in FIG1A (1), the interface can display multiple applications: application 1, application 2, ..., application 7 and a camera application. The mobile phone starts the camera application in response to the user clicking the camera application. After the camera application is running, the mobile phone interface displays the interface shown in FIG1A (2). The interface shown in FIG1A (2) can be called the camera photo preview interface. The photo preview interface can include a viewfinder 11, a light source 17, a zoom factor 12 (default is 1x), an album icon 13, a shooting control 14 and a camera rotation control, etc. The brightness of the light source 17 can determine whether the current shooting environment is an illumination environment. For example, when the brightness of the light source 17 is high, so that the ambient illumination of the current shooting environment is greater than the preset brightness threshold, it can be determined that the current scene is a high-illuminance scene (or a highlight scene).

[0092] The phone can take a photo in response to the user clicking the shooting control 14. The album icon 13 displays thumbnails of the photos. The camera rotation control can be used to switch cameras. Among them, the viewfinder 11 is used to obtain a preview image of the shooting, and can display the preview image in real time.

[0093] In the photo preview scenario, the mobile phone can support digital zoom. When using the photo function, the user can select different zoom ratios by operating on the touch screen. As an example, as shown in (2) in Figure 1A, the user clicks on the zoom ratio 12 in (2) in Figure 1A, and the interface is displayed as shown in (3) in Figure 1A, and the zoom ratio selection item 15 appears (for example, the highest zoom ratio is 8x and the lowest zoom ratio is 1x). The user drags the zoom ratio 12 upward in the selection item 15, and releases it when the zoom ratio is 2x. The interface is displayed as shown in (4) in Figure 1A, that is, the zoom ratio is selected as 2x. It can be seen that the display area of ​​the light source 17 shown in (4) in Figure 1A in the viewfinder 11 also becomes larger. Of course, after the zoom ratio is selected, the zoom ratio selection item 15 can be hidden, that is, the selected zoom ratio is displayed as 2x on the interface.

[0094] Of course, the above description is based on the example of a 2x zoom ratio selected in the photo preview scene, and the present application embodiment is not limited thereto. For example, the user can also adjust the zoom ratio to more than 2x.

[0095] It should be understood that (2)-(4) in FIG1A shows an interface diagram of a user taking a photo with the mobile phone in portrait mode, but the present application is not limited thereto. For example, the user can take a photo with the mobile phone in landscape mode.

[0096] In some other embodiments, in the video preview mode, for a shooting scene with a zoom ratio of 2x or more (for example, 3x) and a high-illumination environment, the sensor uses a full-size crop mode to output the image.

[0097] Here, cropping refers to cropping the captured image to obtain a field of view corresponding to the target zoom factor. For the description of the high illumination environment, please refer to the previous description, and for the sake of brevity, it will not be repeated here.

[0098] For ease of understanding, the following describes an example of a high-illumination environment in a video preview scenario in conjunction with FIG1B . As shown in FIG1B (1), in response to a user clicking a video control, the mobile phone displays a video preview interface, such as the video preview interface 181 shown in FIG1B (2), as shown in FIG1B (2).

[0099] As shown in (2) of Figure 1B, the video preview interface 181 includes an album icon 13, a video control 16, a light source 17, and a zoom factor 12 (default is 1x). The light source 17 indicates that the current video preview scene is a highlighted scene.

[0100] Similarly, in the video preview scenario, the mobile phone can also support digital zoom. When using the video recording function, the user can select different zoom ratios by operating on the touch screen. As an example, as shown in (2) in Figure 1B, in response to the user clicking the zoom ratio 12 in (2) in Figure 1B, the mobile phone displays the interface 182 shown in (3) in Figure 1B, and the interface 182 includes the zoom ratio selection item 15 (for example, the highest zoom ratio is 8x and the lowest zoom ratio is 1x). In response to the user dragging the zoom ratio 12 upward in the selection item 15 and releasing it when the zoom ratio is 2x, the mobile phone displays the interface 183 shown in (4) in Figure 1B, that is, the zoom ratio is selected as 2x. Of course, after the zoom ratio is selected, the zoom ratio selection item 15 can be hidden, that is, the selected zoom ratio is displayed as 2x on the interface.

[0101] Of course, the above description is based on the example of the zoom ratio of 2x selected in the video preview scene, and the embodiment of the present application is not limited thereto. For example, the user can also adjust the zoom ratio to more than 2x.

[0102] It should be understood that the interface shown in (2)-(4) in Figure 1B can be an example interface of an application scenario of an embodiment of the present application.

[0103] In other embodiments, in video preview mode, for scenes with a zoom ratio of 2x or greater (eg, 3x) and a high dynamic range, the sensor uses a full-size crop and staggered mode for image output.

[0104] For the relevant description of high dynamic range, please refer to the previous description. For the sake of brevity, it will not be repeated here.

[0105] The embodiments of the present application are applicable to scenarios where sensors use a quad Bayer coding (QBC) array for image output, or in other words, are suitable for quad sensors. The pixel arrangement of image data output by a quad sensor differs from that of image data output by traditional sensors. The image data output by a quad sensor consists of four pixels (represented as RG, G, and GB), each of which is composed of four sub-pixels. The pixel arrangement of image data output by a quad sensor is described below with reference to FIG2A.

[0106] As shown in Figure 2A, the image data output by a quad sensor contains four color pixels: R, Gr, Gb, and B. Each color pixel is composed of four sub-pixels: R0, R1, R2, and R3; Gr0, Gr1, Gr2, and Gr3; Gb0, Gb1, Gb2, and Gb3; and B0, B1, B2, and B3. Within each color pixel (or, in other words, within adjacent 2x2 sub-pixels of the same color), for example, R0, R1, R2, and R3, there are differences in sensitivity between sub-pixels of the same color. Therefore, Quad Bayer Coding Sensitivity Correction (QSC) calibration is required. This sensitivity difference ultimately leads to differences in brightness.

[0107] This is because, if QSC correction (or QSC calibration) is not performed, the image data output by the Quad sensor will have uneven brightness, resulting in poor image quality. Poor image quality can be manifested as streaks in the image, such as crosstalk or other forms of streaks.

[0108] Figure 2B shows a comparison of the sensitivity differences between pixels before and after QSC correction. The vertical axis in Figure 2B represents the sensitivity values ​​corresponding to different color pixels (or sub-pixels). As shown in Figure 2B, before QSC correction, the sensitivity differences between sub-pixels R0, R1, R2, and R3 were large. After QSC correction, the sensitivity differences between sub-pixels R0, R1, R2, and R3 were reduced.

[0109] In the above four shooting scenarios, specifically including: (1) in the photo preview mode, the zoom ratio is 2x or more (for example, 3x) and the shooting scene is in a high-light environment; (2) in the photo preview mode, the zoom ratio is 2x or more (for example, 3x) and the shooting scene is in a high dynamic range; (3) in the video preview mode, the zoom ratio is 2x or more (for example, 3x) and the shooting scene is in a high-light environment; (4) in the video preview mode, the zoom ratio is 2x or more (for example, 3x) and the shooting scene is in a high dynamic range, the sensor uses full-size cropping for image output. In the above four shooting scenarios, if the aperture of the electronic device changes, the image data needs to be calibrated using calibration data to ensure the image effect. Calibration data refers to data used to compensate the collected image data.

[0110] Currently, when electronic devices undergo factory calibration, QSC calibration data is burned into the device's memory, such as an EEPROM. This allows calibration data to be used to correct image data after the sensor starts streaming, before the sensor starts streaming, or during camera initialization. Sensor streaming can be understood as the sensor starting to output image data, or transmitting a data stream. However, this method cannot support scenarios where the aperture changes, or sensors with variable aperture features. That is, when the electronic device switches from one aperture to another, the calibration data for the aperture before the switch is still used. This results in poor calibration results, thus affecting image quality.

[0111] In view of this, an embodiment of the present application proposes a method for shooting. In a scenario where the sensor adopts a quadruple Bayer array QBC, it supports the issuance of calibration data after the aperture changes or after the sensor starts streaming, that is, it can dynamically update the QSC calibration data. The image data compensated by the calibration data can effectively reduce the brightness difference between pixels, thereby reducing the stripes in the image caused by the brightness difference and improving the image effect.

[0112] The following first describes the software system used in the embodiment of the present application in conjunction with Figure 3.

[0113] Figure 3 is a schematic diagram of the architecture (including software systems and some hardware) of the application embodiment of the present application. As shown in Figure 3, the application architecture is divided into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the application architecture can be divided into five layers, from top to bottom, namely, the application layer (Application Layer), the application framework layer (Application Framework Layer), the hardware abstraction layer (Hardware Abstraction Layer), the kernel layer (Kernel Layer) and the hardware layer (Hardware Layer).

[0114] As shown in Figure 3, the application layer includes a camera and a gallery. It is understood that Figure 3 shows only some of the applications. In fact, the application layer can also include other applications (including system applications and / or third-party applications), which is not limited in this application. For example, the application layer also includes applications such as information, alarm clock, weather, stopwatch, compass, timer, flashlight, calendar, and Alipay.

[0115] As shown in Figure 3, the application framework layer includes the camera access interface. For example, the camera access interface includes camera management and camera devices. The hardware abstraction layer includes the automatic exposure (AE) module, the variable aperture HAL module, and the sensor service processing module (for example, the sensor node HAL module). The AE module is used to automatically adjust the sensor exposure time and image brightness when the external lighting conditions change.

[0116] It is understood that the hardware abstraction layer shown in FIG3 may also include other modules, and the embodiments of the present application are not limited thereto. For example, the hardware abstraction layer may also optionally include a camera algorithm library. The camera algorithm library includes an image processing algorithm module.

[0117] The kernel layer is used to drive hardware resources. It can include multiple driver modules. As shown in Figure 3, the kernel layer includes a variable aperture driver and an image sensor driver. Optionally, the kernel layer may also include a digital signal processor driver and a graphics processor driver.

[0118] The hardware layer includes various hardware resources. As shown in Figure 3, the hardware layer includes hardware related to the camera module, such as the image sensor and the variable aperture component.

[0119] Optionally, the camera module further includes an electrically programmable read-only memory (EEPROM) or other storage resources. The EEPROM can store QSC calibration data corresponding to the aperture.

[0120] Optionally, the hardware layer also includes other sensors (such as an ambient light sensor), an image signal processor, a digital signal processor, and a graphics processor.

[0121] It should be understood that the software architecture shown in Figure 3 is only an example description, and the embodiments of the present application are not limited thereto. In fact, each layer in Figure 3 may include more other modules.

[0122] The following briefly describes the method timing interaction process when applied to the software architecture of Figure 3 in conjunction with Figure 4. The sensor node HAL module and the variable aperture HAL module shown in Figure 4 can be located in the HAL layer of Figure 3. The variable aperture driver and image sensor driver shown in Figure 4 can be located in the kernel layer of Figure 3. The kernel layer in Figure 4 communicates with the hardware abstraction layer through the middle layer (or interface, also referred to as the middle layer). As shown in Figure 4, the following steps are included but are not limited to:

[0123] Step 1: The sensor node HAL module sends an ExecuteProcessRequest to the variable aperture HAL module.

[0124] The execution process request may correspond to each image frame, or in other words, when each image frame is output, the sensor node HAL module will issue an execution process request.

[0125] Optionally, the execution process request includes an aperture parameter, for example, the aperture parameter is an aperture value (ie, VA code).

[0126] Step 2: The variable aperture HAL module obtains aperture parameters. The aperture parameters are used to implement aperture switching.

[0127] After receiving a request to execute a process, the variable aperture module obtains the aperture parameters from the request. For example, the variable aperture HAL module calls the aperture parameter method function, which converts the aperture parameters into variable aperture parameter values. For example, the aperture parameter method function is the GetVAparams method function, which obtains the aperture value (i.e., VA code) from the AE tag. The AE tag can be understood as a parameter issued by the AE module; the GetVAparams method function is a function that obtains the aperture parameters. In other words, by calling the GetVAparams method function, the aperture value can be obtained from the AE tag.

[0128] Step 3: The variable aperture HAL module sends the aperture parameters to the middle layer.

[0129] Step 4: The middle layer sends the aperture parameters to the variable aperture driver.

[0130] Step 5: The variable aperture driver applies the aperture parameters.

[0131] Exemplarily, the variable aperture driver applies the aperture parameter to perform aperture switching, switching from the current aperture to a first aperture corresponding to the aperture parameter.

[0132] Of course, after the variable aperture driver applies the aperture parameters, it will drive the hardware of the corresponding aperture (for example, the first aperture) to work.

[0133] In step 6-1, the variable aperture driver returns the application result of the aperture parameter to the middle layer.

[0134] In step 6-2, the middle layer passes the application result of the aperture parameters to the variable aperture HAL module.

[0135] In step 6-3, the variable aperture HAL module passes the application result of the aperture parameter to the sensor node HAL module.

[0136] Step 7: The sensor node HAL module sends the QSC calibration data to the middle layer.

[0137] Specifically, after determining that the aperture parameters have changed, the sensor node HAL module will issue a command including QSC calibration data, which corresponds to the switched aperture (for example, the QSC calibration data corresponding to the first aperture) so as to update the QSC calibration data in time to adapt to the switched aperture.

[0138] In step 8, the middle layer passes the QSC calibration data to the image sensor driver.

[0139] Step 9: The image sensor driver applies the QSC calibration data, or in other words, the QSC calibration data takes effect.

[0140] Optionally, the QSC calibration data comprises a sequence or matrix of luminance offset values. For example, if the QSC calibration data is a matrix, each element in the matrix corresponds to a pixel (specifically, a sub-pixel of a pixel), and the value of the matrix element is the calibration data value corresponding to the pixel (or sub-pixel).

[0141] For each pixel's sub-pixel, its pixel value can be compensated using the corresponding calibration data value. For example, if the sub-pixel value of a pixel is 100, by superimposing the corresponding calibration data value on this pixel value (for example, the calibration data value corresponding to the sub-pixel of this pixel is -20), the compensated pixel value is 80. The corresponding calibration data value is superimposed on the pixel value of each pixel to reduce brightness differences.

[0142] It should be understood that the embodiments of this application are described using only QSC calibration data (e.g., first calibration data, second calibration data) as the naming, and the embodiments of this application are not limited thereto. In fact, calibration data may also be named or referred to by other names, such as compensation data, correction data, and corrected data. However, regardless of the naming, the essence of the terminology remains unchanged, that is, the relevant explanation of the terminology can refer to the above description of calibration data.

[0143] In step 10-1, the image sensor driver returns the application result of the QSC calibration data to the middle layer.

[0144] In step 10-2, the middle layer passes the application results of the QSC calibration data to the sensor node HAL module.

[0145] Regarding the process in Figure 4, steps 3 to 6-3 can be understood as the process of changing the aperture, or the process of switching the aperture. Steps 7 to 10-2 can be understood as the process of sending QSC calibration data.

[0146] In some embodiments, steps 3 to 6-3 occur in one image frame, and steps 7 to 10-2 occur in the next image frame. For example, steps 3 to 6-3 occur in the N+1th frame, and steps 7 to 10-2 occur in the N+2th frame.

[0147] To facilitate understanding of the timing of issuing calibration data in the embodiment of the present application, the following description is made in conjunction with the timing in Figure 5. As shown in Figure 5, Figure 5 shows the time axis after the sensor starts to flow. T1 to T14 can be understood as the corresponding partial timestamps after the sensor starts to flow. Taking the N-1th frame as an example, the timestamp corresponding to the start-of-frame delimiter (SOF) of the N-1th frame is T1, and the timestamp corresponding to the end-of-frame delimiter (EOF) of the N-1th frame is T2, and the timestamps corresponding to other image frames are similar. The time interval between the EOF of the N-1th frame and the SOF of the Nth frame can be called Vblank, that is, the duration between T2 and T4. The frame interval between the N-1th frame and the Nth frame is the time interval between T1 and T4.

[0148] In the embodiment of the present application, the prerequisite for sending the QSC calibration data is that the aperture changes and the sensor starts to flow. That is to say, the sending of the QSC calibration data in the embodiment of the present application includes two stages, the first stage is the stage of adjusting the aperture, and the second stage is the stage of sending the QSC calibration data. Optionally, the second stage may occur after the first stage. Or, optionally, the second stage is carried out synchronously during the process of the first stage. Generally speaking, the aperture has been switched before the QSC calibration data is sent. In short, after these two stages are completed, the image frame output by the sensor matches the latest QSC calibration data and the latest aperture effect. The detailed timing of these two stages is introduced below.

[0149] Phase 1

[0150] Before the timestamp corresponding to the SOF of frame N (i.e., before T4), the sensor receives the exposure and aperture parameters (used to switch the aperture to the first aperture) from the AE module. After receiving these parameters, the sensor sets the exposure parameters starting at the SOF of frame N. Furthermore, the sensor begins adjusting the aperture starting at the EOF of frame N+1 (i.e., T8), which can be understood as switching from the current aperture to the first aperture.

[0151] Among them, the AE module sends the exposure parameters and aperture parameters to the sensor based on the decision or the instruction issued by the camera application, and sends the aperture parameters to the aperture.

[0152] The embodiments of the present application do not limit the object or timing of triggering the aperture change. The aperture can be automatically switched by the electronic device in conjunction with the shooting mode or shooting scene, or the user can actively trigger the aperture switch. For example, when the user is shooting in professional shooting mode, the user actively changes the aperture, triggering the aperture adjustment instruction.

[0153] Optionally, in a scenario where the user triggers the aperture switching, the timestamp corresponding to the user triggering the aperture switching may be T2-1 in FIG5 , or a timestamp earlier than T2-1, which is not specifically limited.

[0154] Optionally, the corresponding timestamp when the AE module decides the exposure parameter and the aperture parameter may be T2-2.

[0155] After receiving the exposure parameters, the sensor performs exposure based on the exposure parameters, starting at a timestamp before the SOF of frame N+2 and after the SOF of frame N+1 (e.g., T7 in Figure 5 ). For example, the sensor performs exposure based on the exposure parameters from T9 until T11, and outputs the exposure result at frame N+2. However, since the present embodiment discards frame N+2, it can be assumed that the exposure result is not output until the frame following frame N+2 (i.e., frame N+3). For details on the exposure parameter validation mechanism, please refer to the relevant technical description and will not be repeated here.

[0156] Since the aperture adjustment starts at the EOF (i.e., T8) of the N+1th frame and the aperture adjustment takes a certain amount of time, the aperture adjustment process will affect the image brightness of the N+2th frame. Therefore, the image data corresponding to the N+2th frame needs to be discarded (or skipped), that is, the image corresponding to the N+2th frame is not displayed.

[0157] Optionally, the image corresponding to frame N+1 can be displayed during the display duration of frame N+2. That is, because frame N+2 is the frame detected by the sensor when the aperture changes, its image brightness may be abnormal, or its exposure may be affected. To avoid presenting an image frame with abnormal brightness to the user, the last normal image frame before the aperture adjustment (i.e., frame N+1) can be displayed.

[0158] Phase II

[0159] The sensor starts writing the QSC calibration data corresponding to the first aperture at the SOF of the N+2th frame. For example, the sensor starts writing the QSC calibration data corresponding to the first aperture at timestamp T10.

[0160] It should be noted that to minimize the impact of writing the QSC calibration data on image quality, it is possible to consider writing the QSC calibration data corresponding to the first aperture during the frame time to avoid affecting image quality. The frame time here can specifically be the interval between the SOF of the N+2th frame and the SOF of the N+3th frame.

[0161] Furthermore, since the QSC function (or correction function) is disabled when the calibration data corresponding to the first aperture is written, the newly written QSC calibration data has not yet taken effect. From this perspective, the image quality of frame N+2 will also be affected, so frame N+2 needs to be discarded. As mentioned above, the image corresponding to frame N+1 can be used for display to ensure that the preview image presented to the user is an image frame with appropriate brightness, without affecting the image quality of the displayed image.

[0162] Since writing the QSC calibration data corresponding to the first aperture takes a certain amount of time, in order to reduce the time required to write the QSC calibration data, the embodiment of the present application preprocesses the QSC calibration data to compress the number of bits corresponding to the QSC calibration data.

[0163] In some embodiments, the QSC calibration data corresponding to the first aperture is calibration data obtained by preprocessing original QSC calibration data (or second calibration data, or calibration data before preprocessing).

[0164] The embodiments of the present application do not limit the specific method of preprocessing. In some embodiments, the application scenarios applicable to the embodiments of the present application are related to the zoom ratio (for example, applied to a zoom ratio of 2x or more). Since for scenes with a zoom ratio of 2x or more, the sensor will crop the image data accordingly based on the zoom ratio when outputting the image, in order to minimize the impact of the QSC calibration data on the image frame, the QSC calibration data is preprocessed and only the QSC calibration data corresponding to the cropped image is sent. This ensures the data compensation effect while reducing the time consumed in sending the QSC calibration data.

[0165] For ease of understanding, let's look at the image cropping process when the sensor outputs the image, as shown in Figure 6. As shown in Figure 6, the size of the cropped image is obviously smaller than the size of the full-size image. The image size corresponding to the QSC calibration data before preprocessing. For example, in full-size mode, the image size is 8192*6144, and after cropping, the image size of the cropped center area is 4096*3072. In this way, by preprocessing the QSC calibration data, only the QSC calibration data corresponding to the image of the cropped center area is sent, reducing the time it takes to transmit the QSC calibration data.

[0166] Optionally, the number of bits corresponding to the QSC calibration data corresponding to the first aperture is smaller than the number of bits corresponding to the original QSC calibration data.

[0167] For example, the number of bits corresponding to the original QSC calibration data is 3536 bytes, and the corresponding transmission time is approximately 43 milliseconds. The QSC calibration data corresponding to the first aperture is 1008 bytes, and the corresponding transmission time is approximately 15 milliseconds.

[0168] When the number of bits corresponding to the QSC calibration data corresponding to the first aperture is reduced, the transmission time will also be reduced accordingly. The writing process of the QSC calibration data corresponding to the first aperture occurs between the SOF of the N+2th frame and the SOF of the N+3th frame, and the image corresponding to the affected N+2th frame is discarded.

[0169] Optionally, the transmission duration of the QSC calibration data corresponding to the first aperture is less than a frame interval between the N+2th frame and the N+3th frame.

[0170] For example, the transmission duration of the QSC calibration data corresponding to the first aperture is 15 milliseconds, and the interframe interval between the SOF of the N+2th frame and the SOF of the N+3th frame is 33 milliseconds. It can be seen that the transmission duration of the QSC calibration data corresponding to the first aperture is less than the interframe interval, that is, the QSC calibration data corresponding to the first aperture can be sent within a single image frame, thus only affecting one frame of data.

[0171] After the transmission time of the QSC calibration data corresponding to the first aperture is reduced, the number of affected image frames will also be reduced accordingly, that is, the affected image frames only include the N+2th frame, and will not affect a larger number of image frames, thereby obtaining a better shooting experience.

[0172] After the first and second stages described above, the sensor can take effect on the QSC calibration data corresponding to the first aperture and the effects corresponding to the first aperture when it outputs the N+3th frame, that is, starting from T13. Furthermore, in the N+3th frame, the exposure parameters match the aperture parameters. Since the aperture has been switched to the first aperture and the newly issued QSC calibration data corresponding to the first aperture has also taken effect, the image effect corresponding to the N+3th frame obtained by using the QSC calibration data corresponding to the switched first aperture for compensation is better than that obtained by using the QSC calibration data corresponding to the original aperture (i.e., the aperture before the switch).

[0173] It should be noted that the timing of writing the QSC calibration data in the embodiment of the present application occurs after the sensor starts to flow. As shown in Figure 5, compared to the timing of writing the QSC calibration data before the flow in the related art (or although the camera is turned on, the sensor has not yet produced an image), the QSC calibration data written in the embodiment of the present application occurs after the sensor starts to flow, specifically after the aperture changes. The advantage of this is that the QSC calibration data that matches the switched aperture can be sent down, or the QSC calibration data can be dynamically updated based on the change of the aperture, so that the QSC calibration data corresponds to the switched aperture, thereby achieving better correction or compensation and obtaining better image effects.

[0174] The present embodiment does not specifically limit the storage space where the QSC calibration data (e.g., the QSC calibration data corresponding to the first aperture) is written. Taking the QSC calibration data corresponding to the first aperture as an example, the storage space where the QSC calibration data corresponding to the first aperture is stored can be implemented in the following different ways.

[0175] In one implementation, the QSC calibration data corresponding to the first aperture is stored in the OTP.

[0176] For example, if the QSC calibration data corresponding to the first aperture needs to be written, the QSC calibration data corresponding to the first aperture may be written into the sensor OTP.

[0177] In another implementation, the QSC calibration data corresponding to the first aperture is stored in a non-volatile memory EEPROM.

[0178] For example, if the QSC calibration data corresponding to the first aperture needs to be written, the QSC calibration data corresponding to the first aperture can be written in the EEPROM, and then the QSC calibration data written corresponding to the first aperture can be transferred to the sensor to take effect. The advantage of doing this is that when the OTP space reserved for the sensor is relatively small or even no OTP space is reserved, writing the calibration data into the EEPROM can be an option. In addition, since the sensor OTP itself has a relatively small storage space and is easily damaged, in this case, in order to improve reliability, the QSC calibration data corresponding to the first aperture can be stored by attaching an EEPROM to the camera module.

[0179] In another implementation, the QSC calibration data corresponding to the first aperture is stored in a system configuration file oeminfo.

[0180] For example, when the QSC calibration data corresponding to the first aperture needs to be written, the QSC calibration data corresponding to the first aperture can be stored in oeminfo. Therefore, when the sensor does not reserve OTP space and does not have EEPROM, the first calibration data can also be stored in oeminfo.

[0181] The following describes the method flow for shooting according to an embodiment of the present application in conjunction with the flow shown in FIG7 . As shown in FIG7 , the method includes:

[0182] Step 701: At a first moment, start the camera sensor.

[0183] The first moment can be understood as the moment when the user triggers the camera. For example, the first moment is the moment when the sensor in Figure 5 starts streaming.

[0184] The embodiment of the present application does not specifically limit how to activate the camera sensor. Optionally, in response to a first user operation, the camera sensor is activated at a first moment. After the electronic device detects the operation of activating the camera, the camera can be activated to begin collecting image data.

[0185] This embodiment of the application does not limit the method by which a user triggers the camera. The first operation is an operation for triggering the camera of the electronic device to turn on. The first operation includes, but is not limited to, touch operation, key operation, voice control, etc. This embodiment of the application does not specifically limit the specific form of the first operation.

[0186] Optionally, the first operation is an operation of opening a camera application. For example, as shown in (1) in FIG1A , the first operation is an operation of the user clicking on the camera application to start the camera.

[0187] Step 702: At a second moment, obtain exposure parameters and aperture parameters of the Nth frame; wherein the first moment is before the second moment.

[0188] Optionally, the aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture.

[0189] In some embodiments, the exposure parameters and aperture parameters of the Nth frame are determined by an automatic exposure module, and the exposure parameters and the aperture parameters are cached in the camera driver.

[0190] The embodiment of the present application does not limit the order in which the exposure parameters and the aperture parameters are acquired. In other words, the embodiment of the present application does not limit whether the exposure parameters and the aperture parameters are acquired at the same time.

[0191] At the second moment, obtaining the exposure parameters and aperture parameters of the Nth frame may include the following three situations: (1) obtaining the exposure parameters and aperture parameters at the same time after the parameters are adjusted; (2) obtaining the exposure parameters first, and then obtaining the aperture parameters corresponding to the exposure parameters; (3) obtaining the aperture parameters first, and then obtaining the exposure parameters corresponding to the aperture parameters.

[0192] The embodiments of the present application do not limit the triggering operation for acquiring the exposure parameters and aperture parameters. Optionally, in some embodiments, acquiring the exposure parameters and aperture parameters for the Nth frame includes: acquiring the exposure parameters and aperture parameters in response to a second operation. In other words, the exposure parameters and aperture parameters may be acquired in response to the second operation.

[0193] The second operation can be understood as an operation that triggers the aperture of the electronic device to switch. The embodiment of the present application does not limit the specific form of the second operation.

[0194] Optionally, the second operation is an operation of manually adjusting the aperture by the user. That is, at the second moment, the electronic device can obtain the exposure parameter and the aperture parameter in response to the operation of manually adjusting the aperture by the user.

[0195] For example, the second operation is the user manually adjusting the aperture in the professional photography mode. FIG8A shows an example of an interface for adjusting the aperture in the professional mode. As shown in FIG8A (1), taking the photo preview interface 80 as an example, the interface 80 includes the professional mode. In response to the user clicking the professional mode operation, the mobile phone displays the interface 81 shown in FIG8A (2).

[0196] Interface 81 shown in (2) of FIG8A is a preview interface in professional mode. In professional mode, the relevant parameters of the camera settings can be manually adjusted by the user. As shown in (2) of FIG8A, the display area 801 of the camera interface 81 includes a plurality of camera parameter setting options. Display area 801 includes at least an aperture adjustment control 802, or icon "A". Icon "A" can also be understood as an aperture setting button.

[0197] In response to the user clicking on the aperture adjustment control 802, the mobile phone displays the interface 82 shown in (3) of Figure 8A. The interface 82 includes at least a display area 803. The display area 803 displays the controls for adjusting the aperture. In response to the user performing an aperture adjustment operation in area 803 (e.g., sliding left or right), the mobile phone switches to the corresponding aperture. In the interface shown in Figure 8A, the second operation is the user manually adjusting the aperture in area 803 of the interface 81.

[0198] It is understood that the aperture adjustment interface shown in (3) of FIG8A is merely an example description, and the embodiments of the present application are not limited thereto. For example, the aperture adjustment interface in display area 803 may also be in the form of a dial, and accordingly, the user's operation of adjusting the aperture size may be an operation of adjusting the dial.

[0199] It can also be understood that other camera parameters in the professional mode are also shown in the display area 801. Optionally, the area 801 shown in (2) in Figure 8A also includes the following camera parameters: metering mode (corresponding to the icon "M"), ISO parameter (corresponding to the icon ISO, for example, the ISO value in the figure is 100), shutter speed (corresponding to the icon "S", for example, the shutter speed in the figure is 1 / 40), exposure compensation (corresponding to the icon "EV."), focus mode (corresponding to the icon "AF."), white balance parameter (corresponding to the icon "WB."). For the explanation of the camera parameters, please refer to the description in the relevant technology and will not be repeated here.

[0200] Exemplarily, the second operation is the operation after the user switches the current photo shooting mode to the large aperture shooting mode. Figure 8B shows an example of the interface when the user switches from the normal photo shooting mode to the large aperture mode. As shown in interface 83 (1) in Figure 8B, interface 83 is a photo preview interface. In response to the user sliding to the right in the photo shooting mode area of ​​interface 83, the mobile phone displays interface 84 shown in (2) in Figure 8B. More photo shooting mode options are presented in interface 84, and the photo shooting mode options include at least the large aperture mode. In response to the user clicking on the large aperture mode, the mobile phone enters the large aperture shooting mode and adjusts the aperture accordingly. In the interface shown in Figure 8B, the second operation is the user clicking on the large aperture mode.

[0201] It should be understood that the entrances to the professional mode or large aperture mode shown above are merely examples, and the embodiments of the present application are not limited thereto.

[0202] Based on the interface shown in FIG. 8A or FIG. 8B , the mobile phone can execute aperture control in response to the user's manual aperture adjustment operation, thereby adjusting the aperture based on user needs to assist the user in taking photos with better image quality.

[0203] Alternatively, optionally, the second operation is an operation of manually adjusting the aperture of the electronic device.

[0204] At the second moment, the electronic device can obtain exposure and aperture parameters based on the automatic aperture adjustment operation. For example, if the electronic device is equipped with a variable aperture, after detecting a specific shooting scene, the electronic device will automatically adjust the aperture according to the lighting conditions of the shooting scene. At this time, the electronic device will obtain exposure and aperture parameters.

[0205] Step 703: at a third moment, sending the exposure parameter to the camera sensor; the third moment is a moment before the moment corresponding to the start of frame delimiter SOF of the Nth frame; the second moment is before the third moment.

[0206] It should be noted that after the exposure parameters are sent to the camera sensor, they can take effect from a timestamp before the SOF of frame N+2 and after the SOF of frame N+1 (e.g., T7 in FIG5 ). For example, the exposure parameters take effect from T9 shown in FIG5 , i.e., exposure based on the exposure parameters begins at T9; for another example, the exposure parameters take effect from T8 shown in FIG5 . It should be understood that the times when the exposure parameters take effect shown here are merely exemplary descriptions, and the embodiments of the present application are not limited thereto. In addition, the process of writing the calibration data affects frame N+2, and frame N+2 is discarded (e.g., the image of frame N+1 is displayed during the display duration of frame N+2). Therefore, the brightness of frame N+3 is consistent with the brightness of frame N and the brightness of frame N+1, and the exposure parameters match the aperture parameters in frame N+3. Matching the exposure parameters with the aperture parameters means that the aperture has been successfully switched in frame N+3, the QSC calibration data corresponding to the aperture also matches the aperture, and the exposure parameters have taken effect. Of course, after the aperture is switched successfully, the subsequent AE module can send corresponding exposure parameters based on the actual situation. For example, the exposure parameters corresponding to the switched aperture are sent in the N+3th frame. For the sending mechanism and the effectiveness mechanism of the exposure parameters, please refer to the description of the relevant technology. For the sake of brevity, we will not go into details here.

[0207] In some embodiments, after time T2-2 and before time T4, the camera sensor HAL obtains the aperture and exposure parameters input by the AE module. The camera sensor HAL sends the exposure parameters to the sensor driver. The sensor driver can send the exposure parameters to the hardware (i.e., the camera sensor) for subsequent validation.

[0208] In some embodiments, the camera sensor HAL sends aperture parameters to the aperture HAL. After obtaining the aperture parameters, the aperture HAL can send the aperture parameters to the aperture motor driver. The aperture motor driver can send the aperture parameters to the hardware (aperture component) for validation.

[0209] Exemplarily, the aperture HAL calls an aperture parameter method function, and converts the aperture parameter into a variable aperture parameter value (VA code) through the aperture parameter method function.

[0210] Step 704: At a fourth moment, the aperture component adjusts the aperture based on the aperture parameter. The fourth moment is the moment corresponding to the end-of-frame delimiter EOF of the N+1th frame.

[0211] Exemplarily, the fourth moment is T8 in Figure 5. That is, starting from the EOF of the (N+1)th frame, the aperture is adjusted based on the aperture parameter, that is, the current aperture is switched to the first aperture indicated by the aperture parameter according to the aperture parameter.

[0212] It is understood that adjusting the aperture also takes a certain amount of time. For example, Figure 5 shows the time required for adjusting the aperture. Adjusting the aperture affects frame N+2 but does not affect frame N+3. Frame N+2 can also be called an aperture change frame.

[0213] Step 705: at a fifth moment, the first calibration data is sent to the camera sensor; the fifth moment is the moment corresponding to the SOF of the N+2th frame.

[0214] Optionally, the first calibration data is calibration data corresponding to the first aperture. Accordingly, the aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture. In other words, when the electronic device switches from the current aperture to the first aperture, the first calibration data issued is the calibration data corresponding to the switched first aperture, i.e., the first calibration data.

[0215] During the display duration corresponding to the N+3th frame, the image corresponding to the N+3th frame is displayed based on the exposure parameters, the aperture parameters, and the first calibration data. That is, after the calibration data is delivered, the exposure parameters and aperture parameters of the image of the N+3th frame displayed match. For related descriptions, please refer to the description of FIG. 5 above and will not be repeated here.

[0216] In some embodiments, when the camera sensor of the electronic device is a Quad sensor, the first calibration data is QSC calibration data corresponding to the first aperture.

[0217] For example, the fifth moment is T10 in Figure 5. That is, starting from the SOF of the N+2th frame, the QSC calibration data corresponding to the first aperture is sent to the sensor.

[0218] In some embodiments, the camera sensor HAL sends the QSC calibration data corresponding to the first aperture to the sensor driver. The sensor driver sends the QSC calibration data corresponding to the first aperture to the hardware (i.e., the camera sensor) for subsequent validation.

[0219] In an embodiment of the present application, the electronic device supports switching the calibration data corresponding to the aperture after the sensor starts streaming. In other words, when the aperture changes, the electronic device can correspondingly switch the calibration data corresponding to the aperture, so that image data compensation can be performed based on the appropriate calibration data, thereby improving image quality. Compared to the method of only burning a set of calibration data before the sensor starts streaming, the embodiment of the present application can also achieve calibration data switching after the sensor starts streaming, thereby improving image quality.

[0220] Furthermore, optionally, the first calibration data is preprocessed from the second calibration data, wherein the number of bits corresponding to the second calibration data is greater than the number of bits corresponding to the first calibration data. As previously described, by preprocessing the calibration data to reduce the number of bits corresponding to the calibration data, the transmission time when the calibration data is sent is reduced, thereby minimizing the number of affected image frames.

[0221] For example, after receiving the exposure and aperture parameters before the SOF of frame N, the electronic device begins adjusting the aperture based on the aperture parameters at the EOF of frame N+1 and sends the QSC calibration data for the adjusted aperture at the SOF of frame N+2. This allows the exposure and aperture parameters to match at frame N+3. In this process, only frame N+2 is affected; no further frames are affected.

[0222] Optionally, the transmission duration of the first calibration data is less than the frame interval between the N+2th frame and the N+3th frame. As previously described, the process of writing the first calibration data occurs between the SOF of the N+2th frame and the SOF of the N+3th frame to avoid affecting more image frames.

[0223] Optionally, during the display duration corresponding to the N+3th frame, the image corresponding to the N+3th frame is displayed based on the exposure parameter, the aperture parameter, and the first calibration data. As previously described, starting from the N+3th frame, the exposure parameter and the aperture parameter match, and the image data can be compensated accordingly based on the first calibration data. Therefore, during the display duration corresponding to the N+3th frame, the image corresponding to the N+3th frame can be displayed.

[0224] Optionally, the image corresponding to the N+1th frame is displayed during the display duration of the N+2th frame. That is, the camera sensor discards the N+2th frame when outputting the image. As previously mentioned, the N+2th frame is affected during the writing of the first calibration data. Therefore, the image of the previous frame (for example, the image corresponding to the N+1th frame) can be displayed when the image is sent for display to avoid affecting the user experience.

[0225] The embodiments of the present application are applicable to preview scenes (including photo preview and video preview) with a zoom ratio greater than a preset zoom ratio (e.g., 2x) in a high-dynamic environment or a high-illumination environment. For example, the scene shown in (4) in FIG. 1A . Another example is the scene shown in (4) in FIG. 1B .

[0226] It should be understood that the scenario shown in FIG. 1A or the scenario shown in FIG. 1B is merely an exemplary description, and the embodiments of the present application are not limited thereto.

[0227] The above description, in conjunction with Figures 1A to 8B, details the method for photographing provided by an embodiment of the present application. The following description of an apparatus embodiment of the present application will be made in conjunction with Figures 9 and 10. It should be understood that the apparatus for photographing in an embodiment of the present application can execute the method for photographing in the aforementioned embodiment of the present application. That is, the specific working processes of the various products below can refer to the corresponding processes in the aforementioned method embodiments.

[0228] FIG9 shows a schematic structural diagram of an electronic device 1000 applicable to the present application.

[0229] The electronic device 1000 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0230] Among them, the sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0231] It should be noted that the structure shown in FIG9 does not constitute a specific limitation on the electronic device 1000. In other embodiments of the present application, the electronic device 1000 may include more or fewer components than those shown in FIG9, or the electronic device 1000 may include a combination of some of the components shown in FIG9, or the electronic device 1000 may include sub-components of some of the components shown in FIG9. The components shown in FIG9 may be implemented in hardware, software, or a combination of software and hardware.

[0232] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). The different processing units may be independent devices or integrated devices.

[0233] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0234] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0235] In some embodiments, the processor 110 may include one or more interfaces. For example, the processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and a USB interface.

[0236] 9 is merely a schematic illustration and does not limit the connection relationship between the modules of the electronic device 1000. Optionally, the modules of the electronic device 1000 may also adopt a combination of the multiple connection modes in the above embodiments.

[0237] The charging management module 140 is used to receive power from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive current from the wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive electromagnetic waves through the wireless charging coil of the electronic device 1000 (the current path is shown as a dotted line). While the charging management module 140 is charging the battery 142, it can also power the electronic device 1000 through the power management module 141.

[0238] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (e.g., leakage, impedance). Optionally, the power management module 141 can be provided in the processor 110, or the power management module 141 and the charging management module 140 can be provided in the same device.

[0239] The wireless communication function of the electronic device 1000 can be implemented through components such as the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

[0240] Electronic device 1000 can implement display functions using a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0241] Display screen 194 can be used to display images or videos. Display screen 194 includes a display panel. The display panel can use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a micro OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 1000 may include one or N display screens 194, where N is a positive integer greater than 1.

[0242] The electronic device 1000 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0243] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can perform algorithmic optimization on image noise, brightness, and color. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0244] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard red, green, blue (RGB), YUV or other format. In some embodiments, the electronic device 1000 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0245] In some embodiments, the electronic device 1000 includes a plurality of apertures, wherein the plurality of apertures includes at least a first aperture. Optionally, the camera 193 of the electronic device 1000 includes the plurality of apertures.

[0246] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 1000 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0247] Video codecs are used to compress or decompress digital video. Electronic device 1000 may support one or more video codecs. This allows electronic device 1000 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0248] The electronic device 1000 can implement audio functions, such as music playback and recording, through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0249] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 1000 can adaptively adjust the brightness of display screen 194 based on the sensed ambient light brightness. Ambient light sensor 180L can also be used to automatically adjust white balance during photography.

[0250] In some embodiments, the ambient light sensor 180L can obtain the ambient illuminance of the current shooting environment. When the ambient illuminance is greater than or equal to a first illuminance threshold, the current shooting environment can be considered to be relatively bright, and can be considered to be a high-brightness shooting environment.

[0251] The touch sensor 180K is also referred to as a touch-sensitive device. The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also referred to as a touch screen. The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor 180K can transmit the detected touch operations to an application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 1000 and at a different location from the display screen 194.

[0252] The buttons 190 include a power button and a volume button. The buttons 190 can be mechanical buttons or touch buttons. The electronic device 1000 can receive button input signals and implement functions related to the case input signals.

[0253] Motor 191 can generate vibration. Motor 191 can be used for incoming call reminders or for touch feedback. Motor 191 can produce different vibration feedback effects for touch operations acting on different applications. Motor 191 can also produce different vibration feedback effects for touch operations acting on different areas of the display screen 194. Different application scenarios (for example, time reminders, receiving messages, alarm clocks, and games) can correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0254] In some embodiments, the processor 110 is used to start the camera sensor at a first moment; at a second moment, obtain the exposure parameters and aperture parameters of the Nth frame; wherein the first moment is before the second moment; at a third moment, the exposure parameters are sent to the camera sensor; the third moment is the moment corresponding to the start of frame delimiter SOF of the Nth frame; the second moment is before the third moment; at a fourth moment, the aperture component adjusts the aperture based on the aperture parameters, and the fourth moment is the moment corresponding to the end of frame delimiter EOF of the N+1th frame; at a fifth moment, the first calibration data is sent to the camera sensor; the fifth moment is the moment corresponding to the SOF of the N+2th frame.

[0255] Optionally, in some embodiments, the processor 110 is further used to call the display screen 194 to display the image corresponding to the N+3 frame based on the exposure parameters, the aperture parameters and the first calibration data within the display time corresponding to the N+3 frame.

[0256] Optionally, in some embodiments, the processor 110 is further configured to call the display screen 194 to display the image corresponding to the N+1th frame within the display duration of the N+2th frame.

[0257] It can be understood that the method for shooting in the embodiment of the present application can be applied to the electronic device shown in Figure 9. The specific implementation steps can be referred to the introduction of the method embodiment above and will not be repeated here.

[0258] FIG10 is a schematic block diagram of an apparatus 800 for photographing according to an embodiment of the present application. It should be understood that the apparatus 800 can execute the photographing methods shown in FIG4 to FIG7.

[0259] As shown in FIG10 , the apparatus 800 includes an input unit 810 and a processing unit 820 . The apparatus 800 may be an electronic device. Optionally, the apparatus 800 further includes a display unit 830 .

[0260] In some embodiments, the processing unit 820 is used to start the camera sensor at a first moment; obtain the exposure parameters and aperture parameters of the Nth frame at a second moment; wherein the first moment is before the second moment; at a third moment, the exposure parameters are sent to the camera sensor; the third moment is the moment corresponding to the start of frame delimiter SOF of the Nth frame; the second moment is before the third moment; at a fourth moment, the aperture component adjusts the aperture based on the aperture parameters, and the fourth moment is the moment corresponding to the end of frame delimiter EOF of the N+1th frame; at a fifth moment, the first calibration data is sent to the camera sensor; the fifth moment is the moment corresponding to the SOF of the N+2th frame.

[0261] In some embodiments, the processing unit 820 is configured to activate the camera sensor, including: activating the camera sensor in response to a first operation of the user.

[0262] Optionally, the input unit 810 is used to detect a first operation of the user.

[0263] Optionally, the first operation is an operation of opening a camera application.

[0264] In some embodiments, the processing unit 820 is configured to obtain the exposure parameters and aperture parameters of the Nth frame, including:

[0265] In response to a second operation, the exposure parameter and the aperture parameter are acquired.

[0266] Optionally, the second operation is an operation of manually adjusting the aperture by a user, or the second operation is an operation of automatically adjusting the aperture by the electronic device.

[0267] Optionally, the input unit 810 is used to detect a user's manual operation of adjusting the aperture.

[0268] In some embodiments, the display unit 830 is configured to display the image corresponding to the N+3th frame based on the exposure parameter, the aperture parameter, and the first calibration data within a display duration corresponding to the N+3th frame.

[0269] In some embodiments, the aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture.

[0270] In some embodiments, the first calibration data is calibration data corresponding to the first aperture.

[0271] Optionally, the device 800 includes a plurality of apertures, and the plurality of apertures includes at least a first aperture.

[0272] In some embodiments, the first calibration data is calibration data obtained by preprocessing the second calibration data, wherein the number of bits corresponding to the second calibration data is greater than the number of bits corresponding to the first calibration data.

[0273] In some embodiments, the transmission duration of the first calibration data is less than the frame interval between the N+2th frame and the N+3th frame.

[0274] In some embodiments, the display unit 830 is further configured to display the image corresponding to the N+1th frame within the display duration of the N+2th frame.

[0275] In some embodiments, the display unit 830 is also used to display a first preview interface, which includes a zoom ratio option; the processing unit 820 is used to respond to the user's operation of adjusting the zoom ratio to a first zoom ratio based on the zoom ratio option, and, upon detecting that the current shooting environment is a high-illumination environment, call the camera sensor to adopt a full-size cropping mode; wherein, the first preview interface is a photo preview interface or a video preview interface; the first zoom ratio is greater than a preset zoom ratio; the high-illumination environment includes: the ambient illumination of the current shooting environment is greater than a first illumination threshold.

[0276] In some embodiments, the display unit 830 is also used to display a second preview interface, which includes a zoom ratio option; the processing unit 820 is used to respond to the user's operation of adjusting the zoom ratio to a second zoom ratio based on the zoom ratio option, and, upon detecting that the current shooting environment is a high dynamic environment, call the camera sensor to adopt a full-size cropping mode; wherein, the second preview interface is a photo preview interface or a video preview interface; the second zoom ratio is greater than a preset zoom ratio; the high dynamic environment includes: the dynamic range value satisfies the dynamic range DR constraint condition.

[0277] In one possible example, the input unit 810 and the processing unit 820 can be implemented by a processor or a processing unit. The display unit 830 can be implemented by a display screen or a display unit. It should be understood that the above-mentioned device 800 is embodied in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and this embodiment of the application does not specifically limit this.

[0278] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) that executes one or more software or firmware programs and a memory, an integrated logic circuit, and / or other suitable devices that can provide the above functions. In a simple embodiment, those skilled in the art will appreciate that the device 800 may take the form shown in Figure 9.

[0279] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0280] The present application also provides a computer program product, which, when executed by a processor, implements the method described in any method embodiment of the present application.

[0281] The computer program product can be stored in a memory and finally converted into an executable target file that can be executed by a processor through preprocessing, compilation, assembly and linking.

[0282] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, implements the method described in any method embodiment of the present application. The computer program can be a high-level language program or an executable target program.

[0283] The computer-readable storage medium may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0284] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and equipment and the technical effects produced can refer to the corresponding processes and technical effects in the aforementioned method embodiments, and will not be repeated here.

[0285] In the several embodiments provided in this application, the disclosed systems, devices and methods can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not executed. The device embodiments described above are merely schematic, and the division of units is only a logical function division. There may be other division methods in actual implementation, and multiple units or components may be combined or integrated into another system. In addition, the coupling between the units or the coupling between the components may be direct coupling or indirect coupling, and the above coupling includes electrical, mechanical or other forms of connection.

[0286] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0287] Additionally, the terms "system" and "network" are often used interchangeably. The term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the related objects.

[0288] The terms (or numbers) "first", "second", ... etc. that appear in the embodiments of the present application are for descriptive purposes only, that is, they are only for distinguishing different objects, such as different "calibration data", etc., and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", ... etc. may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, "at least one (item)" refers to one or more. "Multiple" means two or more. "At least one of the following (item)" or similar expressions refers to any combination of these items, including any combination of a single (item) or plural (items).

[0289] For example, the meaning of expressions similar to "the item includes at least one of the following: A, B, and C" in the embodiments of the present application, unless otherwise specified, generally means that the item can be any one of the following: A; B; C; A and B; A and C; B and C; A, B and C; A and A; A, A and A; A, A and B; A, A and C, A, B and B; A, C and C; B and B, B, B and B, B, B and C, C and C; C, C and C, and other combinations of A, B and C. The above examples use A, B, and C as an example to illustrate the optional items of the item. When the expression is "the item includes at least one of the following: A, B, ..., and X", that is, when the expression has more elements, the items that can be applied to the item can also be obtained according to the above rules.

[0290] In short, the above description is only a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included in the scope of protection of this application.

Claims

1. A method for photographing, characterized in that: The method is applied to an electronic device, and the method comprises: At the first moment, the camera sensor is activated; At a second moment, acquiring exposure parameters and aperture parameters of the Nth frame; wherein the first moment is before the second moment; At a third moment, the exposure parameter is sent to the camera sensor; the third moment is a moment corresponding to the start of frame delimiter SOF of the Nth frame; the second moment is before the third moment; At a fourth moment, the aperture component adjusts the aperture based on the aperture parameter, and the fourth moment is a moment corresponding to the end-of-frame delimiter EOF of the N+1th frame; At a fifth moment, the first calibration data is sent to the camera sensor; the fifth moment is the moment corresponding to the SOF of the N+2th frame.

2. The method according to claim 1, characterized in that The starting the camera sensor comprises: In response to a first operation of a user, the camera sensor is activated.

3. The method according to claim 2, characterized in that The first operation is an operation of opening a camera application.

4. The method according to any one of claims 1 to 3, characterized in that The step of obtaining the exposure parameter and aperture parameter of the Nth frame includes: In response to the second operation, the exposure parameter and the aperture parameter are acquired.

5. The method according to claim 4, characterized in that The second operation is an operation of manually adjusting the aperture by a user, or the second operation is an operation of automatically adjusting the aperture by the electronic device.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Within the display time corresponding to the N+3th frame, the image corresponding to the N+3th frame is displayed based on the exposure parameter, the aperture parameter and the first calibration data.

7. The method according to any one of claims 1 to 6, characterized in that The first calibration data is calibration data corresponding to the first aperture.

8. The method according to any one of claims 1 to 7, characterized in that The aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture.

9. The method according to any one of claims 1 to 8, characterized in that The electronic device includes a plurality of apertures, and the plurality of apertures includes at least a first aperture.

10. The method according to any one of claims 1 to 9, characterized in that The first calibration data is calibration data obtained by preprocessing the second calibration data, wherein the number of bits corresponding to the second calibration data is greater than the number of bits corresponding to the first calibration data.

11. The method according to any one of claims 1 to 10, characterized in that The transmission duration of the first calibration data is less than the frame interval between the N+2th frame and the N+3th frame.

12. The method according to any one of claims 1 to 11, characterized in that The method further comprises: During the display duration of the N+2th frame, the image corresponding to the N+1th frame is displayed.

13. The method according to any one of claims 1 to 12, characterized in that After starting the camera sensor, the method further includes: Displaying a first preview interface, wherein the first preview interface includes a zoom ratio option; In response to a user adjusting the zoom ratio to a first zoom ratio based on the zoom ratio option, and detecting that the current shooting environment is a high-illuminance environment, the camera sensor adopts a full-size cropping mode; Among them, the first preview interface is a photo preview interface or a video preview interface; the first zoom ratio is greater than a preset zoom ratio; the high illumination environment includes: the ambient illumination of the current shooting environment is greater than a first illumination threshold.

14. The method according to any one of claims 1 to 12, characterized in that After starting the camera sensor, the method further includes: Displaying a second preview interface, wherein the second preview interface includes a zoom ratio option; In response to the user adjusting the zoom ratio to a second zoom ratio based on the zoom ratio option, and detecting that the current shooting environment is a high-dynamic environment, the camera sensor adopts a full-size cropping mode; Among them, the second preview interface is a photo preview interface or a video preview interface; the second zoom ratio is greater than a preset zoom ratio; the high dynamic environment includes: the dynamic range value satisfies the dynamic range DR constraint condition.

15. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the processor and the memory are coupled, and the memory is used to store a computer program. When the computer program is executed by the processor, the electronic device executes the method according to any one of claims 1 to 14.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the electronic device executes the method according to any one of claims 1 to 14.

17. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method as described in any one of claims 1 to 14.

Images