Shooting method and electronic equipment

Abstract

The invention discloses a shooting method and electronic equipment, and is applied to the technical field of terminals. The method comprises the following steps: starting a camera sensor at a first moment; at a second moment, obtaining an exposure parameter and an aperture parameter of the Nth frame; at a third moment, the exposure parameters are issued to a camera sensor; at the fourth moment, the aperture assembly adjusts the aperture based on the aperture parameters; and issuing the first calibration data to the camera sensor at a fifth moment. According to the embodiment of the invention, the calibration data can be issued after the camera sensor starts stream, the image quality is improved, and the shooting experience of a user can be improved.

Description

A method and electronic device for taking pictures

[0001] This application claims priority to Chinese Patent Application No. 202311637550.X, filed with the State Intellectual Property Office of China on November 30, 2023, entitled "A Method and Electronic Device for Taking Pictures", the entire contents of which are incorporated herein by reference.

[0002] This application relates to the field of image processing technology, and more specifically, to a method and electronic device for taking pictures.

[0003] With the rapid development of smart devices, camera functionality has become an essential feature. Users' demands and experiences regarding camera (photos and / or videos) on smart devices are constantly increasing. When shooting, smart devices control the degree of background blur, or bokeh effect, by adjusting the camera's aperture. The aperture is a device used to control the amount of light entering the image. Generally speaking, a larger aperture results in a shallower depth of field, and consequently, a more pronounced bokeh effect; a smaller aperture results in less light entering the image, and consequently, a deeper depth of field, and consequently, a weaker bokeh effect.

[0004] To achieve better image quality, smart devices use calibration data pre-stored in registers to correct the image data from the camera sensor. However, this method is not very effective for smart devices with variable apertures, thus affecting image quality.

[0005]

[0006] In view of this, this application provides a method, electronic device, computer-readable storage medium, and computer program product for shooting, which can switch calibration data after the aperture changes, improve image quality, and help improve the user's shooting experience.

[0007] In a first aspect, a method for taking pictures is provided, the method being applied to an electronic device, the method comprising:

[0008] The camera sensor is activated immediately.

[0009] At the second moment, the exposure parameters and aperture parameters of the Nth frame are obtained; wherein, the first moment is before the second moment;

[0010] At the third moment, the exposure parameters are sent to the camera sensor; the third moment is the moment corresponding to the frame start delimiter (SOF) of the Nth frame; the second moment is before the third moment.

[0011] At the fourth moment, the aperture assembly adjusts the aperture based on the aperture parameters, and the fourth moment is the moment corresponding to the end-of-frame (EOF) delimiter of the (N+1)th frame.

[0012] At the fifth moment, the first calibration data will be sent to the camera sensor; the fifth moment is the moment corresponding to the SOF of the N+2th frame.

[0013] The above method can be executed by an electronic device or a chip within the electronic device. Based on the above scheme, the electronic device activates the camera sensor at the first moment, adjusts the aperture based on the aperture parameters after receiving them, and sends the first calibration data to the camera sensor after starting aperture adjustment. Compared to the method of burning a set of calibration data before the camera sensor starts up, this application embodiment supports sending calibration data after the camera sensor starts up, improving image quality and helping to enhance the user's shooting experience.

[0014] In one possible implementation, the electronic device includes a plurality of apertures, at least one of which is a first aperture. Alternatively, embodiments of this application are applied to electronic devices equipped with variable apertures.

[0015] This application applies to scenarios where the camera sensor uses a quad Bayer coding (QBC) array for image output, or more specifically, it is suitable for quad sensors. The pixel arrangement of image data output by a quad sensor differs from that of image data output by a traditional sensor. For image data output by a quad sensor, there are brightness differences (or light sensitivity differences) between sub-pixels of the same color, thus requiring quad Bayer coding sensitivity correction (QSC) calibration. This is because without QSC correction (or QSC calibration), the image data output by the quad sensor will exhibit uneven brightness, resulting in poor image quality.

[0016] It is understood that the embodiments of this application are applicable to scenarios where users use electronic devices to take photos or videos.

[0017] In one possible implementation, the electronic device activates the camera sensor in response to the user's first action.

[0018] Optionally, the first operation is to open a camera application. The electronic device has a camera application installed.

[0019] The embodiments of this application do not limit the specific content of the first operation. The first operation can be understood as opening the camera application in any way. The first operation includes, but is not limited to, any of the following forms: via voice command, physical button, and UI interaction, etc.

[0020] For example, the first operation is the user clicking on the camera application in the interface of the electronic device.

[0021] In one possible implementation, the aperture parameter is used to instruct the camera sensor to switch the current aperture to a first aperture.

[0022] The aforementioned exposure and aperture parameters are obtained from the parameters issued by the automatic exposure module. It should be noted that this application does not limit whether the exposure and aperture parameters are obtained at the same time, or in other words, it does not limit the order in which they are obtained. For example, after adjusting the parameters, both the exposure and aperture parameters can be obtained simultaneously. Alternatively, after adjusting the parameters, the exposure parameters can be obtained first, followed by the corresponding aperture parameters. Or, after adjusting the parameters, the aperture parameters can be obtained first, followed by the corresponding exposure parameters.

[0023] In one possible implementation, the exposure and aperture parameters of the Nth frame are obtained, including:

[0024] In response to the second operation, the exposure parameters and the aperture parameters are acquired.

[0025] The exposure and aperture parameters mentioned above may be acquired in response to the second operation. This application embodiment does not limit the specific form of the second operation. The second operation can be understood as an operation that triggers a switch in the aperture of the electronic device.

[0026] Optionally, the second operation is a user manually adjusting the aperture, or the second operation is the electronic device automatically adjusting the aperture.

[0027] In other words, at the second moment, the electronic device can respond to the user's manual aperture adjustment and acquire exposure and aperture parameters. For example, the second operation is the user manually adjusting the aperture in professional shooting mode. Or, the second operation is the user switching from the current normal shooting mode to a large aperture shooting mode.

[0028] Alternatively, at a second moment, the electronic device can acquire exposure and aperture parameters based on automatic aperture adjustment. For example, the electronic device may be equipped with a variable aperture; upon detecting a specific shooting scene, the electronic device will automatically adjust the variable aperture, at which point it will acquire the exposure and aperture parameters.

[0029] In one possible implementation, within the display duration corresponding to the (N+3)th frame, the image corresponding to the (N+3)th frame is displayed based on the exposure parameters, the aperture parameters, and the first calibration data. That is, after the calibration data is sent, the exposure parameters and aperture parameters of the displayed image in the (N+3)th frame are matched. Since the aperture has been switched to the first aperture, and the first QSC calibration data has taken effect, the image quality of the obtained image corresponding to the (N+3)th frame is better than the image quality obtained using the QSC calibration data corresponding to the original aperture. In other words, compared to using the QSC calibration data corresponding to the original aperture (i.e., the aperture before the switch) for compensation, using the QSC calibration data corresponding to the first aperture after the switch for compensation results in better image quality.

[0030] In one possible implementation, the first calibration data is the calibration data corresponding to the first aperture. For example, when the camera sensor is a quad sensor, the first calibration data corresponding to the first aperture is QSC calibration data. Thus, after switching to the first aperture, using the QSC calibration data corresponding to the first aperture can compensate for the image data output by the quad sensor, improving image quality.

[0031] In one possible implementation, the aperture parameter is used to instruct the camera sensor to switch the current aperture to a first aperture. Using the aperture parameter, the aperture assembly switches the current aperture to the aperture corresponding to the aperture parameter, such as the first aperture, to achieve aperture switching.

[0032] Since transmitting the first calibration data requires a certain transmission time, in order to minimize the impact on image frames during the transmission process, this embodiment of the application can preprocess the calibration data, i.e., the first calibration data transmitted is the preprocessed calibration data. After preprocessing, the number of bits occupied by the corresponding data is reduced, thereby reducing the transmission time.

[0033] In one possible implementation, the first calibration data is calibration data obtained by preprocessing the second calibration data, wherein the number of bits corresponding to the second calibration data is greater than the number of bits corresponding to the first calibration data.

[0034] This application does not limit the specific method of preprocessing in its embodiments. One possible implementation is that, since a portion of the image area is cropped in high zoom shooting scenarios, the QSC calibration data can be cropped accordingly. This reduces the amount of QSC calibration data and the number of affected image frames while ensuring the data compensation effect.

[0035] In one possible implementation, the transmission duration of the first calibration data is less than the frame interval between the (N+2)th frame and the (N+3)th frame.

[0036] In one possible implementation, the method further includes:

[0037] During the display duration of the (N+2)th frame, the image corresponding to the (N+1)th frame is displayed.

[0038] Since the process of sending the first calibration data affects the image of the N+2 frame, causing abnormal brightness in the image of the N+2 frame, and the image of the N+2 frame is not QSC compensated, the image of the N+2 frame needs to be discarded when sending the data for display. At this time, the image corresponding to the N+1 frame can be displayed to minimize the impact caused by the failure to display the image of the N+2 frame.

[0039] In one possible implementation, activating the camera sensor includes:

[0040] In response to receiving a first operation from the user, the camera sensor is activated, the first operation being used to turn on the camera.

[0041] The embodiments of this application can be applied to shooting scenarios in high zoom levels (including photo preview or video preview) and in high-light environments.

[0042] In one possible implementation, after activating the camera sensor, the method further includes:

[0043] Display a first preview interface, which includes zoom magnification options;

[0044] In response to the user's operation of adjusting the zoom ratio to the first zoom ratio based on the zoom ratio option, and upon detecting that the current shooting environment is a high-light environment, the camera sensor adopts a full-size cropping output mode;

[0045] Wherein, the first preview interface is a photo preview interface or a video preview interface; the first zoom ratio is greater than a preset zoom ratio; the high illumination environment includes: the ambient illumination of the current shooting environment is greater than a first illumination threshold.

[0046] The embodiments of this application can be applied to shooting scenarios in high zoom ratio preview scenarios (including photo preview or video preview) and shooting environments with high dynamic range.

[0047] In one possible implementation, after activating the camera sensor, the method further includes:

[0048] Display a second preview interface, which includes zoom magnification options;

[0049] In response to the user's operation of adjusting the zoom ratio to the second zoom ratio based on the zoom ratio option, and upon detecting that the current shooting environment is a high dynamic range environment, the camera sensor adopts a full-size cropping output mode;

[0050] The second preview interface is either a photo preview interface or a video preview interface; the second zoom ratio is greater than the preset zoom ratio; the high dynamic environment includes: the dynamic range value satisfies the dynamic range (DR) constraint condition.

[0051] It should be noted that in all the above shooting scenarios, the sensor uses full-size crop mode for image output. Therefore, if the aperture changes in these scenarios, QSC calibration is required to improve image quality.

[0052] In a second aspect, an electronic device is provided, including a unit for performing any of the methods in the first aspect. The electronic device may be a terminal or a chip within a terminal. The electronic device includes a communication unit, a display unit, and a processing unit.

[0053] When the electronic device is a terminal, the processing unit may be a processor, the communication unit may be a communication interface, and the display unit may be a graphics processing module and a screen; the terminal may also include a memory for storing computer program code, which, when the processor executes the computer program code stored in the memory, causes the terminal to perform any of the methods in the first aspect.

[0054] When the electronic device is a chip within a terminal, the processing unit can be a logic processing unit inside the chip, the communication unit can be a communication interface, pins, or circuits, and the display unit can be a graphics processing unit inside the chip. The chip may also include a memory, which can be memory within the chip (e.g., registers, caches, etc.) or memory located outside the chip (e.g., read-only memory, random access memory, etc.). The memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, it causes the chip to execute any of the methods in the first aspect.

[0055] Thirdly, a computer-readable storage medium is provided that stores computer program code, which, when executed by an electronic device, causes the electronic device to perform any of the methods in the first aspect.

[0056] Fourthly, a computer program product is provided, the computer program product comprising: computer program code, which, when executed by an electronic device, causes the electronic device to perform any of the methods in the first aspect.

[0057] Figure 1A is an example diagram of an application scenario of an embodiment of this application;

[0058] Figure 1B is another example diagram of an application scenario of this application embodiment;

[0059] Figure 2A is a schematic diagram of the pixel array distribution of QBC;

[0060] Figure 2B is a schematic diagram comparing the results before and after QSC correction;

[0061] Figure 3 is a schematic diagram of a software architecture applied in an embodiment of this application;

[0062] Figure 4 is a timing interaction diagram applied to the software architecture in Figure 3;

[0063] Figure 5 is a timing example diagram of the distribution of QSC calibration data according to an embodiment of this application;

[0064] Figure 6 is a schematic diagram comparing the preprocessing of QSC correction data;

[0065] Figure 7 is a flowchart illustrating a method for taking pictures according to an embodiment of this application;

[0066] Figure 8A is an example of a user interface for adjusting the aperture according to an embodiment of this application;

[0067] Figure 8B is another example of a user interface for adjusting the aperture according to an embodiment of this application;

[0068] Figure 9 is a schematic diagram of the structure of an electronic device applicable to this application;

[0069] Figure 10 is a schematic block diagram of a shooting device according to an embodiment of this application.

[0070] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0071] In the embodiments of this application, unless otherwise stated, "multiple" can mean two or more.

[0072] The embodiments of this application are applicable to electronic devices, which may be mobile phones, smart screens, tablets, wearable electronic devices, in-vehicle electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), projectors, etc.

[0073] This application applies to electronic devices including a camera sensor and multiple apertures. These multiple apertures can possess different characteristics. By setting multiple apertures, different apertures correspond to different amounts of light entering the device (i.e., the amount of light entering the camera body). Thus, when the electronic device switches to different apertures, it can achieve different depth-of-field effects, or bokeh effects, based on the different amounts of light entering the camera. In other words, compared to electronic devices with a single aperture that cannot adjust the amount of light entering the camera, the advantage of setting multiple apertures is that the amount of light entering the camera can be dynamically adjusted, optimizing the depth-of-field effect. Furthermore, while electronic devices with a fixed aperture can adjust aperture parameters, these parameters are simulated through an internal algorithm, which is less effective than adjusting the physical aperture (i.e., switching between different apertures). The advantages of setting multiple apertures in electronic devices are described below in different scenarios.

[0074] In some embodiments, during the shooting process, the electronic device can select an appropriate aperture based on the distance to the target object, thereby ensuring a bokeh effect in the photo. Different depths of field affect the bokeh effect on the background, resulting in different bokeh effects in the image.

[0075] In some embodiments, in low-light shooting environments (e.g., nighttime scenes), electronic devices can switch to a large aperture to ensure sufficient light intake, thereby presenting a better preview effect in nighttime shooting scenarios and making the preview image of the target object clearer.

[0076] For clarity, low-light environments can be understood as dimly lit environments. Low-light environments include those where the ambient illuminance of the shooting environment is less than a preset brightness threshold. Conversely, high-light environments include those where the ambient illuminance of the shooting environment is greater than or equal to a preset brightness threshold. High-light environments can also be understood as bright environments.

[0077] Ambient illuminance refers to the intensity of light in the shooting environment in which the user is taking a picture. The value of ambient illuminance can be represented by the following indicators: lighting value (LV), lux, or luxindex, etc.

[0078] LV is used to estimate ambient light, and its specific calculation formula is as follows:

[0079] Where Exposure is the exposure time, Aperture is the aperture size, Iso is the ISO, and Luma is the average value of Y in the XYZ color space.

[0080] For example, ambient illuminance is represented by LV, and correspondingly, the luminance threshold is the LV threshold. When the ambient illuminance of the shooting environment is greater than the LV threshold, the current shooting environment is a high-illuminance environment.

[0081] Optionally, in some implementations, the ambient light level of the current shooting environment can be obtained through an ambient light sensor, and then it can be determined whether the current environment is a high-light environment based on the obtained ambient light level.

[0082] In some embodiments, in professional shooting mode, the electronic device adjusts the aperture accordingly in response to the user's manual adjustment of the aperture parameters, thereby providing the user with a richer shooting experience.

[0083] Optionally, the aperture assembly of the electronic device includes at least a first aperture and a second aperture, wherein the amount of light entering the device corresponding to the first aperture is different from the amount of light entering the device corresponding to the second aperture. For example, the first aperture is a large aperture, and the second aperture is a small aperture; the amount of light entering the device corresponding to the first aperture is greater than the amount of light entering the device corresponding to the second aperture. In the embodiments of this application, each aperture has corresponding calibration data. For example, the first aperture corresponds to first calibration data, and the second aperture corresponds to second calibration data. The meaning of the calibration data will be explained later in Figure 4.

[0084] The preceding text describes different scenarios for setting up electronic devices with multiple apertures. The following describes different shooting scenarios to which the embodiments of this application are applicable.

[0085] In some embodiments, in the photo preview mode, for shooting scenes with a zoom ratio of 2x or higher (e.g., 3x) and in high-light environments, the sensor uses a full-size crop mode to output the image.

[0086] In other embodiments, in the photo preview mode, for shooting scenes with a zoom ratio of 2x or higher (e.g., 3x) and high dynamic range, the sensor uses a full-size crop stagger mode to output the image. The image output method of the full-size crop stagger mode can be found in the descriptions in related technologies, and will not be elaborated here.

[0087] To clarify, the dynamic range of a shooting scene can be categorized into high dynamic range and low dynamic range based on preset conditions. For example, if the dynamic range value of the shooting scene meets the dynamic range (DR) constraint conditions, it is considered high dynamic range; if the dynamic range value of the shooting scene does not meet the DR constraint conditions, it is considered low dynamic range.

[0088] Optionally, the DR constraints can be determined based on the histogram of the RAW image of the scene. Specifically, the dynamic range of the scene is determined based on the percentage of overexposed pixels and the percentage of underexposed pixels in the image.

[0089] It should be understood that the above-described process for classifying high dynamic range is merely an exemplary description, and the embodiments of this application are not limited thereto. In fact, it is also possible to determine whether a scenario is a high dynamic range scenario based on other methods in the art.

[0090] For ease of understanding, the following explanation uses the photo preview application scenario example in Figure 1A as an example. This application embodiment does not limit the specific type of electronic device. The following description uses a mobile phone as an example to illustrate the method for taking photos according to this application embodiment.

[0091] Figure 1A is an example of an application scenario for a high-illuminance environment photo preview according to an embodiment of this application. As shown in Figure 1A (1), the mobile phone interface can display multiple applications: application 1, application 2, ..., application 7 and camera application. The mobile phone responds to the user's click on the camera application and launches the camera application. After the camera application runs, the mobile phone interface displays the interface shown in Figure 1A (2). The interface shown in Figure 1A (2) can be called the camera's photo preview interface. The photo preview interface can include a viewfinder 11, a light source 17, a zoom ratio 12 (default is 1x), an album icon 13, a shooting control 14, and a camera rotation control, etc. The brightness of the light source 17 can determine whether the current shooting environment is an illumination environment. For example, when the brightness of the light source 17 is high, making the ambient illumination of the current shooting environment greater than a preset brightness threshold, the current scene can be determined to be a high-illuminance scene (or a high-brightness scene).

[0092] The phone can take a picture in response to the user tapping the camera control 14. The album icon 13 displays thumbnails of the photos. The camera rotation control can be used to switch cameras. The viewfinder 11 is used to capture a preview image, which can be displayed in real time.

[0093] In the photo preview scenario, the mobile phone supports digital zoom. When using the photo function, the user can select different zoom ratios by operating on the touch screen. As an example, as shown in Figure 1A (2), the user clicks the zoom ratio 12 in Figure 1A (2), and the interface displays the interface shown in Figure 1A (3), with zoom ratio selection option 15 appearing (for example, the highest zoom ratio is 8x, and the lowest zoom ratio is 1x). The user drags the zoom ratio 12 upward in the selection option 15, and releases it when the zoom ratio is 2x. The interface displays as shown in Figure 1A (4), that is, the zoom ratio is selected as 2x. It can be seen that the display area of ​​the light source 17 shown in Figure 1A (4) in the viewfinder 11 also becomes larger. Of course, after selecting the zoom ratio, the zoom ratio selection option 15 can be hidden, that is, the interface displays the selected zoom ratio as 2x.

[0094] Of course, the above example uses a 2x zoom level selected in the photo preview scene, and the embodiments of this application are not limited to this. For example, users can also adjust the zoom level to 2x or higher.

[0095] It should be understood that Figure 1A (2)-(4) shows a schematic diagram of an interface for a user to take a photo in portrait mode on a mobile phone, but this application is not limited to this. For example, a user can take a photo in landscape mode on a mobile phone.

[0096] In some other embodiments, in video preview mode, for shooting scenes with a zoom ratio of 2x or higher (e.g., 3x) and in high-light environments, the sensor uses a full-size crop mode to output the image.

[0097] Cropping refers to cropping the acquired image to obtain a field of view corresponding to the target zoom level. A description of high-light environments can be found above; for brevity, it will not be repeated here.

[0098] For ease of understanding, the following is an example diagram of a high-light environment in the video preview scenario, with reference to Figure 1B. As shown in the photo preview interface in Figure 1B (1), the mobile phone responds to the user's operation of clicking the video control and displays the video preview interface, such as the video preview interface 181 shown in Figure 1B (2).

[0099] As shown in Figure 1B(2), the video preview interface 181 includes an album icon 13, a video control 16, a light source 17, and a zoom ratio 12 (default is 1x). The light source 17 indicates that the current video preview scene is a highlighted scene.

[0100] Similarly, in the video preview scenario, the mobile phone can also support digital zoom. When using the video recording function, the user can select different zoom levels by operating on the touch screen. As an example, as shown in Figure 1B (2), in response to the user's operation of clicking the zoom level 12 in Figure 1B (2), the mobile phone displays interface 182 as shown in Figure 1B (3), which includes zoom level selection option 15 (for example, the highest zoom level is 8x and the lowest zoom level is 1x). In response to the user's operation of dragging the zoom level 12 upward in selection option 15 and releasing the finger when the zoom level is 2x, the mobile phone displays interface 183 as shown in Figure 1B (4), that is, the zoom level is selected as 2x. Of course, after selecting the zoom level, the zoom level selection option 15 can be hidden, that is, the selected zoom level is displayed on the interface as 2x.

[0101] Of course, the above example uses a 2x zoom level selected in a video preview scenario, and the embodiments of this application are not limited to this. For example, users can also adjust the zoom level to 2x or higher.

[0102] It should be understood that the interfaces shown in (2)-(4) of Figure 1B can be an example interface of an application scenario of this application embodiment.

[0103] In other embodiments, in video preview mode, for shooting scenes with a zoom ratio of 2x or higher (e.g., 3x) and high dynamic range, the sensor uses a full-size crop stagger mode to output images.

[0104] For a description of high dynamic range, please refer to the previous text. For the sake of brevity, it will not be repeated here.

[0105] This application's embodiments are applicable to scenarios where the sensor uses a quad Bayer array (QBC) for image output, or in other words, applicable to quad sensors. The pixel arrangement of the image data output by a quad sensor differs from that of image data output by a traditional sensor. The image data output by a quad sensor consists of 4 pixels (which can be denoted as RGGB), and each pixel consists of 4 sub-pixels. The pixel arrangement of the image data output by a quad sensor is described below with reference to Figure 2A.

[0106] As shown in Figure 2A, the image data output by the Quad sensor contains pixels of four colors: R, Gr, Gb, and B. Each color pixel consists of four sub-pixels: R0, R1, R2, and R3; Gr0, Gr1, Gr2, and Gr3; Gb0, Gb1, Gb2, and Gb3; and B0, B1, B2, and B3. For each color pixel's sub-pixels (or adjacent 2x2 sub-pixels of the same color), such as R0, R1, R2, and R3, there are differences in photosensitivity among sub-pixels of the same color. Therefore, Quad Bayer coding sensitivity correction (QSC) calibration is required. These differences in photosensitivity ultimately lead to differences in brightness.

[0107] This is because without QSC correction (or QSC calibration), the image data output by the Quad sensor will exhibit uneven brightness, resulting in poor image quality. Poor image quality can manifest as stripes in the image, such as crosshair crosstalk stripes or other forms of stripes.

[0108] Figure 2B shows a comparison of pixel sensitivity differences before and after QSC correction. In Figure 2B, the vertical axis represents the sensitivity values ​​corresponding to different color pixels (or sub-pixels). As shown in Figure 2B, for sub-pixels R0, R1, R2, and R3, the sensitivity differences are significant before QSC correction. However, after QSC correction, the sensitivity differences among sub-pixels R0, R1, R2, and R3 decrease.

[0109] The four shooting scenarios mentioned above specifically include: (1) shooting scenarios with a zoom ratio of 2x or higher (e.g., 3x) and high illumination in photo preview mode; (2) shooting scenarios with a zoom ratio of 2x or higher (e.g., 3x) and high dynamic range in photo preview mode; (3) shooting scenarios with a zoom ratio of 2x or higher (e.g., 3x) and high illumination in video preview mode; and (4) shooting scenarios with a zoom ratio of 2x or higher (e.g., 3x) and high dynamic range in video preview mode. In all these scenarios, the sensor uses full-size cropping for image output. If the aperture of the electronic device changes in any of these four shooting scenarios, calibration data is required to calibrate the image data to ensure image quality. Calibration data refers to the data used to compensate for the acquired image data.

[0110] Currently, during the factory calibration of electronic device modules, QSC calibration data is burned into the electronic device's memory, such as EEPROM. This allows the calibration data to be called to correct the image data after sensor startup, either before the sensor starts outputting image data or during camera initialization. Sensor startup can be understood as the sensor beginning to output image data, or transmitting a data stream. However, this method cannot support scenarios where the aperture changes, or rather, it does not support sensors with variable aperture characteristics. That is, when the electronic device switches from one aperture to another, it still uses the calibration data from the previous aperture, which will lead to poor calibration results and thus affect image quality.

[0111] In view of this, this application proposes a method for shooting, which supports the transmission of calibration data after the aperture changes or after the sensor starts up, in the scenario where the sensor adopts a quadruple Bayer array QBC. That is, it can dynamically update the QSC calibration data. The image data after calibration data compensation can effectively reduce the brightness difference between pixels, thereby reducing the stripes caused by brightness differences in the image and improving the image effect.

[0112] The software system used in the embodiments of this application will be described below with reference to Figure 3.

[0113] Figure 3 is a schematic diagram of the architecture (including software system and some hardware) applied in the embodiments of this application. As shown in Figure 3, the application architecture is divided into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the application architecture can be divided into five layers, from top to bottom: Application Layer, Application Framework Layer, Hardware Abstraction Layer, Kernel Layer, and Hardware Layer.

[0114] As shown in Figure 3, the application layer includes a camera and a gallery. It is understood that Figure 3 shows only a portion of the applications; in fact, the application layer can also include other applications (including system applications and / or third-party applications), which this application does not limit. For example, the application layer may also include applications such as messaging, alarm clock, weather, stopwatch, compass, timer, flashlight, calendar, and Alipay.

[0115] As shown in Figure 3, the application framework layer includes a camera access interface. For example, the camera access interface includes camera management and camera devices. The hardware abstraction layer includes an automatic exposure (AE) module, a variable aperture HAL module, and a sensor processing module (e.g., a sensor node HAL module). The AE module is used to automatically adjust the sensor exposure time to regulate image brightness when ambient lighting conditions change.

[0116] It is understood that the hardware abstraction layer shown in Figure 3 may also include other modules, and the embodiments of this application are not limited thereto. For example, optionally, the hardware abstraction layer may also include a camera algorithm library. The camera algorithm library includes image processing algorithm modules.

[0117] The kernel layer is used to drive hardware resources. The kernel layer can include multiple driver modules. As shown in Figure 3, the kernel layer includes a variable aperture driver and an image sensor driver. Optionally, the kernel layer may also include a digital signal processor driver and a graphics processor driver, etc.

[0118] The hardware layer includes various hardware resources. As shown in Figure 3, the hardware layer includes hardware related to the camera module, such as the image sensor and the variable aperture component.

[0119] Optionally, the camera module may also include an electrically programmable read-only memory (EEPROM) or other storage resources. The EEPROM may store QSC calibration data corresponding to the aperture.

[0120] Optionally, the hardware layer may also include other sensors (such as an ambient light sensor), an image signal processor, a digital signal processor, and a graphics processor.

[0121] It should be understood that the software architecture shown in Figure 3 is merely an example description, and the embodiments of this application are not limited thereto. In fact, the various layers in Figure 3 may include many other modules.

[0122] The following is a simplified description of the method timing interaction flow applied to the software architecture in Figure 3, using Figure 4 as an example. The sensor node HAL module and variable aperture HAL module shown in Figure 4 can be located in the HAL layer of Figure 3. The variable aperture driver and image sensor driver shown in Figure 4 can be located in the kernel layer of Figure 3. The kernel layer and hardware abstraction layer in Figure 4 communicate through an intermediate layer (or interface, also simply referred to as the intermediate layer). As shown in Figure 4, the process includes, but is not limited to, the following steps:

[0123] Step 1: The sensor node HAL module sends an ExecuteProcessRequest to the variable aperture HAL module.

[0124] The execution process request can correspond to each image frame; in other words, the sensor node HAL module will issue an execution process request when each image frame is generated.

[0125] Optionally, the process request may include aperture parameters. For example, the aperture parameter may be an aperture value (i.e., VA code).

[0126] Step 2: The variable aperture HAL module acquires the aperture parameters. These parameters are used to switch the aperture.

[0127] Upon receiving an execution process request, the variable aperture module retrieves the aperture parameters from the request. For example, the variable aperture HAL module calls an aperture parameter method function to convert the aperture parameters into variable aperture parameter values. For instance, the aperture parameter method function might be the `GetVAparams` function, which retrieves the aperture value (i.e., the VA code) from the AE tag. Here, the AE tag can be understood as a parameter issued by the AE module; the `GetVAparams` function is a function for retrieving aperture parameters. In other words, the aperture value can be obtained from the AE tag by calling the `GetVAparams` function.

[0128] Step 3: The variable aperture HAL module sends the aperture parameters to the intermediate layer.

[0129] Step 4: The intermediate layer sends the aperture parameters to the variable aperture driver.

[0130] Step 5: Apply aperture parameters to the variable aperture drive.

[0131] For example, the variable aperture driver applies aperture parameters to switch the aperture, switching from the current aperture to the first aperture corresponding to the aperture parameters.

[0132] Of course, after the variable aperture driver applies the aperture parameters, it will drive the corresponding aperture hardware (e.g., the first aperture) to work.

[0133] Step 6-1: The variable aperture driver returns the application results of the aperture parameters to the intermediate layer.

[0134] Step 6-2: The intermediate layer transmits the application results of the aperture parameters to the variable aperture HAL module.

[0135] Step 6-3: The variable aperture HAL module transmits the application results of the aperture parameters to the sensor node HAL module.

[0136] Step 7: The sensor node HAL module sends QSC calibration data to the middle layer.

[0137] Specifically, after the sensor node HAL module determines that the aperture parameters have changed, it will issue a command including QSC calibration data. This QSC calibration data corresponds to the aperture after the switch (for example, the QSC calibration data corresponding to the first aperture) so as to update the QSC calibration data in a timely manner to adapt to the switched aperture.

[0138] Step 8: The intermediate layer transmits the QSC calibration data to the image sensor driver.

[0139] Step 9: The image sensor driver applies the QSC calibration data, or in other words, the QSC calibration data takes effect.

[0140] Optionally, the QSC calibration data consists of a sequence or matrix of luminance offset values. Taking a matrix as an example, each element in the matrix corresponds to a pixel (specifically, a sub-pixel of a pixel), and the value of each element in the matrix is ​​the calibration data value corresponding to the pixel (or sub-pixel).

[0141] For each sub-pixel of a pixel, its pixel value can be compensated using corresponding calibration data values. Taking a sub-pixel of a single pixel as an example, assuming its pixel value is 100, by superimposing the corresponding calibration data value (e.g., the calibration data value for this sub-pixel is -20), the compensated pixel value becomes 80. This process of superimposing corresponding calibration data values ​​for each pixel value aims to reduce brightness differences.

[0142] It should be understood that the embodiments in this application are described using the name of QSC calibration data (such as first calibration data, second calibration data), and the embodiments in this application are not limited to this. In fact, calibration data can also have other names or titles, such as compensation data, correction data, rectification data, etc. However, no matter how it is named, the essence of the terminology remains unchanged, that is, the relevant explanations of the terminology can refer to the above description of calibration data.

[0143] Step 10-1: The image sensor driver returns the application results of the QSC calibration data to the intermediate layer.

[0144] Step 10-2: The intermediate layer transmits the application results of the QSC calibration data to the sensor node HAL module.

[0145] Regarding the process in Figure 4, steps 3 to 6-3 above can be understood as the process of aperture change, or the process of switching apertures. Steps 7 to 10-2 above can be understood as the process of issuing QSC calibration data.

[0146] In some embodiments, steps 3 to 6-3 occur in one image frame, while steps 7 to 10-2 occur in the next image frame. For example, steps 3 to 6-3 occur in the (N+1)th frame, while steps 7 to 10-2 occur in the (N+2)th frame.

[0147] To facilitate understanding of the timing of the calibration data distribution in this application embodiment, the following description is based on the timing shown in Figure 5. Figure 5 illustrates the timeline after sensor flow starts. T1 to T14 can be understood as partial timestamps corresponding to the sensor flow start. Taking frame N-1 as an example, the timestamp corresponding to the start-of-frame delimiter (SOF) of frame N-1 is T1, and the timestamp corresponding to the end-of-frame delimiter (EOF) of frame N-1 is T2. The timestamps for other image frames follow the same pattern. The time interval between the EOF of frame N-1 and the SOF of frame N can be called Vblank, which is the duration between T2 and T4. The frame interval between frame N-1 and frame N is the time interval between T1 and T4.

[0148] In this embodiment, the QSC calibration data is issued only after the aperture changes and the sensor starts current flow. That is, the issuance of QSC calibration data in this embodiment includes two stages: the first stage is the aperture adjustment stage, and the second stage is the issuance of QSC calibration data. Optionally, the second stage may occur after the first stage. Alternatively, the second stage may be performed simultaneously during the first stage. Generally, the aperture has already switched before the QSC calibration data issuance is completed. In short, after both stages are completed, the image frame output by the sensor matches the latest QSC calibration data and the latest aperture effect. The detailed timing of these two stages is described below.

[0149] Phase 1

[0150] Before the timestamp corresponding to SOF in frame N, i.e., before T4, the sensor receives the exposure and aperture parameters (used to switch the aperture to the first aperture) from the AE module. After obtaining the exposure and aperture parameters, the sensor sets the exposure parameters starting at SOF in frame N; and the sensor adjusts the aperture starting at EOF in frame N+1 (i.e., T8), which can be understood as switching from the current aperture to the first aperture.

[0151] The AE module sends exposure and aperture parameters to the sensor and aperture parameters to the aperture based on the decision or instructions issued by the camera application.

[0152] This application does not limit the object or timing of triggering aperture changes. It can be an electronic device automatically switching the aperture based on the shooting mode or scene, or it can be a user actively triggering the aperture switch. For example, when a user is shooting in professional shooting mode, actively changing the aperture triggers an aperture adjustment command.

[0153] Optionally, in the scenario where the user triggers the aperture switching, the timestamp corresponding to the user triggering the aperture switching can be T2-1 in Figure 5, or a timestamp earlier than T2-1, without any specific limitation.

[0154] Optionally, the timestamps corresponding to the exposure and aperture parameters decided by the AE module can be T2-2.

[0155] After receiving the aforementioned exposure parameters, the sensor performs exposure based on the exposure parameters, starting from a timestamp before the SOF of the (N+2)th frame and after the SOF of the (N+1)th frame (e.g., T7 in Figure 5). For example, the sensor performs exposure based on the exposure parameters from T9 to T11, and outputs the exposure result at the (N+2)th frame; however, since the (N+2)th frame is discarded in this embodiment, it can be assumed that the exposure result is output in the next frame after the (N+2)th frame (i.e., the (N+3)th frame). The mechanism by which the exposure parameters take effect can be found in the relevant technical descriptions, which will not be repeated here.

[0156] Since the aperture adjustment starts at EOF (T8) of frame N+1, and the aperture adjustment takes a certain amount of time, the process of adjusting the aperture will affect the image brightness of frame N+2. Therefore, the image data corresponding to frame N+2 needs to be discarded (or skipped), that is, the image corresponding to frame N+2 is not displayed.

[0157] Optionally, the image corresponding to the N+1 frame can be displayed during the display duration of the N+2 frame. That is, since the N+2 frame is the frame output by the sensor when the aperture changes, its image brightness will be abnormal, or its exposure will be affected. In order to avoid presenting the user with an image frame with abnormal brightness, the last normal image frame before the aperture adjustment (i.e., the N+1 frame) can be displayed.

[0158] Phase Two

[0159] The sensor begins writing the QSC calibration data corresponding to the first aperture at the SOF of frame N+2. For example, the QSC calibration data corresponding to the first aperture is written starting at timestamp T10.

[0160] It should be noted that, in order to minimize the impact of writing QSC calibration data on image quality, it is advisable to perform the writing of the QSC calibration data corresponding to the first aperture during the frame interval (frame time) to avoid affecting image quality. Specifically, the frame interval here can be the interval between the SOF of the (N+2)th frame and the SOF of the (N+3)th frame.

[0161] Furthermore, since the QSC function (or calibration function) is disabled when writing the calibration data corresponding to the first aperture, the latest written QSC calibration data has not yet taken effect. From this perspective, the image quality of the (N+2)th frame will also be affected, therefore the (N+2)th frame needs to be discarded. As mentioned earlier, the image corresponding to the (N+1)th frame can be used for display to ensure that the preview image presented to the user is an image frame with appropriate brightness, which will not affect the displayed image effect.

[0162] Since writing the QSC calibration data corresponding to the first aperture takes a certain amount of time, in order to reduce the time required to write the QSC calibration data, this embodiment of the application preprocesses the QSC calibration data to compress the number of bits corresponding to the QSC calibration data.

[0163] In some embodiments, the QSC calibration data corresponding to the first aperture is the calibration data after preprocessing the original QSC calibration data (or the second calibration data, or the calibration data before preprocessing).

[0164] This application does not limit the specific method of preprocessing. In some embodiments, the application scenarios applicable to this application are related to zoom ratio (e.g., applied to zoom ratios of 2x and above). Since for zoom ratios of 2x and above, the sensor will crop the image data accordingly when outputting the image, in order to minimize the impact of sending QSC calibration data on the image frame, the QSC calibration data is preprocessed so that only the QSC calibration data corresponding to the cropped image is sent. This ensures the data compensation effect while reducing the time spent sending QSC calibration data.

[0165] To facilitate understanding, let's refer to Figure 6, which illustrates the image cropping process during sensor output. As shown in Figure 6, the size of the cropped image is clearly smaller than the size of the full-size image. The image size corresponds to the QSC calibration data before preprocessing. For example, in full-size mode, the image size is 8192*6144, while after cropping, the size of the cropped central region is 4096*3072. Thus, by preprocessing the QSC calibration data, only the QSC calibration data corresponding to the cropped central region is sent, reducing the transmission time of the QSC calibration data.

[0166] Optionally, the number of bits corresponding to the QSC calibration data for the first aperture is less than the number of bits corresponding to the original QSC calibration data.

[0167] For example, the original QSC calibration data corresponds to 3536 bytes, with a corresponding transmission time of approximately 43 milliseconds. The QSC calibration data corresponding to the first aperture is 1008 bytes, with a corresponding transmission time of approximately 15 milliseconds.

[0168] When the number of bits corresponding to the QSC calibration data for the first aperture decreases, the transmission time will also decrease accordingly. The writing process of the QSC calibration data corresponding to the first aperture occurs between the SOF of the (N+2)th frame and the SOF of the (N+3)th frame, and the image corresponding to the affected (N+2)th frame is discarded.

[0169] Optionally, the transmission duration of the QSC calibration data corresponding to the first aperture is less than the frame interval between the (N+2)th and (N+3)th frames.

[0170] For example, the transmission time of the QSC calibration data corresponding to the first aperture is 15 milliseconds, and the frame interval between the SOF of the (N+2)th frame and the SOF of the (N+3)th frame is 33 milliseconds. It can be seen that the transmission time of the QSC calibration data corresponding to the first aperture is less than the frame interval, meaning that the transmission of the QSC calibration data corresponding to the first aperture can be completed within one image frame, thus only affecting one frame of data.

[0171] With the transmission time of QSC calibration data corresponding to the first aperture reduced, the number of affected image frames will also decrease accordingly. That is, only the N+2th image frame will be affected, and no more image frames will be affected, thus achieving a better shooting experience.

[0172] After the first and second stages described above, when the sensor outputs the (N+3)th frame (starting from T13), the QSC calibration data and effect corresponding to the first aperture become effective. Furthermore, in the (N+3)th frame, the exposure parameters match the aperture parameters. Since the aperture has been switched to the first aperture, and the newly issued QSC calibration data corresponding to the first aperture is also effective, the image effect obtained in the (N+3)th frame is better than the image obtained using the QSC calibration data corresponding to the switched first aperture, compared to using the original aperture (i.e., the aperture before the switch) for compensation.

[0173] It should be noted that the timing of writing the QSC calibration data in this embodiment occurs after the sensor starts up. As shown in Figure 5, compared to the timing of writing QSC calibration data before starting up in related technologies (or in other words, although the camera is turned on, the sensor has not yet displayed an image), the QSC calibration data written in this embodiment occurs after the sensor starts up, specifically after the aperture changes. The advantage of this is that it can send QSC calibration data that matches the changed aperture, or in other words, it can dynamically update the QSC calibration data based on the aperture change, so that the QSC calibration data corresponds to the changed aperture, thereby achieving better correction or compensation and obtaining better image quality.

[0174] This application does not specifically limit the storage space for writing QSC calibration data (e.g., QSC calibration data corresponding to the first aperture). Taking the QSC calibration data corresponding to the first aperture as an example, the storage space where the QSC calibration data corresponding to the first aperture is located can have the following different implementations.

[0175] One implementation involves storing the QSC calibration data corresponding to the first aperture in the OTP.

[0176] For example, if it is necessary to write the QSC calibration data corresponding to the first aperture, the QSC calibration data corresponding to the first aperture can be written into the sensor OTP.

[0177] In another implementation, the QSC calibration data corresponding to the first aperture is stored in a non-volatile memory EEPROM.

[0178] For example, if it is necessary to write the QSC calibration data corresponding to the first aperture, the QSC calibration data corresponding to the first aperture can be written to the EEPROM, and then the written QSC calibration data corresponding to the first aperture is transferred to the sensor to take effect. The advantage of doing so is that when the OTP space reserved by the sensor is small, or even non-existent, writing the calibration data to the EEPROM can be an option. Furthermore, since the sensor OTP itself has a small storage space and is prone to damage, in this case, to improve reliability, the storage of the QSC calibration data corresponding to the first aperture can be achieved by attaching an external EEPROM to the camera module.

[0179] Another implementation method involves storing the QSC calibration data corresponding to the first aperture in the system configuration file oeminfo.

[0180] For example, when it is necessary to write the QSC calibration data corresponding to the first aperture, the QSC calibration data corresponding to the first aperture can be stored in oeminfo. Therefore, when the sensor does not reserve OTP space and does not set EEPROM, the first calibration data can also be stored in oeminfo.

[0181] The following describes the method flow for taking pictures according to an embodiment of this application, with reference to the flow shown in Figure 7. As shown in Figure 7, the method includes:

[0182] Step 701: At the first moment, activate the camera sensor.

[0183] The first moment can be understood as the moment when the user triggers the camera. For example, the first moment is the moment when the sensor starts streaming in Figure 5.

[0184] This application does not specifically limit how the camera sensor is activated. Optionally, in response to a user's first operation, the camera sensor is activated at a first moment. After the electronic device detects the operation of turning on the camera, it can activate the camera to begin acquiring image data.

[0185] This application does not limit the method by which the user triggers the camera. The first operation is an operation to trigger the opening of the camera of the electronic device. The first operation includes, but is not limited to, touch operation, button operation, voice control, etc. This application does not specifically limit the specific form of the first operation.

[0186] Optionally, the first operation is to open the camera application. For example, as shown in Figure 1A (1), the first operation is for the user to click on the camera application to start the camera.

[0187] Step 702: At the second time point, obtain the exposure parameters and aperture parameters of the Nth frame; wherein, the first time point is before the second time point.

[0188] Optionally, the aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture.

[0189] In some embodiments, the exposure parameters and aperture parameters of the Nth frame are determined by the automatic exposure module, and the exposure parameters and aperture parameters are cached in the camera driver.

[0190] This application does not limit the order in which the exposure parameters and aperture parameters are acquired. In other words, this application does not limit whether the exposure parameters and aperture parameters are acquired at the same time.

[0191] At the second moment, obtaining the exposure parameters and aperture parameters of the Nth frame can include the following three cases: (1) obtaining the exposure parameters and aperture parameters simultaneously after parameter adjustment; (2) obtaining the exposure parameters first, and then obtaining the aperture parameters corresponding to the exposure parameters; (3) obtaining the aperture parameters first, and then obtaining the exposure parameters corresponding to the aperture parameters.

[0192] This application does not limit the triggering operation for acquiring exposure parameters and aperture parameters. Optionally, in some embodiments, acquiring the exposure parameters and aperture parameters of the Nth frame includes: acquiring the exposure parameters and aperture parameters in response to a second operation. That is, the aforementioned exposure parameters and aperture parameters can be acquired in response to a second operation.

[0193] The second operation can be understood as an operation that triggers the aperture of the electronic device to switch. This application does not limit the specific form of the second operation in its embodiments.

[0194] Optionally, the second operation is a user manually adjusting the aperture. That is, at the second moment, the electronic device can respond to the user's manual aperture adjustment operation and acquire exposure parameters and aperture parameters.

[0195] For example, the second operation is the user manually adjusting the aperture in professional shooting mode. Figure 8A shows an example of the interface for adjusting the aperture in professional mode. As shown in Figure 8A (1), taking the photo preview interface 80 as an example, interface 80 includes professional mode. In response to the user's operation of clicking on professional mode, the mobile phone displays interface 81 as shown in Figure 8A (2).

[0196] The interface 81 shown in Figure 8A(2) is the preview interface in professional mode. In professional mode, the relevant parameters of the camera settings can be manually adjusted by the user. As shown in Figure 8A(2), the display area 801 in the camera interface 81 includes multiple camera parameter setting options. The display area 801 includes at least the aperture adjustment control 802, or the icon "A". The icon "A" can also be understood as the aperture setting button.

[0197] The mobile phone responds to the user's tap on the aperture adjustment control 802 and displays the interface 82 shown in Figure 8A (3). The interface 82 includes at least a display area 803. The display area 803 displays the control for adjusting the aperture. The mobile phone responds to the user's aperture adjustment operation (e.g., sliding left or right) performed in area 803 and switches to the corresponding aperture. In the interface shown in Figure 8A, the second operation is the user's manual adjustment of the aperture in area 803 of the interface 81.

[0198] It is understood that the aperture adjustment interface shown in Figure 8A (3) is only an example description, and the embodiments of this application are not limited thereto. For example, the aperture adjustment interface in display area 803 can also be in the form of a dial, and correspondingly, the user's operation to adjust the aperture size can be the operation of adjusting the dial.

[0199] It can also be understood that display area 801 also shows other camera parameters in professional mode. Optionally, area 801 shown in Figure 8A (2) also includes the following camera parameters: metering mode (corresponding to icon "M"), ISO parameter (corresponding to icon ISO, for example, the ISO value in the figure is 100), shutter speed (corresponding to icon "S", for example, the shutter speed in the figure is 1 / 40), exposure compensation (corresponding to icon "EV."), focus mode (corresponding to icon "AF."), and white balance parameter (corresponding to icon "WB."). For an explanation of the camera parameters, please refer to the description in the relevant technology, which will not be repeated here.

[0200] For example, the second operation is the operation after the user switches the current shooting mode to the large aperture shooting mode. Figure 8B shows an example of the interface when the user switches from the normal shooting mode to the large aperture mode. As shown in interface 83 in Figure 8B (1), interface 83 is the shooting preview interface. In response to the user's operation of swiping to the right in the shooting mode area of ​​interface 83, the phone displays interface 84 as shown in Figure 8B (2). Interface 84 presents more shooting mode options, including at least the large aperture mode. In response to the user's operation of clicking the large aperture mode, the phone enters the large aperture shooting mode and adjusts the aperture accordingly. In the interface shown in Figure 8B, the second operation is the operation of the user clicking the large aperture mode.

[0201] It should be understood that the professional mode or large aperture mode entry shown above is only an example description, and the embodiments of this application are not limited thereto.

[0202] Based on the interface shown in Figure 8A or Figure 8B, the mobile phone can respond to the user's manual aperture adjustment operation, execute aperture setting, and thus adjust the aperture according to the user's needs to help the user take photos with better image quality.

[0203] Alternatively, the second operation may be the operation of manually adjusting the aperture by the electronic device.

[0204] At the second moment, the electronic device can acquire exposure and aperture parameters based on the automatic aperture adjustment. For example, the electronic device is equipped with a variable aperture; after detecting a specific shooting scene, the electronic device will automatically adjust the aperture according to the lighting conditions of the shooting scene, at which time the electronic device will acquire exposure and aperture parameters.

[0205] Step 703: At the third moment, the exposure parameters are sent to the camera sensor; the third moment is the moment before the time corresponding to the frame start delimiter (SOF) of the Nth frame; the second moment is before the third moment.

[0206] It should be noted that after the exposure parameters are sent to the camera sensor, they can take effect from a certain timestamp, before the SOF of the (N+2)th frame and after the SOF of the (N+1)th frame (e.g., T7 in Figure 5). For example, the exposure parameters may take effect from T9 as shown in Figure 5, meaning that exposure is performed based on the exposure parameters starting from T9; or the exposure parameters may take effect from T8 as shown in Figure 5. It should be understood that the times at which the exposure parameters take effect shown here are merely illustrative descriptions, and the embodiments of this application are not limited thereto. In addition, the process of writing calibration data will affect the (N+2)th frame, which will be discarded (e.g., the image of the (N+1)th frame is displayed during the display duration of the (N+2)th frame). Therefore, the brightness of the (N+3)th frame is consistent with the brightness of the (N)th frame and the brightness of the (N+1)th frame, and the exposure parameters match the aperture parameters in the (N+3)th frame. The meaning of the exposure parameters matching the aperture parameters is that the aperture has been successfully switched in the (N+3)th frame, the QSC calibration data corresponding to the aperture is also matched with the aperture, and the exposure parameters have taken effect. Of course, after the aperture switch is successful, the AE module can send the corresponding exposure parameters according to the actual situation. For example, it can send the exposure parameters corresponding to the switched aperture at the N+3 frame. For the mechanism of sending and taking effect of the exposure parameters, please refer to the description of the relevant technology. For the sake of brevity, it will not be elaborated here.

[0207] In some embodiments, after time T2-2 and before time T4, the camera sensor HAL acquires the aperture and exposure parameters input from the AE module. The camera sensor HAL sends the exposure parameters to the sensor driver. The sensor driver can then send the exposure parameters to the hardware (i.e., the camera sensor) for subsequent application.

[0208] In some embodiments, the camera sensor HAL sends aperture parameters to the aperture HAL. After acquiring the aperture parameters, the aperture HAL can send the aperture parameters to the aperture motor driver. The aperture motor driver can then send the aperture parameters to the hardware (aperture assembly) for activation.

[0209] For example, the aperture HAL calls the aperture parameter method function, which converts the aperture parameters into variable aperture parameter values ​​(VA code).

[0210] Step 704: At the fourth moment, the aperture assembly adjusts the aperture based on the aperture parameters, where the fourth moment is the moment corresponding to the end-of-frame (EOF) delimiter of the (N+1)th frame.

[0211] For example, the fourth moment is T8 in Figure 5. That is, starting from the EOF of the (N+1)th frame, the aperture is adjusted based on the aperture parameters, that is, the current aperture is switched to the first aperture indicated by the aperture parameters.

[0212] It's understandable that adjusting the aperture requires a certain amount of time; for example, Figure 5 shows the time required to adjust the aperture. Adjusting the aperture affects frame N+2, but not frame N+3. Frame N+2 can also be called the aperture change frame.

[0213] Step 705: At the fifth moment, the first calibration data is sent to the camera sensor; the fifth moment is the moment corresponding to the SOF of the N+2th frame.

[0214] Optionally, the first calibration data is calibration data corresponding to the first aperture. Accordingly, the aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture. That is, when the electronic device switches from the current aperture to the first aperture, the first calibration data sent is the calibration data corresponding to the switched first aperture, i.e., the first calibration data.

[0215] Specifically, within the display duration corresponding to the (N+3)th frame, the image corresponding to the (N+3)th frame is displayed based on the exposure parameters, the aperture parameters, and the first calibration data. In other words, after the calibration data is sent, the exposure parameters and aperture parameters of the displayed image in the (N+3)th frame are matched. A related description can be found in Figure 5 above, and will not be repeated here.

[0216] In some embodiments, when the camera sensor of the electronic device is a Quad sensor, the first calibration data is the QSC calibration data corresponding to the first aperture.

[0217] For example, the fifth moment is T10 in Figure 5. That is, starting from the SOF of the (N+2)th frame, the QSC calibration data corresponding to the first aperture is sent to the sensor.

[0218] In some embodiments, the camera sensor HAL sends QSC calibration data corresponding to the first aperture to the sensor driver. The sensor driver then sends the QSC calibration data corresponding to the first aperture to the hardware (i.e., the camera sensor) for subsequent application.

[0219] In this embodiment, the electronic device supports switching the calibration data corresponding to the aperture after the sensor starts up. That is, when the aperture changes, the electronic device can correspondingly switch the calibration data to perform image data compensation based on the appropriate calibration data, thereby improving image quality. Compared to methods that can only burn a set of calibration data before the sensor starts up, this embodiment can achieve calibration data switching even after the sensor starts up, improving image output quality.

[0220] Further, optionally, the first calibration data is calibration data after preprocessing the second calibration data, wherein the number of bits corresponding to the second calibration data is greater than the number of bits corresponding to the first calibration data. As mentioned above, by preprocessing the calibration data, the number of bits corresponding to the calibration data is reduced, thereby reducing the transmission time when sending the calibration data, and thus minimizing the number of affected image frames.

[0221] For example, after receiving the exposure and aperture parameters before the SOF (Solution of Flight) of frame N, the electronic device starts adjusting the aperture based on the aperture parameters at EOF (Exit of Flight) of frame N+1, and sends the QSC (Quick Score) calibration data of the adjusted aperture at SOF of frame N+2. This ensures that the exposure and aperture parameters match at frame N+3. In this process, only frame N+2 is affected, and no other frames are impacted.

[0222] Optionally, the transmission duration of the first calibration data is less than the frame interval between the (N+2)th and (N+3)th frames. As mentioned earlier, the process of writing the first calibration data occurs between the SOF of the (N+2)th and (N+3)th frames to avoid affecting more image frames.

[0223] Optionally, within the display duration corresponding to the (N+3)th frame, the image corresponding to the (N+3)th frame is displayed based on the exposure parameters, the aperture parameters, and the first calibration data. As mentioned above, starting from the (N+3)th frame, the exposure parameters and aperture parameters are matched, and the image data can be compensated accordingly based on the first calibration data. Therefore, within the display duration corresponding to the (N+3)th frame, the image corresponding to the (N+3)th frame can be displayed.

[0224] Optionally, the image corresponding to the N+1 frame is displayed during the display duration of the N+2 frame. That is, the camera sensor discards the N+2 frame when outputting the image. As mentioned above, the N+2 frame is affected during the writing of the first calibration data, so the image of the previous frame (e.g., the image corresponding to the N+1 frame) can be displayed when sending the image to avoid affecting the user experience.

[0225] This application embodiment is applied to high dynamic range environments (including photo preview and video preview) in preview scenarios where the zoom ratio is greater than a preset zoom ratio (e.g., 2x), or high illumination environments. For example, the scenario shown in Figure 1A (4). Another example is the scenario shown in Figure 1B (4).

[0226] It should be understood that the scenarios shown in Figure 1A or Figure 1B are merely illustrative descriptions, and the embodiments of this application are not limited thereto.

[0227] The method for taking pictures provided by the embodiments of this application has been described in detail above with reference to Figures 1A to 8B. The apparatus embodiments of this application will now be described with reference to Figures 9 and 10. It should be understood that the apparatus for taking pictures in the embodiments of this application can execute the aforementioned method for taking pictures in the embodiments of this application; that is, the specific working processes of the various products described below can be referred to the corresponding processes in the foregoing method embodiments.

[0228] Figure 9 shows a schematic diagram of the structure of an electronic device 1000 applicable to this application.

[0229] Electronic device 1000 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc.

[0230] The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0231] It should be noted that the structure shown in FIG9 does not constitute a specific limitation on the electronic device 1000. In other embodiments of this application, the electronic device 1000 may include more or fewer components than those shown in FIG9, or the electronic device 1000 may include a combination of some of the components shown in FIG9, or the electronic device 1000 may include sub-components of some of the components shown in FIG9. The components shown in FIG9 may be implemented in hardware, software, or a combination of software and hardware.

[0232] Processor 110 may include one or more processing units. For example, processor 110 may include at least one of the following processing units: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and neural network processing unit (NPU). These different processing units may be independent devices or integrated devices.

[0233] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0234] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0235] In some embodiments, processor 110 may include one or more interfaces. For example, processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and a USB interface.

[0236] The connection relationships between the modules shown in Figure 9 are merely illustrative and do not constitute a limitation on the connection relationships between the modules of the electronic device 1000. Optionally, the modules of the electronic device 1000 may also adopt a combination of various connection methods described in the above embodiments.

[0237] The charging management module 140 receives power from a charger, which can be either a wireless or wired charger. In some wired charging embodiments, the charging management module 140 receives current from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives electromagnetic waves (current path shown as dashed lines) via the wireless charging coil of the electronic device 1000. While charging the battery 142, the charging management module 140 can also supply power to the electronic device 1000 via the power management module 141.

[0238] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (e.g., leakage current, impedance). Optionally, the power management module 141 can be located within the processor 110, or the power management module 141 and the charging management module 140 can be located in the same device.

[0239] The wireless communication function of electronic device 1000 can be realized through devices such as antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0240] Electronic device 1000 can implement display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0241] Display screen 194 can be used to display images or videos. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a micro OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 1000 may include one or N displays screens 194, where N is a positive integer greater than 1.

[0242] Electronic device 1000 can achieve shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0243] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0244] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into a standard red-green-blue (RGB), YUV, or other image signal format. In some embodiments, the electronic device 1000 may include one or N cameras 193, where N is a positive integer greater than 1.

[0245] In some embodiments, the electronic device 1000 includes a plurality of apertures, the plurality of apertures including at least a first aperture. Optionally, the camera 193 of the electronic device 1000 includes the plurality of apertures.

[0246] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 1000 is selecting a frequency, the DSP is used to perform Fourier transforms on the frequency energy.

[0247] Video codecs are used to compress or decompress digital video. Electronic device 1000 may support one or more video codecs. Thus, electronic device 1000 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, and MPEG 4.

[0248] Electronic device 1000 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0249] The ambient light sensor 180L is used to sense the ambient light intensity. The electronic device 1000 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light intensity. The ambient light sensor 180L can also be used to automatically adjust the white balance during shooting.

[0250] In some embodiments, the ambient light sensor 180L can acquire the ambient illuminance of the current shooting environment. When the ambient illuminance is greater than or equal to a first illuminance threshold, the current shooting environment can be considered relatively bright, and can be considered a high-brightness shooting environment.

[0251] Touch sensor 180K, also known as a touch device, can be disposed on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a touch display. Touch sensor 180K is used to detect touch operations applied to or near it. Touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be disposed on the surface of electronic device 1000, and in a different location from display screen 194.

[0252] Button 190 includes a power button and volume buttons. Button 190 can be a mechanical button or a touch button. Electronic device 1000 can receive button input signals and implement functions related to the button input signals.

[0253] Motor 191 can generate vibrations. Motor 191 can be used for incoming call notifications or for haptic feedback. Motor 191 can produce different vibration feedback effects for touch operations applied to different applications. Motor 191 can also produce different vibration feedback effects for touch operations applied to different areas of the display screen 194. Different application scenarios (e.g., time reminders, receiving messages, alarm clocks, and games) can correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0254] In some embodiments, the processor 110 is configured to: activate the camera sensor at a first moment; acquire the exposure parameters and aperture parameters of the Nth frame at a second moment; wherein the first moment is prior to the second moment; send the exposure parameters to the camera sensor at a third moment; the third moment is the moment corresponding to the frame start delimiter (SOF) of the Nth frame; the second moment is prior to the third moment; adjust the aperture based on the aperture parameters at a fourth moment; the fourth moment is the moment corresponding to the frame end delimiter (EOF) of the (N+1)th frame; and send first calibration data to the camera sensor at a fifth moment; the fifth moment is the moment corresponding to the SOF of the (N+2)th frame.

[0255] Optionally, in some embodiments, the processor 110 is further configured to invoke the display screen 194 to display the image corresponding to the N+3rd frame within the display duration corresponding to the N+3rd frame, based on the exposure parameters, the aperture parameters, and the first calibration data.

[0256] Optionally, in some embodiments, the processor 110 is further configured to invoke the display screen 194 to display the image corresponding to the N+1 frame within the display duration of the N+2th frame.

[0257] It is understood that the method for taking pictures in this application embodiment can be applied to the electronic device shown in FIG9. The specific implementation steps can be referred to the description of the method embodiment above, and will not be repeated here.

[0258] Figure 10 is a schematic block diagram of an apparatus 800 for taking pictures according to an embodiment of this application. It should be understood that the apparatus 800 can perform the methods for taking pictures shown in Figures 4 to 7.

[0259] As shown in Figure 10, the device 800 includes an input unit 810 and a processing unit 820. The device 800 may be an electronic device. Optionally, the device 800 may also include a display unit 830.

[0260] In some embodiments, the processing unit 820 is configured to: activate the camera sensor at a first moment; acquire the exposure parameters and aperture parameters of the Nth frame at a second moment; wherein the first moment is before the second moment; send the exposure parameters to the camera sensor at a third moment; the third moment is the moment corresponding to the frame start delimiter (SOF) of the Nth frame; the second moment is before the third moment; adjust the aperture based on the aperture parameters at a fourth moment; the fourth moment is the moment corresponding to the frame end delimiter (EOF) of the (N+1)th frame; and send first calibration data to the camera sensor at a fifth moment; the fifth moment is the moment corresponding to the SOF of the (N+2)th frame.

[0261] In some embodiments, the processing unit 820 is used to activate the camera sensor, including activating the camera sensor in response to a first operation by the user.

[0262] Optionally, the input unit 810 is used to detect the user's first operation.

[0263] Optionally, the first operation is to open the camera application.

[0264] In some embodiments, the processing unit 820 is used to obtain the exposure parameters and aperture parameters of the Nth frame, including:

[0265] In response to the second operation, the exposure parameters and the aperture parameters are acquired.

[0266] Optionally, the second operation is a user manually adjusting the aperture, or the second operation is the electronic device automatically adjusting the aperture.

[0267] Optionally, the input unit 810 is used to detect the user's manual adjustment of the aperture.

[0268] In some embodiments, the display unit 830 is configured to display the image corresponding to the N+3rd frame based on the exposure parameters, the aperture parameters, and the first calibration data during the display duration corresponding to the N+3rd frame.

[0269] In some embodiments, the aperture parameter is used to instruct the camera sensor to switch the current aperture to a first aperture.

[0270] In some embodiments, the first calibration data is calibration data corresponding to the first aperture.

[0271] Optionally, the device 800 includes a plurality of apertures, of which at least a first aperture is included.

[0272] In some embodiments, the first calibration data is calibration data after preprocessing the second calibration data, wherein the number of bits corresponding to the second calibration data is greater than the number of bits corresponding to the first calibration data.

[0273] In some embodiments, the transmission duration of the first calibration data is less than the frame interval between the (N+2)th frame and the (N+3)th frame.

[0274] In some embodiments, the display unit 830 is further configured to display the image corresponding to the N+1 frame during the display duration of the N+2 frame.

[0275] In some embodiments, the display unit 830 is further configured to display a first preview interface, the first preview interface including a zoom ratio option; the processing unit 820 is configured to respond to a user's operation of adjusting the zoom ratio to a first zoom ratio based on the zoom ratio option, and, upon detecting that the current shooting environment is a high-illuminance environment, invoke the camera sensor to adopt a full-size cropping output mode; wherein, the first preview interface is a photo preview interface or a video preview interface; the first zoom ratio is greater than a preset zoom ratio; the high-illuminance environment includes: the ambient illuminance of the current shooting environment is greater than a first illuminance threshold.

[0276] In some embodiments, the display unit 830 is further configured to display a second preview interface, the second preview interface including a zoom ratio option; the processing unit 820 is configured to respond to a user's operation of adjusting the zoom ratio to a second zoom ratio based on the zoom ratio option, and, upon detecting that the current shooting environment is a high dynamic range environment, invoke the camera sensor to adopt a full-size cropping output mode; wherein, the second preview interface is a photo preview interface or a video preview interface; the second zoom ratio is greater than a preset zoom ratio; the high dynamic range environment includes: the dynamic range value satisfies the dynamic range DR constraint condition.

[0277] In one possible example, the input unit 810 and the processing unit 820 can be implemented by a processor or a processing unit. The display unit 830 can be implemented by a display screen or a display unit. It should be understood that the above-described device 800 is embodied in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and this application embodiment does not specifically limit this.

[0278] For example, a "unit" can be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor) and memory that executes one or more software or firmware programs, integrated logic circuitry, and / or other suitable devices that can provide the above functions. In a simple embodiment, those skilled in the art will recognize that device 800 can take the form shown in FIG. 9.

[0279] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0280] This application also provides a computer program product that, when executed by a processor, implements the methods described in any of the method embodiments of this application.

[0281] The computer program product can be stored in memory and, after processes such as preprocessing, compilation, assembly, and linking, is finally converted into an executable object file that can be executed by a processor.

[0282] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, implements the methods described in any of the method embodiments of this application. The computer program may be a high-level language program or an executable object program.

[0283] The computer-readable storage medium can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0284] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process and technical effects of the above-described apparatus and equipment can be referred to the corresponding processes and technical effects in the foregoing method embodiments, and will not be repeated here.

[0285] In the several embodiments provided in this application, the systems, apparatuses, and methods disclosed can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not performed. The apparatus embodiments described above are merely illustrative; the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components can be combined or integrated into another system. Furthermore, the coupling between units or components can be direct coupling or indirect coupling, including electrical, mechanical, or other forms of connection.

[0286] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0287] Furthermore, the terms "system" and "network" are often used interchangeably in this paper. The term "and / or" in this paper merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this paper generally indicates that the preceding and following related objects have an "or" relationship.

[0288] The terms (or numbers) "first," "second," etc., appearing in the embodiments of this application are for descriptive purposes only, that is, only to distinguish different objects, such as different "calibration data," etc., and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first," "second," etc., may explicitly or implicitly include one or more features. In the description of the embodiments of this application, "at least one (item)" refers to one or more. "Multiple" means two or more. "At least one (item) below" or similar expressions refer to any combination of these items, including any combination of a single (item) or a plurality of (items).

[0289] For example, expressions like "the item includes at least one of the following: A, B, and C" appearing in the embodiments of this application generally mean, unless otherwise specified, that the item can be any one of the following: A; B; C; A and B; A and C; B and C; A, B and C; A and A; A, A and A; A, A and B; A, A and C, A, B and B; A, C and C; B and B, B, B and B, B, B and C, C and C; C, C and C, and other combinations of A, B, and C. The above uses three elements, A, B, and C, as examples to illustrate the possible entries for the item. When expressed as "the item includes at least one of the following: A, B, ..., and X," that is, when the expression contains more elements, then the applicable entries for the item can also be obtained according to the aforementioned rules.

[0290] In summary, the above description is merely a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

A method for taking pictures, characterized in that, The method is applied to an electronic device and includes: activating a camera sensor at a first moment; acquiring exposure parameters and aperture parameters for the Nth frame at a second moment, wherein the first moment precedes the second moment; sending the exposure parameters to the camera sensor at a third moment, wherein the third moment corresponds to the start-of-frame (SOF) delimiter of the Nth frame; the second moment precedes the third moment; adjusting the aperture based on the aperture parameters at a fourth moment, wherein the fourth moment corresponds to the end-of-frame (EOF) delimiter of the (N+1)th frame; and sending first calibration data to the camera sensor at a fifth moment, wherein the fifth moment corresponds to the SOF of the (N+2)th frame. The method according to claim 1, characterized in that, The activation of the camera sensor includes: activating the camera sensor in response to a first operation by the user. The method according to claim 2, characterized in that, The first operation is to open the camera application. The method according to any one of claims 1 to 3, characterized in that, The step of obtaining the exposure parameters and aperture parameters of the Nth frame includes: in response to the second operation, obtaining the exposure parameters and the aperture parameters. The method according to claim 4, characterized in that, The second operation is either a user manually adjusting the aperture, or the second operation is the electronic device automatically adjusting the aperture. The method according to any one of claims 1 to 5, characterized in that, The method further includes: within the display duration corresponding to the N+3rd frame, displaying the image corresponding to the N+3rd frame based on the exposure parameters, the aperture parameters, and the first calibration data. The method according to any one of claims 1 to 6, characterized in that, The first calibration data is the calibration data corresponding to the first aperture. The method according to any one of claims 1 to 7, characterized in that, The aperture parameter is used to instruct the camera sensor to switch the current aperture to the first aperture. The method according to any one of claims 1 to 8, characterized in that, The electronic device includes multiple apertures, of which at least a first aperture is included. The method according to any one of claims 1 to 9, characterized in that, The first calibration data is the calibration data after preprocessing the second calibration data, wherein the number of bits corresponding to the second calibration data is greater than the number of bits corresponding to the first calibration data. The method according to any one of claims 1 to 10, characterized in that, The transmission duration of the first calibration data is less than the frame interval between the (N+2)th frame and the (N+3)th frame. The method according to any one of claims 1 to 11, characterized in that, The method further includes: displaying the image corresponding to the N+1th frame within the display duration of the N+2th frame. The method according to any one of claims 1 to 12, characterized in that, After activating the camera sensor, the method further includes: displaying a first preview interface, the first preview interface including a zoom ratio option; responding to a user's operation of adjusting the zoom ratio to a first zoom ratio based on the zoom ratio option; and detecting that the current shooting environment is a high-light environment, the camera sensor adopts a full-size cropping output mode; wherein, the first preview interface is a photo preview interface or a video preview interface; the first zoom ratio is greater than a preset zoom ratio; the high-light environment includes: the ambient illuminance of the current shooting environment is greater than a first illuminance threshold. The method according to any one of claims 1 to 12, characterized in that, After activating the camera sensor, the method further includes: displaying a second preview interface, the second preview interface including a zoom ratio option; responding to a user's operation of adjusting the zoom ratio to a second zoom ratio based on the zoom ratio option, and detecting that the current shooting environment is a high dynamic range environment, the camera sensor adopts a full-size cropping output mode; wherein, the second preview interface is a photo preview interface or a video preview interface; the second zoom ratio is greater than a preset zoom ratio; the high dynamic range environment includes: the dynamic range value satisfies the dynamic range DR constraint condition. An electronic device, characterized in that, The device includes a processor and a memory coupled together, the memory being used to store a computer program that, when executed by the processor, causes the electronic device to perform the method of any one of claims 1 to 14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the electronic device to perform the method of any one of claims 1 to 14. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 14.

Images