Photographing parameter setting method and electronic equipment

Abstract

The invention provides a shooting parameter setting method and electronic equipment, and the method comprises the steps that the electronic equipment has enough time to write configuration information of N frames before the (N + 1) th frame is output by prolonging the frame length of the N frames. Therefore, the camera sensor can perform plotting on the N + 1 frames based on the written configuration information of the N frames. Therefore, the problem of frame loss caused by incomplete writing of the configuration information is avoided.

Description

Technical Field

[0001] The present application relates to the field of terminal equipment, and in particular to a method and device for setting photographing parameters. Background Art

[0002] With the development of terminal technology, the application scenarios of terminal devices are becoming more and more extensive. For example, users can use mobile phones to take pictures to meet the needs of users to record their daily lives. During the photo-taking process, the terminal can automatically switch to different image output modes according to the current environment to meet the current environment needs, thereby improving the image effect. Summary of the invention

[0003] The present application provides a method for setting photographing parameters and an electronic device. In the method, the electronic device can avoid the problem of frame loss caused by a long writing time of the image output mode configuration information by delaying the effective timing of the image output mode configuration information.

[0004] In a first aspect, the present application provides a method for setting photographing parameters. The method is applied to an electronic device, and the method includes: at a first moment, obtaining a first camera parameter of the N-1th frame, the first camera parameter including a first frame length. At a second moment, obtaining the second image output mode configuration information and the second camera parameter of the Nth frame; wherein the second image output mode configuration information includes configuration parameters corresponding to the second image output mode, the second camera parameter includes a second frame length, and the second frame length is less than the first frame length. At a third moment, writing the first camera parameter to the camera sensor, the camera sensor exposes and outputs the Nth frame based on the first camera parameter; the frame length of the Nth frame is the first frame length, and the image output mode of the Nth frame is the first image output mode. At a fourth moment, writing the second image output mode configuration information and the second camera parameter to the camera sensor, the camera sensor exposes and outputs the N+1th frame based on the second camera parameter and the second image output mode configuration information; wherein, at the fourth moment in the Nth frame, the frame length of the N+1th frame is the second frame length, and the image output mode of the N+1th frame is the second image output mode. In this way, the electronic device delays the effective time of the image output mode configuration information by extending the frame length of the N frame, so that the electronic device has enough time to write the image output mode configuration information into the camera sensor. The effective time of the image output mode configuration information is the exposure time of the N+1 frame.

[0005] Exemplarily, the first moment may be Fig. 9 At the time T1-1 in the , the Sensor node generates the Sensor configuration information of the N-1 frame, and the Sensor node sends the Sensor configuration information of the N-1 frame to the CRM. The second time can be Fig. 9The third moment is optionally T3-1, that is, the Sensor node generates N frames of Sensor configuration information, and the Sensor node sends the N frames of Sensor configuration information to the CRM. Fig. 9 At the T3-2 moment in the , that is, the CRM sends the Sensor configuration information of the N-1 frame to the Sensor driver, and the Sensor driver writes the configuration information of the N-1 frame into the register of the Sensor. The fourth moment can be optionally Fig. 9 At time T5-2 in the figure, the CRM sends the Sensor configuration information of frame N to the Sensor driver, and the Sensor driver writes the configuration information of frame N into the register of the Sensor.

[0006] In a possible implementation, the duration between the effective point of the image output mode of the Nth frame and the frame start delimiter SOF of the Nth frame is the first duration, and the duration between the effective point of the image output mode of the N+1th frame and the SOF of the N+1th frame is the first duration; the camera sensor exposes and outputs the N+1th frame based on the second camera parameter and the second image output mode configuration information, including: the camera sensor obtains the second image output mode configuration information at the effective point of the image output mode of the N+1th frame, and the camera sensor outputs the N+1th frame at the SOF of the N+1th frame. In this way, when the effective point of each frame remains unchanged in the relative position of each frame, the effective point of the N+1 frame can be postponed by extending the frame length of the N frame, so that the electronic device has enough time to write the image output mode configuration information to the camera sensor.

[0007] Exemplarily, the first effective point may be Fig. 9 At time T4-1 in , that is, the Sensor exposes the Nth frame based on the first frame length.

[0008] Exemplarily, the second effective point may be Fig. 9 At time T7-1 in the image, that is, the Sensor exposes frame N+1 based on the Sensor configuration information of frame N.

[0009] In a possible implementation, before writing the first camera parameter into the camera sensor, the method further includes: at a fifth moment, obtaining IFE configuration information of the Nth frame, wherein the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; writing the second image output mode configuration information and the second camera parameter into the camera sensor, and further including: writing the IFE configuration information into the IFE. In this way, the IFE synchronously writes the IFE configuration information to take effect, thereby achieving synchronization with the Sensor configuration information.

[0010] Exemplarily, the fifth moment may be Fig. 9The T3-1 moment in the video.

[0011] In a possible implementation, the method further includes: IFE performs image processing on the N+1th frame input by the camera sensor based on the IFE configuration information. In this way, by delaying the frame length of N frames, the effective time of the image output mode configuration information is postponed, so that the electronic device has enough time to write the image output mode configuration information into the camera sensor. The Sensor can obtain the complete N frames of Sensor configuration information before exposing the N+1 frame. And the N+1 frame is exposed based on the Sensor configuration information of the N frames. IFE also writes the IFE configuration information in the N frames and takes effect in the N+1 frame. Thereby achieving the synchronous effectiveness of the IFE configuration information and the Sensor configuration information to avoid frame loss caused by the asynchronous configuration information.

[0012] In a possible implementation, the first frame length is n times the second frame length, where n is greater than or equal to 1 and less than 2.

[0013] In a second aspect, the present application provides a method for setting photographing parameters, which is applied to an electronic device, and the method includes: at a first moment, obtaining second image output mode configuration information of the Nth frame and IFE configuration information of the Nth frame, wherein the second image output mode configuration information includes effective configuration information and configuration parameters corresponding to the second image output mode, and the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; at a second moment, outputting the Nth frame; wherein the image output mode of the Nth frame is the first image output mode; at a third moment, writing the configuration parameters corresponding to the second image output mode into a camera sensor; wherein the third moment is within the Nth frame; at a fourth moment, outputting the N+1th frame; wherein the image output mode of the N+1th frame is the first image output mode; at a fifth moment within the N+1th frame, writing the effective configuration information into the camera sensor, and writing the IFE configuration information into the IFE, the camera sensor outputs the N+2th frame based on the second image output mode configuration information, and the image output mode of the N+2th frame is the second image output mode. In this way, the electronic device delays the effective timing of the configuration information of frame N so that the configuration information of frame N takes effect at frame N+2, thereby allowing the electronic device to have enough time to write the output mode configuration information before exposing frame N+2 to avoid the problem of frame loss.

[0014] Exemplarily, the first moment may be Fig.10 The second moment can be T1-1 in Fig.10 The third moment can be optionally Fig.10 The fourth moment may be T3-1 in Fig.10 The fifth moment may be T4-1 in Fig.10 The T5-1 moment in the video.

[0015] In a possible implementation, the duration between the effective point of the image output mode of the N+1th frame and the frame start delimiter SOF of the N+1th frame is the first duration, and the duration between the effective point of the image output mode of the N+2th frame and the SOF of the N+2th frame is the first duration; the camera sensor outputs the N+2th frame based on the second image output mode configuration information, including: the camera sensor obtains the second image output mode configuration information at the effective point of the image output mode of the N+2th frame, and the camera sensor outputs the N+2th frame at the SOF of the N+2th frame. In this way, the present application sets the effective timing of the Sensor configuration of the electronic device to take effect at the N+2 frame, so that the electronic device has enough time to write the image output mode configuration information to the camera sensor, thereby avoiding the frame loss problem caused by the IFE configuration information and the Sensor configuration information being out of sync.

[0016] In a possible implementation, the fifth moment is before the effective point of the image output mode of the N+2th frame.

[0017] In a possible implementation, the IFE performs image processing on the N+2th frame input by the camera sensor based on the IFE configuration information.

[0018] In a third aspect, the present application provides an electronic device, characterized in that it includes: one or more processors, a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the following steps: at a first moment, obtaining a first camera parameter of the N-1th frame, the first camera parameter including a first frame length, and at a second moment, obtaining second image output mode configuration information and a second camera parameter of the Nth frame; wherein the second image output mode configuration information includes configuration parameters corresponding to the second image output mode, the second camera parameter includes a second frame length, and the second frame length is less than the first frame length; at a third moment, writing the first camera parameter to a camera sensor, and the camera sensor exposes the Nth frame based on the first camera parameter; the frame length of the Nth frame is the first frame length, and the image output mode of the Nth frame is the first image output mode; at a fourth moment, writing the second image output mode configuration information and the second camera parameter to the camera sensor, and the camera sensor exposes the N+1th frame based on the second camera parameter and the second image output mode configuration information; wherein, at the fourth moment in the Nth frame, the frame length of the N+1th frame is the second frame length, and the image output mode of the N+1th frame is the second image output mode.

[0019] In one possible implementation, when the computer program is executed by one or more processors, the electronic device performs the following steps: the camera sensor exposes the Nth frame based on the first camera parameter at the first effective point; the camera sensor exposes the N+1th frame based on the second camera parameter and the second output mode configuration information at the second effective point; the duration between the second effective point and the start of frame delimiter SOF of the N+1th frame is equal to the duration between the first effective point and the SOF of the Nth frame.

[0020] In one possible implementation, when the computer program is executed by one or more processors, the electronic device performs the following steps: at the fifth moment, obtain IFE configuration information of the Nth frame, the IFE configuration information including preprocessing parameters corresponding to the second image output mode; when the computer program is executed by the one or more processors, the electronic device performs the following steps: write the IFE configuration information into the IFE.

[0021] In a possible implementation, when the computer program is executed by one or more processors, the electronic device executes the following steps: the IFE performs image processing on the N+1th frame input by the camera sensor based on the IFE configuration information.

[0022] In a possible implementation, the first frame length is n times the second frame length, where n is greater than or equal to 1 and less than 2.

[0023] In a fourth aspect, the present application provides an electronic device, characterized in that it includes: one or more processors, a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the following steps: at a first moment, obtaining the second image output mode configuration information of the Nth frame and the IFE configuration information of the Nth frame, wherein the second image output mode configuration information includes effective configuration information and configuration parameters corresponding to the second image output mode, and the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; at the first moment, obtaining the second image output mode configuration information of the Nth frame and the IFE configuration information of the Nth frame, wherein the second image output mode configuration information includes effective configuration information and configuration parameters corresponding to the second image output mode, and the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; At the second moment, the Nth frame is exposed; wherein the image output mode of the Nth frame is the first image output mode; at the third moment, the configuration parameters corresponding to the second image output mode are written into the camera sensor; wherein the third moment is within the Nth frame; at the fourth moment, the N+1th frame is exposed; wherein the image output mode of the N+1th frame is the first image output mode; at the fifth moment within the N+1th frame, the effective configuration information is written into the camera sensor, and the IFE configuration information is written into the IFE, and the camera sensor exposes the N+2th frame based on the second image output mode configuration information, and the image output mode of the N+2th frame is the second image output mode.

[0024] In one possible implementation, when the computer program is executed by one or more processors, the electronic device performs the following steps: the camera sensor exposes the N+1th frame at the first effective point; the camera sensor exposes the N+2th frame at the second effective point; the duration between the second effective point and the start of frame delimiter SOF of the N+1th frame is equal to the duration between the first effective point and the SOF of the N+2th frame.

[0025] In a possible implementation manner, the fifth moment is before the second effective point.

[0026] In a possible implementation, when the computer program is executed by one or more processors, the electronic device executes the following steps: the IFE performs image processing on the N+2th frame input by the camera sensor based on the IFE configuration information.

[0027] In a fifth aspect, the present application provides a computer-readable medium for storing a computer program, wherein the computer program includes instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0028] In a sixth aspect, the present application provides a computer-readable medium for storing a computer program, wherein the computer program includes instructions for executing the method in the second aspect or any possible implementation of the second aspect.

[0029] In a seventh aspect, the present application provides a computer program comprising instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0030] In an eighth aspect, the present application provides a computer program comprising instructions for executing the method in the second aspect or any possible implementation of the second aspect.

[0031] In a ninth aspect, the present application provides a chip, the chip comprising a processing circuit and a transceiver pin, wherein the transceiver pin and the processing circuit communicate with each other through an internal connection path, and the processing circuit executes the method in the first aspect or any possible implementation of the first aspect to control the receiving pin to receive a signal and to control the sending pin to send a signal.

[0032] In a tenth aspect, the present application provides a chip, the chip comprising a processing circuit and a transceiver pin, wherein the transceiver pin and the processing circuit communicate with each other through an internal connection path, and the processing circuit executes the method in the second aspect or any possible implementation of the second aspect to control the receiving pin to receive a signal and control the sending pin to send a signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1is a schematic diagram of the hardware structure of an electronic device shown as an example;

[0034] Figure 2 is a schematic diagram of an exemplary camera module;

[0035] Figure 3 A schematic diagram of the software structure of an electronic device is shown as an example;

[0036] Figure 4 A schematic diagram of an exemplary user interface;

[0037] Figure 5 is a schematic diagram of module interaction shown as an example;

[0038] Figure 6a and Figure 6b is a schematic diagram of module interaction shown as an example;

[0039] Figure 7 is an exemplary image frame exposure timing diagram;

[0040] Figure 8 is an exemplary image frame exposure timing diagram;

[0041] Fig. 9 is an exemplary image frame exposure timing diagram;

[0042] Fig.10 is an exemplary image frame exposure timing diagram;

[0043] Fig.11a and Fig.11b is a schematic diagram of module interaction shown as an example;

[0044] Fig.12 is an exemplary image frame exposure timing diagram;

[0045] Fig.13 is an exemplary image frame exposure timing diagram;

[0046] Fig.14 is an exemplary image frame exposure timing diagram;

[0047] Fig.15 Schematic diagram of the structure of the device shown as an example. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0050] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0051] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0052] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0053] To facilitate understanding, the technical terms involved in the embodiments of the present application are first explained.

[0054] Dynamic Range (Dyn) is the ratio of the maximum brightness to the minimum brightness in a video or image signal. In many fields, dynamic range is used to indicate the ratio of the maximum value to the minimum value of a variable. In digital images, dynamic range indicates the ratio between the maximum grayscale value and the minimum grayscale value within the displayable range of the image. The dynamic range in nature is quite large. The brightness of a night scene under the starry sky is about 0.001cd / m2, and the brightness of the sun itself is as high as 1,000,000,000cd / m2. Such a dynamic range reaches 1,000,000,000 / 0.001=10 13 However, in real scenes in nature, the brightness of the sun and the brightness of the stars are not obtained at the same time. For natural scenes in the real world, the dynamic range is 10 -3 to 10 6In most color digital images, each channel of R, G, and B is stored in an 8-bit byte. That is to say, the range of each channel is 0 to 255 grayscale levels. Here, 0 to 255 is the dynamic range of the image. Since the dynamic range of the same scene in the real world is in the range of 10-3 to 106, we call it High Dynamic Range (HDR). The dynamic range of ordinary pictures is Low Dynamic Range (LDR). The imaging process of a digital camera is actually the mapping of the high dynamic range of the real world to the low dynamic range of the photo. The mapping of the high dynamic range in the real world to the low dynamic range of the photo is often a nonlinear process.

[0055] Gain: The process of amplifying or reducing the electrical signal or digital signal after photoelectric conversion. Increasing the gain will make the image brighter than the actual scene; decreasing the gain will make the image darker than the actual scene.

[0056] Seamless switching: seamless switching is a fast switching without stopping. During the camera output process, the software automatically switches to different output modes according to the current environment scene. This switching process is called seamless switching. Functionally, there should be no delay during the switching process, otherwise it will cause lag for users.

[0057] Image output mode: In an embodiment of the present application, the terminal can switch the image output mode based on the current environmental information. The specific switching method will be described in detail in the following embodiments.

[0058] Exemplarily, the image output mode includes but is not limited to a Binning image output mode and an HDR image output mode, wherein the Binning mode is the default image output mode of the camera sensor.

[0059] HDR output modes include but are not limited to the following:

[0060] 1.SHDR (stagger HDR, overlapping exposure HDR technology) mode: The camera sensor outputs two frames of images. First, a long-exposure frame image is exposed to focus on collecting dark information, and then a short-exposure frame image is exposed to focus on collecting bright information. The two frames of images are simultaneously input into the chip platform image processing module, and one frame of image is generated after a certain algorithm processing.

[0061] 2. DCG (dual conversion gain) mode: The gain of each pixel of the camera sensor can be controlled individually. When working in DCG mode, only one exposure is performed, but the reading is divided into two times. One time, HCG (high conversion gain, that is, High CG) is used to capture dark information (also called low brightness information, that is, the brightness is less than or equal to the preset threshold), and the other time, LCG (low conversion gain, that is, low CG) is used to capture bright information (also called high brightness information, that is, the brightness is greater than or equal to the preset threshold). The HCG and LCG two frames of images are simultaneously input into the chip platform image processing module, and a frame of image is generated after a certain algorithm processing.

[0062] 3. IDCG (intra dual conversion gain) mode: The difference from the DCG mode is that in DCG, the camera sensor inputs two frames of HCG and LCG images into the chip platform at the same time, which are then fused into one frame of image by the chip platform; in IDCG mode, HCG and LCG are fused into one frame of image inside the camera sensor and then input into the chip platform.

[0063] 4. IDCG comb+VS (very short) mode: VS refers to a special mode, which refers to a frame with a very short exposure time. IDCG and VS follow the SHDR frame output method, first exposing a long exposure frame (IDCG, which is the fusion of HCG and LCG), and then exposing a short exposure frame (VS). The short exposure frame is used to capture dark information. The two frames of images are simultaneously input into the chip platform image processing module, and a frame of image is generated after a certain algorithm processing.

[0064] In the embodiments of the present application, the terminal device may be a mobile terminal with a shooting function, such as a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), or the like; or, it may be a professional shooting device such as a digital camera, a SLR camera / micro-single camera, a sports camera, a gimbal camera, a drone, etc. The embodiments of the present application do not limit the specific type of the terminal device.

[0065] Figure 1 1 shows a schematic diagram of the structure of the electronic device 100. It should be understood that Figure 1The illustrated electronic device 100 is merely one example of an electronic device, and the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have a different configuration of components. Figure 1 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0066] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0067] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0068] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0069] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0070] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0071] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0072] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0073] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0074] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0075] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0076] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0077] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0078] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0079] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0080] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0081] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0082] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0083] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0084] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0085] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0086] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0087] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0088] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0089] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0090] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0091] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0092] Figure 2 For a schematic diagram of an exemplary camera module, please refer to Figure 2 , Figure 2(a) and (b) schematically show the front and back of the electronic device 100. The front of the electronic device 100 can be understood as the side facing the user when the user uses the electronic device 100, and the back of the electronic device 100 can be understood as the side facing away from the user when the user uses the electronic device 100.

[0093] The camera module 102 is used to capture still images or videos. The camera module 102 can be arranged on the front and / or back of the electronic device 100. When the camera module 102 is arranged on the front of the electronic device 100, the front camera 102-1 can be used to shoot the scene on the front side of the electronic device 100, such as for selfies, and in some embodiments it can be referred to as a front camera. When the camera module 102 is arranged on the back of the electronic device 100, the rear camera 102-2 can be used to shoot the scene on the back side of the electronic device 100, and in some embodiments it can be referred to as a rear camera. When shooting, the user can select the corresponding camera module according to the shooting requirements.

[0094] It should be noted that the embodiment of the present application does not limit the number of camera modules 102, which can be one, two, four or even more. For example, one or more camera modules 102 can be set on the front of the electronic device 100, and / or one or more camera modules 102 can be set on the back of the electronic device 100. When multiple camera modules 102 are set, the multiple camera modules 102 can be completely the same or different, for example, the multiple camera modules 102 have different lens optical parameters, different lens setting positions, different lens shapes, etc. The embodiment of the present application does not limit the relative positions of the multiple camera modules when they are set.

[0095] The image sensor component is mainly used for imaging. Specifically, the image sensor component collects images and exposes image frames.

[0096] Exemplarily, the image sensor is further configured to output a start of frame delimiter (SOF) signal for identifying the frame head of the current frame, and output an end of frame delimiter (EOF) signal for identifying the frame end of the current frame. Exemplarily, the Frame Time of each frame is the length from SOF to the next SOF, including the effective frame length between SOF and EOF and the non-exposure blank line (vertical blank, vblank, also known as blank invalid line time, vertical blanking or field blanking) in each frame. The specific corresponding relationship will be described in Figure 7 In the description.

[0097] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0098] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0099] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0100] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0101] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0102] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0103] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0104] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0105] Figure 3 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0106] The layered architecture of the electronic device 100 divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided from top to bottom into an application layer, a framework layer, a hardware abstract layer (HAL), and a kernel layer. It is understandable that Figure 2 This is just an example, that is, the layers divided in the electronic device are not limited to Figure 3 The layers shown, for example, between the application framework layer and the HAL layer, may also include an Android runtime and system libraries layer, etc.

[0107] The application layer can include a series of application packages.

[0108] like Figure 3 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0109] The framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0110] like Figure 3 As shown, the application framework layer may include a window manager, a content provider, a view system, a camera service, a resource manager, a notification manager, and the like.

[0111] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0112] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0113] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0114] The camera service may also be referred to as a camera service, etc., which is not limited in this application. The service is used to call a camera (including a front camera and / or a rear camera) in response to an application request.

[0115] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0116] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0117] The HAL layer is used to abstract the hardware and provide a virtual hardware platform for the operating system. The HAL layer may include but is not limited to: decision modules, CAMX architecture, etc.

[0118] The decision module may be a multi-camera decision module, which may determine the camera in the camera group that actually performs image acquisition based on scene information (including information such as ambient brightness and ambient dynamic range), which may be referred to as a target camera. The target camera may be a front camera or a rear camera in an electronic device, etc.

[0119] In addition, the decision module can also determine the target image output mode that the target camera needs to enable. The image output mode can refer to the above description and will not be repeated here. In addition, the method of selecting the target image output mode can be selected in combination with the shooting scene, shooting parameters (zoom ratio), etc. The implementation details can refer to the description in the subsequent embodiments, which will not be repeated here.

[0120] The CAMX architecture is a logical architecture in the HAL layer. The CAMX architecture includes sensor nodes and image front end (IFE) nodes (IFE node). Among them, the sensor nodes in the CAMX architecture correspond one-to-one to the camera sensors in the electronic device, and the sensor nodes can configure the camera sensors to enable the specified image output mode. For example, the sensor node can pass the camera parameters (also called configuration parameters) required to enable the target image output mode to the target camera, so that the target camera sensor uses the target image output mode to output images. The above-mentioned IFE node is used to configure the preprocessing parameters required for the operation of IFE, so that IFE can preprocess the preview stream collected by the target camera according to the target image output mode. For example, the image frames of the preview stream and the video stream are processed by color correction, downsampling, de-mosaicing, etc.

[0121] The CAMX architecture also includes a CAMX conversion interface (camera serial interface decoder, CSL). The CAMX CSL can receive configuration parameters from the sensor node and convert them into I / O control instructions that can be recognized by the kernel layer. In addition, the CAMX CSL can also instruct the CRM to pass the I / O control instructions corresponding to the camera parameters to the corresponding camera driver through V4L2 in the kernel layer, and the camera driver writes the I / O control instructions corresponding to the camera parameters into the camera sensor, so that the camera sensor can output images according to the specified output mode.

[0122] The CAMX CSL receives the pre-processing parameters from the IFE node and converts them into I / O control instructions that can be recognized by the kernel layer. Then, through V4L2, it instructs the CRM to pass the I / O control instructions corresponding to the pre-processing parameters to the corresponding ISP driver, and the ISP driver writes the I / O control instructions corresponding to the pre-processing parameters into the IFE, so that the IFE has the ability to pre-process the image frames collected according to the target output mode.

[0123] The kernel layer contains at least the Linux video device driver (Video for linux2, V4L2) and the camera driver.

[0124] The camera driver includes but is not limited to: a camera request manager (CRM), a sensor driver, an IFE driver, and the like.

[0125] The V4L2 can be called by the HAL layer. The CRM is used to manage the drivers corresponding to the camera-related devices in the kernel layer, such as the IFE driver and the sensor driver. In some examples, the HAL can instruct the CRM to manage the drivers corresponding to the camera-related devices through V4L2.

[0126] The camera driver can be used to drive a hardware module with a shooting function, such as a camera sensor. In other words, the camera driver is responsible for data interaction with the camera sensor. Of course, the kernel layer can also include driver software such as an audio driver and a sensor driver, and the present application embodiment does not impose any restrictions on this.

[0127] The hardware includes a camera group (the description can be found above and will not be repeated here) and a display screen, etc. The camera group includes but is not limited to physical devices such as IFE, sensors, and lenses.

[0128] Understandably, Figure 3 The layers in the software structure shown and the components included in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown, and each layer may include more or fewer components, which is not limited in the present application.

[0129] Figure 4 is a schematic diagram of an exemplary user interface. Figure 4 , the user clicks the camera application. In response to the received user operation, the camera application displays an image preview interface 201 in the display interface 200. The image preview interface 201 displays the preview image captured by the camera. In the embodiment of the present application, only the processing method of the preview image is used as an example for explanation. In other embodiments, the scheme in the embodiment of the present application can also be applied to other scene shooting scenes such as video recording, and the present application does not limit it.

[0130] Figure 5 This is an exemplary diagram of module interaction. Figure 5 Specifically, after the user clicks on the camera application, the camera application sends a trigger instruction to the camera service to instruct the camera service to be called. Exemplarily, the trigger instruction includes shooting mode information, etc. In the embodiment of the present application, after the camera application is started, it enters the shooting mode by default, such as Figure 4 Correspondingly, the trigger instruction includes but is not limited to the photographing mode information.

[0131] In response to the trigger instruction, the camera service sends a call instruction to the decision module for calling the camera, wherein the call instruction includes but is not limited to shooting mode information, such as a photo mode.

[0132] The decision module can determine the camera in the camera group that actually performs image acquisition based on scene information (including information such as ambient brightness and ambient dynamic range), which can be called the target camera. The target camera can be a front camera or a rear camera in an electronic device.

[0133] The decision module can also determine the target output mode that the target camera needs to enable. In an embodiment of the present application, the decision module can first select a default output mode, which is a binning mode. In the subsequent shooting process, the decision module can select a corresponding output mode based on each image frame captured by the camera and other parameters, such as a combination of shooting scenes, shooting parameters (zoom ratio), etc., to achieve output mode switching.

[0134] Still refer to Figure 5 After the decision module determines the target camera, it can send information such as the target image output mode to the IFE node and Sensor node corresponding to the target camera in the CAMX architecture to instruct the Sensor node and IFE node to generate corresponding image output mode configuration information based on the target image output mode.

[0135] Exemplarily, the Sensor node configures the camera sensor to enable the specified output mode in response to the received target output mode. Specifically, the Sensor node generates the Sensor configuration information. Among them, the Sensor configuration information includes but is not limited to: the output mode configuration information corresponding to the target output mode (also referred to as the Sensor output configuration information), which can also be understood as the output mode configuration information required to enable the target output mode. Optionally, the Sensor configuration information also includes other camera parameters (also referred to as configuration parameters or exposure parameters), such as frame length, etc., which are not limited in this application. In some examples, the camera parameters and the Sensor output configuration information can be issued at the same time or at different times. Among them, the camera parameters are used for the Sensor to expose the image frame, and accordingly, the camera parameters need to be issued before the image frame is exposed. The output mode configuration parameters are used to output the image frame, and accordingly, the output configuration parameters need to be issued before the retreat is true to output the image, which is not limited in this application.

[0136] Still refer to Figure 5 Exemplarily, the IFE generates IFE configuration information (also referred to as IFE output configuration information) in response to the instruction of the decision module. The IFE configuration information includes, but is not limited to: preprocessing parameters required for the IFE operation, so that the IFE can preprocess the preview stream collected by the target camera according to the target output mode. For example, color correction, downsampling, de-mosaicing, etc. are performed on the image frames of the preview stream and the video stream.

[0137] In the embodiment of the present application, the configuration information of IFE and Sensor needs to correspond. For example, when the Sensor works in binning output mode, the preprocessing parameters required for IFE operation also correspond to the binning output mode. If the Sensor works in other modes, such as HDR mode, and the preprocessing parameters required for IFE operation are binning output mode, IFE cannot process the image input by the Sensor and will discard the image frame. Specific examples will be described below ( Figure 8 ) is described in .

[0138] Please continue to refer to Figure 5 , exemplarily, the IFE node sends the IFE configuration information to the conversion interface. The Sensor node sends the Sensor configuration information to the conversion interface. The conversion interface instructs the CRM to pass the I / O control instructions corresponding to the camera parameters to the corresponding camera driver through V4L2 in the kernel layer. The camera driver writes the I / O control instructions corresponding to the camera parameters into the camera sensor, so that the camera sensor outputs images according to the specified output mode.

[0139] Specifically, V4L2 sends the IFE configuration information and the Sensor configuration information to the CRM. The CRM sends the IFE configuration information to the IFE driver at an appropriate timing (the specific timing will be described below in conjunction with the accompanying drawings), and sends the Sensor configuration information to the sensor driver. After the IFE driver receives the IFE configuration information, it writes the IFE configuration information to the IFE. After the sensor driver receives the Sensor configuration information, it writes the Sensor configuration information to the Sensor. The Sensor will capture images according to the Sensor configuration information. For example, the Sensor will capture images according to the camera parameters and the output mode configuration information in the Sensor configuration information. The Sensor outputs the captured image to the IFE. The IFE can process the image based on the IFE configuration information. For example, the IFE can pre-process the image based on the pre-processing parameters corresponding to the target output mode.

[0140] Figure 6a and Figure 6b This is an exemplary diagram of module interaction. Figure 6a For example, during the photo-taking process, the decision module will make a decision for each frame. If it is determined that the output mode remains unchanged, there is no need to send information such as the output mode. In this example, the Sensor node will send the corresponding Sensor configuration information to the CRM in advance (for example, one frame in advance, or multiple frames in advance, which is not limited in this application) for each frame (the specific data transmission process can be referred to in Figure 4, which will not be repeated here). Among them, since the decision module does not switch the image output mode, the corresponding Sensor configuration information includes camera parameters such as frame length, but does not include the Sensor output mode configuration information. In other words, when the Sensor exposes the image frame corresponding to the Sensor configuration information, it is exposed according to the current (i.e., the most recent) image output mode.

[0141] For example, after the CRM obtains the sensor configuration information, it can send the sensor configuration information to the sensor driver at an appropriate time, so that the sensor driver writes the sensor configuration information into the register corresponding to the sensor. The specific implementation method will be described in Fig. 9 In the description.

[0142] Please refer to Figure 6b , if the decision module determines to switch the image output mode, the decision module sends the image output mode to the IFE node and the Sensor node to indicate the switching of the target image output mode. In this example, the Sensor node generates Sensor configuration information. Among them, the Sensor configuration information includes but is not limited to: the image output mode configuration information corresponding to the target image output mode, which can also be understood as the image output mode configuration information required to enable the target image output mode. Optionally, the Sensor configuration information also includes other camera parameters (also referred to as configuration parameters), such as frame length, etc., which are not limited in this application.

[0143] In response to the instruction of the decision module, IFE generates IFE configuration information. The IFE configuration information includes but is not limited to: pre-processing parameters required for IFE operation, so that IFE can pre-process the preview stream collected by the target camera according to the target output mode. For example, color correction, downsampling, de-mosaicing and other processing are performed on the image frames of the preview stream and the video stream.

[0144] Figure 7 For an exemplary image frame exposure timing diagram, please refer to Figure 7, before N-1 frame (including N-1 frame), the Sensor uses mode A (also known as the first output mode) to output each image frame. In this example, the configurations of the Sensor and IFE (including frame length, output mode, etc.) are configured to take effect on the new configuration in the N+1 frame. That is to say, for the configuration information of the N frame, the Sensor node and the IFE node send the Sensor configuration information and IFE configuration information of N frames to the CRM before the N frame (it can be at least one frame in advance, which is not limited in this application). At the N frame, the CRM sends the Sensor configuration information and IFE configuration information of N frames to the Sensor driver and the IFE driver, so that the Sensor driver and the IFE driver write the Sensor configuration information and the IFE configuration information to the Sensor and the IFE respectively within the N frame. The Sensor and IFE are configured to take effect on the configuration information of N frames at N+1, and accordingly, the Sensor and IFE will take effect on the Sensor configuration information and the IFE configuration information of N frames at the N+1 frame.

[0145] For example, refer to Figure 7 , in frame N-1, the Sensor uses mode A (also referred to as the first image output mode) for image exposure. Before moment T1-1, the decision module decides on the target image output mode and the target camera based on environmental information, wherein the target image output mode is the second image output mode (for example, mode B), that is, the current target image output mode (ie, the first image output mode) is switched to the second image output mode (mode B). Optionally, the decision module can make a decision at any time before frame N (ie, moment T3), for example, it can be at frame N-1 or at frame N-2, which is not limited in this application. The time for sending the configuration information and the time for writing the configuration information in the embodiments of the present application are only illustrative examples and can be set according to actual needs, which is not limited in this application.

[0146] Exemplary, combined Figure 6b After the decision module determines the target camera, it can send information such as the target image output mode to the IFE node and Sensor node corresponding to the target camera in the CAMX architecture to instruct the Sensor node and IFE node to generate corresponding image output mode configuration information based on the target image output mode.

[0147] Exemplarily, the Sensor node configures the camera sensor to enable the specified output mode in response to the received target output mode. Specifically, the Sensor node generates Sensor configuration information for N frames. Among them, the Sensor configuration information includes but is not limited to: the output mode configuration information corresponding to the target output mode, which can also be understood as the output mode configuration parameters required to enable the target output mode. Exemplarily, the output mode configuration information includes but is not limited to switching configuration information and effective configuration information. The switching configuration information includes the output mode configuration parameters required for the target output mode, and the effective configuration information is used to indicate that the switching configuration information has been completely written.

[0148] Optionally, the Sensor configuration information also includes other camera parameters (also referred to as configuration parameters), such as frame length, etc., which are not limited in this application.

[0149] Still refer to Figure 5 Exemplarily, the IFE node generates IFE configuration information in response to the instruction of the decision module. The IFE configuration information includes but is not limited to: preprocessing parameters required for IFE operation, so that IFE can preprocess the preview stream collected by the target camera according to the target output mode. For example, color correction, downsampling, de-mosaicing and other processing are performed on the image frames of the preview stream and the video stream.

[0150] Still refer to Figure 7 At time T1-1 (which can be any time before N frames, not limited in this application), the Sensor node sends the Sensor configuration information of N frames to the CRM (including the image output mode configuration information and camera parameters corresponding to the second image output mode, etc.), and the IFE node sends the IFE configuration information of N frames to the CRM. The CRM obtains the Sensor configuration information and IFE configuration information corresponding to N frames (for specific data flow, please refer to Figure 6b , which will not be described in detail here). Optionally, as described above, the camera parameters and image output configuration information in the Sensor configuration information can be sent to the CRM at different times. The specific sending process can refer to the existing technical embodiments, which is not limited in this application.

[0151] In an embodiment of the present application, the Sensor is pre-configured with effective point information. Based on the effective point information, the Sensor can obtain the Sensor configuration information at a specified time (the time is called the effective point) in each frame, and start image exposure based on the Sensor configuration information. The effective point is the starting point of the exposure duration of the image frame, and the EOF of the image frame is the end point of the exposure duration. Optionally, in some examples, the effective point of the image output configuration information may be different from the effective point of the camera parameters. Specifically, as described above, the camera parameters are mainly used when the Sensor exposes the image frame. Therefore, the camera parameters are effective at the exposure starting point, and the Sensor can expose the image frame based on the camera parameters. The Sensor image output configuration information is mainly used by the Sensor to output image frames (for example Figure 7 The time between T3 and T5 in the figure is within the frame length of N frames. The image output time corresponds to the image frame length, for example Figure 7 The time interval between T3 and T5 in the output image is N frames, that is, the frame length is N frames. The Sensor outputs N frames within the output image duration. Optionally, within the output image duration, the Sensor actually performs the output action (i.e. outputs the exposed image to the IFE) between SOF and EOF of the N frames, and at other times within the N frames (such as T4 to T5), the Sensor performs other configurations. Accordingly, the effective point of the output image configuration information of the Sensor is set before the image frame is output, that is, the output image configuration information of the Sensor takes effect as long as it is before the image frame is output. The specific example will be shown in Figure 12 to Figure 14 In the description.

[0152] Optionally, the effective point information is used to indicate the relative time between the effective point and the SOF. For example, the effective point information indicates that the relative time is X. Accordingly, the Sensor starts to expose the image frame at a time X before the SOF of each frame.

[0153] For an example, please refer to Figure 7 At time T2-1, the Sensor determines the time corresponding to the effective point of frame N based on the effective point information. In this example, it is the exposure start point of frame N. The Sensor obtains the written Sensor configuration information and starts to expose frame N based on the camera parameters in the Sensor configuration information. Among them, the interval between time T2-1 and the SOF of frame N (that is, time T3) is X time.

[0154] For example, at time T3, the Sensor outputs N frames based on the Sensor output configuration information. The actual output duration is from time T3 to T4, that is, from the SOF to the EOF of the N frames. In the embodiment of the present application, the duration of the N frames (that is, the time T3 to T4) is referred to as the output duration of the N frames.

[0155] Exemplarily, the sensor obtains the already written sensor configuration information at the effective point. Among them, the image output mode configuration information in the current configuration information of the sensor indicates the first image output mode, that is, mode A. Accordingly, the sensor exposes and outputs N frames according to the camera parameters in the configuration information and the first image output mode.

[0156] Please continue to refer to Figure 7 As mentioned above, for the configuration information of frame N, it is written in frame N and takes effect in frame N+1. Specifically, at time T3-1 within frame N (generally any time between SOR and EOF, which can be set according to actual needs and is not limited in this application), CRM sends the Sensor configuration information corresponding to frame N (including camera parameters and image output mode configuration information corresponding to the second image output mode) to the Sensor driver, and CRM sends the IFE configuration information corresponding to frame N (including image output mode configuration information (the image output mode configuration information indicates the second image output mode)) to the IFE driver.

[0157] At time T3-1, the Sensor driver and the IFE driver write corresponding configuration information to the Sensor and the IFE, respectively. It should be noted that there may be a delay between the above two writing actions, which is not limited in this application.

[0158] In an embodiment of the present application, as described above, the output mode configuration information of the Sensor may further include switching configuration information and effective configuration information. Exemplarily, the Sensor driver writes the switching configuration information to the register corresponding to the Sensor (recorded as the output mode register). After the Sensor driver writes all the switching configuration information to the register, the Sensor writes the effective configuration information to the corresponding register (recorded as the effective register). The writing method of other configuration information (such as camera parameters) can refer to the existing technology, and this application is not limited.

[0159] Exemplarily, the IFE driver writes N frames of IFE configuration information into a register corresponding to the IFE.

[0160] Optionally, the number of registers written by the IFE is less than the number of registers that the Sensor needs to write. Usually, the configuration information of the Sensor is long and the number of registers that need to be written is large.

[0161] At time T4-1, the Sensor determines the time corresponding to the effective point of frame N+1 based on the effective point information, which is the exposure time of frame N+1 in this example. The Sensor obtains the written Sensor configuration information (that is, the configuration information of frame N written at T3-1), and starts to expose frame N+1 based on the Sensor configuration information. The interval between time T4-1 and the SOF of frame N (that is, time T3) is X time.

[0162] Exemplarily, at time T5, the Sensor outputs the N+1 frame based on the Sensor output configuration information. The actual output duration is from time T5 to T6, that is, from the SOF to EOF of the N+1 frame. In the embodiment of the present application, the duration of the N+1 frame (i.e., time T5 to T7) is referred to as the output duration of the N+1 frame.

[0163] For example, the Sensor obtains the configuration of the effective register at the effective point. In an example, if the Sensor has completely written the output mode configuration information of N frames before T4-1, that is, the Sensor has written all the switching configuration information into the corresponding output mode register, and written the effective configuration information into the corresponding effective register. Accordingly, the Sensor determines to start taking into effect the configuration of the output mode register corresponding to the effective register based on the configuration of the effective register. In other words, the Sensor outputs the N+1 frame based on the output mode configuration information of the N frame (that is, modeB, also referred to as the second output mode).

[0164] In another example, if the Sensor fails to write the N-frame output mode configuration information completely before T4-1, that is, the Sensor fails to write the effective configuration information into the corresponding effective register. Accordingly, the Sensor determines at the effective point that the effective register has not been written completely, then the Sensor will not take effect on the configuration information being written, but on the old configuration information. For example, Figure 8 For an exemplary image frame exposure timing diagram, please refer to Figure 8 At time T3-1 within frame N (generally any time between SOR and EOF, which can be set according to actual needs and is not limited in this application), CRM sends Sensor configuration information corresponding to frame N to the Sensor driver (including camera parameters and image output mode configuration information (the image output mode configuration information indicates the second image output mode)), and sends IFE configuration information corresponding to frame N to the IFE driver (including image output mode configuration information (the image output mode configuration information indicates the second image output mode)).

[0165] At time T3-1, the Sensor driver and the IFE driver write corresponding configuration information to the Sensor and the IFE, respectively. It should be noted that there may be a delay between the above two writing actions, which is not limited in this application.

[0166] Exemplarily, the sensor driver writes the switching configuration information into the corresponding register (recorded as the image output mode register). After all the switching configuration information is written into the register, the sensor driver writes the effective configuration information into the corresponding register (recorded as the effective register). The writing method of other configuration information (such as camera parameters) can refer to the existing technology, and this application does not limit it.

[0167] Exemplarily, the IFE driver writes N frames of IFE configuration information into a register corresponding to the IFE.

[0168] like Figure 8 As shown in the figure, in this example, the switching configuration information of the Sensor is long, and there are many output mode registers that need to be written, which causes the Sensor to fail to completely write the switching configuration information before the effective point 1 (i.e., T4-1) of the N+1 frame. At the effective point 1, since the Sensor fails to detect that the effective register writes the effective configuration information, it can also be understood that the Sensor has not received the effective configuration information of the N frame. Accordingly, the Sensor still outputs the N+1 frame according to the old output mode configuration information (i.e., modeA).

[0169] Sersor outputs the exposed N+1 frame to IFE. The current configuration information of IFE is the new image output mode configuration information, which indicates the second image output mode. Accordingly, IFE detects that the image output mode of IFE (i.e., the second image output mode, modeB) does not match the image output mode adopted by the Sensor (i.e., the first image output mode, modeA). IFE will discard the N+1 frame. Accordingly, the old image frame, i.e., frame N, will be displayed on the display. In other words, N frames are displayed on the display during the time when frame N+1 should be displayed. From the user's perspective, the user will perceive the image freeze.

[0170] Assume that at time T5-1, after the sensor has completely written the switching configuration information, it writes the effective configuration information into the effective register. At time T6-1, the sensor obtains the configuration information at the effective point 2. The sensor detects that the effective configuration information has been written into the effective register, and accordingly, the sensor outputs the N+2 frame according to the new switching configuration information (i.e., the second output mode, Mode B).

[0171] The Sensor outputs the N+2 frame to the IFE. The current configuration information of the IFE is the new image output mode configuration information, that is, it indicates the second image output mode. Accordingly, the IFE detects that the image output mode of the IFE (that is, the second image output mode, modeB) matches the image output mode adopted by the Sensor (that is, the second image output mode, mode B). The IFE can further process the N+2 frame based on the preprocessing parameters (that is, the configuration information) corresponding to the image output mode.

[0172] The present application proposes a method for setting photographing parameters, which avoids the frame loss problem by delaying the effective point of N-frame configuration information.

[0173] Fig. 9 For an exemplary image frame exposure timing diagram, please refer to Fig. 9 Before time T-1, the decision module decides to switch the current first image output mode (i.e., mode A) to the second image output mode (mode B). Optionally, the decision module can decide in which frame to switch the image output mode, or the Sensor node and the IFE node can decide in which frame to switch the image output mode, which is not limited in this application.

[0174] Combination Figure 6b , the decision module outputs the image output mode information to the Sensor node and the IFE node, indicating that the first image output mode configuration information is switched to the second image output mode. Among them, the Sensor and IFE are still configured to take effect on the configuration information of the Nth frame in the N+1th frame.

[0175] like Fig. 9 As shown, in this example, the purpose of the scheme is to achieve the effect of delaying the effective point by lengthening the frame length of N frames, specifically lengthening the duration of vblank (the exposure duration remains unchanged). As mentioned above, the configuration of the Sensor takes effect at the N+1 frame. Therefore, if the frame length of N frames needs to be lengthened, that is, the new frame length parameter takes effect at the N frame, then the Sensor needs to write the new frame length at the N-1 frame to obtain the new frame length before the effective point of the N frame (that is, the T4-1 moment, which can also be understood as the exposure start point), so that the Sensor can expose and output N frames based on the new frame length.

[0176] For this purpose, at time T1-1 (which can be any time after receiving the image output mode information from the decision module and before T2-1, and this application does not limit it), the Sensor node responds to the instruction of the decision module and determines that the image output mode needs to be switched in frame N. The Sensor node generates configuration information for the N-1 frame. The configuration information of the N-1 frame includes but is not limited to camera parameters, wherein the frame length in the camera parameters is the second frame length. The current default frame length is the first frame length, for example, 33ms, which can be set according to actual needs and is not limited by this application.

[0177] Optionally, the second frame length is set to the first frame length * Ratio (ratio). The first frame length is the default frame length, which is usually set to 33ms. The Sensor node can determine the value of Ratio based on the value of the product, device, and switching scenario configuration. In the embodiment of the present application, the value of Ratio is usually greater than 1 and less than 2. That is, Fig. 9 As shown, the extended duration of the N frame (ie, time T7 to T8) is smaller than the original frame length (ie, the first frame length) of the N frame (ie, time T5 to T7).

[0178] Optionally, since the output method of frame N-1 remains unchanged, Figure 6a As shown, the IFE node does not need to send the image output mode configuration information in frame N-1. It should be noted that other configuration information except the frame length and image output mode configuration information is implemented according to the existing technical process, and this application will not repeat it.

[0179] For example, Figure 6a As shown, the Sensor node sends the configuration information of frame N-1 to the CRM. The CRM obtains the Sensor configuration information of frame N-1 (including the second frame length).

[0180] At time T2-1, it is the effective point of frame N-1. The Sensor still exposes frame N-1 based on the configuration information such as the first frame length. For example, at time T3, the Sensor outputs frame N-1 based on the Sensor output configuration information. The actual output duration is from time T3 to T4, that is, from SOF to EOF of frame N. In the embodiment of the present application, the duration of the N-1 frame (i.e., time T3 to T4) is referred to as the output duration of frame N.

[0181] At time T3-1 (which can be after the Sensor sends the Sensor configuration information of frame N-1 and before time T5 (i.e., frame N), which is not limited in this application), the Sensor node generates configuration information for frame N. As described above, the Sensor node has obtained from the decision module that the image output mode of frame N will be switched to the second image output mode. Accordingly, the configuration information of frame N includes image output mode configuration information, which includes but is not limited to switching configuration information and effective configuration information. The switching configuration information is used to instruct the Sensor to switch to the second image output mode (i.e., mode B). Furthermore, it is expected that the frame length of frame N+1 will be restored to the first frame length, and accordingly, the frame length in the camera parameters in the configuration information of frame N is the first frame length.

[0182] The Sensor node sends N frames of Sensor configuration information to the CRM. For the specific sending process, please refer to Figure 6b , I will not go into details here.

[0183] Exemplarily, the IFE node also generates IFE configuration information of frame N. The IFE configuration information includes but is not limited to preprocessing parameters required for the second image output mode.

[0184] The IFE node sends N frames of IFE configuration information to the CRM.

[0185] The CRM obtains the Sensor configuration information and IFE configuration information of N frames.

[0186] It should be noted that there may be a certain delay between the generation and delivery of configuration information, which is not limited in this application. Optionally, there may also be a delay between the delivery of configuration information of the IFE node and the delivery of configuration information of the Sensor node, which is not limited in this application.

[0187] At time T3-2 (which can be any time between the SOF of frame N-1 (i.e., time T3) and before the exposure of frame N (i.e., time T4-1), this application does not limit it), CRM sends the configuration information of frame N-1 (including the second frame length) to the Sensor driver. The Sensor driver writes the configuration information of frame N-1 into the register corresponding to the Sensor.

[0188] At time T4-1, which is the effective point of frame N, in this example, it is the exposure start point of frame N, and the sensor configuration information of frame N-1 takes effect. That is, the sensor exposes frame N based on the configuration information of frame N-1 that has been written (including the second frame length and the first image output mode information). Among them, the frame length of frame N is the second frame length, that is, time T5 to T8. The image output mode of frame N is the first image output mode, that is, mode A.

[0189] At time T5-1 (which can be any time after the Sensor node sends the N frame configuration information and before the N+1 frame, and this application does not limit it), the Sensor node generates the Sensor configuration information of the N+1 frame. Among them, since the decision module does not indicate to switch the image output mode in the N+1 frame, accordingly, the Sensor configuration information of the N+1 frame includes but is not limited to the camera parameters, but does not include the image output mode configuration information. Among them, the frame length in the camera parameters of the N+1 frame is the first frame length.

[0190] For example, the Sensor node sends the Sensor configuration information of the N+1 frame to the CRM. The CRM obtains the Sensor configuration information of the N+1 frame. For the specific transmission process, please refer to Figure 6a , I will not go into details here.

[0191] At time T5-2 (which can be any time before time T6-1), CRM sends the Sensor configuration information of N frames acquired at time T3-1 to the Sensor driver, and sends the IFE configuration information of N frames to the IFE driver. Among them, the Sensor configuration information includes but is not limited to camera parameters (where the frame length is the first frame length) and image output mode configuration information. The image output mode configuration information includes but is not limited to switching configuration information and effective configuration information. The switching configuration information indicates that the image output mode is the second image output mode (i.e., mode B).

[0192] After the Sensor driver writes all the switching configuration information into the corresponding register, the Sensor writes the effective configuration information into the effective register. In addition, the IFE driver writes the IFE configuration information of N frames into the corresponding register.

[0193] Exemplarily, as described above, the Sensor is configured with effective point information to indicate the position of the effective point in the frame. In the embodiment of the present application, the effective point is spaced from the SOF of the image frame by a time length of X. Fig. 9 As shown, since the frame length of frame N is the second frame length, it adds the extended frame length, so the original effective point of frame N+1 (i.e., time T6-1) is moved backward, and the actual effective point of frame N+1 (i.e., time T7-1) is separated from the original effective point (i.e., time T6-1) by the "extended time length". In addition, the effective point of frame N+1 (i.e., time T7-1) and the SOF of frame N+1 are separated by the time length X.

[0194] Still refer to Fig. 9At time T7-1, which is the effective point of frame N+1, or the exposure start point of frame N+1, the Sensor detects that the effective configuration information has been written into the effective register, that is, the configuration information of frame N has been completely written. The Sensor exposes and outputs frame N+1 based on the sensor configuration information of frame N. That is, the frame length of frame N+1 is the first frame length, and the output mode is the second output mode (that is, mode B).

[0195] The Sensor outputs the image frame to the IFE. The IFE also takes effect on the IFE configuration information of the Nth frame at the N+1th frame, that is, the IFE processes the image collected by the Sensor based on the second image output mode based on the preprocessing parameters corresponding to the second image output mode.

[0196] In this way, Fig. 9 As shown, even if the time for the Sensor to write the configuration information is long (for example, the time point for writing the effective register has exceeded the original effective point), the effective point of the N+1 frame is delayed, so that the Sensor has enough time to write the configuration information, and the writing of the configuration information of the N frame can be completed before the effective point of the N+1 frame. This further enables the Sensor to collect image frames in time at the N+1 frame based on the new output mode.

[0197] In a possible implementation, if the Sensor node predicts that the time it takes to write the configuration information to the Sensor may not exceed the original effective point based on the length of the Sensor configuration information, the device's capabilities, etc., in this example, the Sensor node can also configure the Ratio to 1, that is, the Sensor still exposes and outputs N frames according to the first duration.

[0198] The embodiment of the present application also provides another shooting parameter setting method, in which the Sensor is configured to take effect on the configuration information of N frames in the N+2 frame. The IFE is still the configuration information of N frames that takes effect on the N+1 frame. Correspondingly, the CRM performs data synchronization based on the N+1 frame effectiveness timing of the IFE and the N+2 frame effectiveness timing of the Sensor. That is, the CRM sends the Sensor configuration information of N frames to the Sensor driver in the N frame, so that the Sensor writes the Sensor configuration information of N frames in the N frame, and the Sensor configuration information takes effect in the N+2 frame, that is, the CRM takes effect on the configuration information in the N+2 frame to the CRM. In addition, the CRM sends the IFE configuration information of N frames to the IFE driver in the N+1 frame, so that the IFE writes the IFE configuration information of N frames in the N+1 frame, and the IFE configuration information takes effect in the N+2 frame. In this way, in the scenario where the Sensor writes the Sensor configuration information of frame N in frame N, the Sensor may finish writing the Sensor configuration information before the effective point of frame N+1, or it may finish writing the Sensor configuration information before the effective point of frame N+2. By writing the effective configuration information of the Sensor in frame N+1, the present application can ensure that the IFE configuration information is synchronized with the Sensor configuration information in frame N+2, that is, it can correctly switch to the second image output mode in frame N+2.

[0199] Combine the following Fig.10 The above method is described in detail. Fig.10 For an exemplary image frame exposure timing diagram, please refer to Fig.10 At time T1-1 (which may be any time before time T3, not limited in this application), the decision module decides to switch the first image output mode to the second image output mode based on the scene environment information. Figure 6b , the decision module indicates the second output mode to the IFE node and the Sensor node to trigger the IFE node and the Sensor node to generate corresponding configuration information.

[0200] The Sensor node generates N frames of Sensor configuration information in response to the instruction of the decision module. The Sensor configuration information includes but is not limited to camera parameters and image output mode configuration information. The camera parameters include but are not limited to the frame length of N frames (for example, the first frame length). The Sensor image output mode configuration information includes but is not limited to switching configuration information and effective configuration information. The switching configuration information includes configuration parameters corresponding to the second image output mode.

[0201] The IFE node generates N frames of IFE configuration information in response to the instruction of the decision module, wherein the IFE configuration information includes but is not limited to preprocessing parameters corresponding to the second image output mode, so that the IFE can preprocess the preview stream collected by the target camera according to the target image output mode (i.e., the second image output mode).

[0202] The Sensor node sends N frames of Sensor configuration information to the CRM, and the IFE node sends N frames of IFE configuration information to the CRM. The CRM obtains N frames of Sensor configuration information and IFE configuration information. For specific data transmission flow, refer to Figure 6b , I will not go into details here.

[0203] Exemplarily, as described above, based on the timing of the configuration information of N frames taking effect at the Sensor in frame N+2 and the timing of the configuration information of N frames taking effect at the IFE in frame N+1, the CRM controls the timing of the switching configuration information of the Sensor and the effective configuration information and the sending of the IFE configuration information, so as to achieve data synchronization between the IFE configuration information and the Sensor configuration information in frame N+2. Fig.10 In the example shown, at time T3-1 (which can be any time within N frames, not limited in this application), Fig.11a As shown, the CRM outputs other configuration information except the effective configuration information to the Sensor driver, such as switching configuration information and camera parameters (such as frame length, etc.).

[0204] Still refer to Fig.10 , at time T2-1, the Sensor exposes N frames, and outputs N frames at time T3. Among them, the exposure of N frames uses the configuration information of N-1 frame. For example, the frame length of N frames is the first frame length, and the output mode is the first output mode (ie, Mode A). At time T3-1 (which can be any time after the SOF of N frames and before the effective point of N+1 frames, not limited in this application), the Sensor driver writes the Sensor configuration information of N frames into the output mode register corresponding to the Sensor, so that the Sensor outputs images according to the specified output mode (ie, the second output mode). Exemplarily, the Sensor configuration information includes but is not limited to camera parameters and switching configuration information. Camera parameters include but are not limited to the frame length of N frames (for example, the first frame length).

[0205] At time T4-1, the Sensor determines that this time point is the effective point of frame N+1 based on the effective point configuration information (for example, the effective point is X time away from the SOF of frame N), which is the exposure start point of frame N+1 in this example. Since the effective configuration information of frame N has not been written into the corresponding register, that is, the Sensor has not obtained the effective configuration information of frame N. Therefore, the Sensor still outputs images according to the old configuration information. That is, the frame length of frame N is the first frame length, and the output mode of frame N is the first output mode, that is, mode A.

[0206] At time T5-1 (it can be at time T6-1, that is, any time before the effective point of frame N+2, which is not limited in this application), if Fig.11b As shown, the CRM sends N frames of effective configuration information to the Sensor driver, and the Sensor driver writes the N frames of effective configuration information into the effective register of the Sensor.

[0207] Exemplarily, at time T5-1 (which may be time T6-1, i.e., any time before the effective point of frame N+2, not limited in this application), CRM sends IFE configuration information of frame N to the IFE driver, and the IFE driver writes the IFE configuration information of frame N into the register of IFE.

[0208] Optionally, the time when the CRM sends N frames of IFE configuration information to the IFE driver and the time when the CRM sends N frames of effective configuration information to the Sensor driver may be different, and this application does not limit this.

[0209] At time T6-1, the Sensor determines that this time point is the effective point of the N+2 frame based on the effective point configuration information (for example, the effective point is X time length away from the SOF of the N+1 frame), which is the exposure start point of the N+2 frame in this example. As mentioned above, before this moment, the Sensor has written the effective configuration information. Accordingly, at this effective point, the Sensor can output images based on the configuration information of the N frames that have been written (including the second output mode and the first frame length, etc.), that is, expose and output the N+2 frame. In other words, the output mode of the N+2 frame is the second output mode, that is, mode B.

[0210] Exemplarily, the Sensor outputs the N+2 frame to the IFE. As described above, the effective timing of the IFE is the N+1 frame. Therefore, the IFE writes the IFE configuration information at the N+1 frame, and the IFE configuration information will take effect at the N+2 frame. Accordingly, the IFE processes the N+1 frame based on the IFE configuration information of the N frame. For example, the IFE performs image processing on the N+1 frame based on the preprocessing parameters corresponding to the second output mode.

[0211] IFE outputs the processed image to the camera application, and the camera application displays frame N+2. Due to the switching of the image output mode, the display effect of frame N+2 is different from that of frame N+1, frame N, and other image frames.

[0212] For example, as described above, the effective point of the camera parameters and the effective point of the output configuration may be different. Figure 7 , Fig. 9 and Fig.10 , which explains the scenario in detail. Figure 7 , Fig.12 For an exemplary image frame exposure timing diagram, please refer to Fig.12 At time T1-1 (which can be any time before N frames, not limited in this application), the Sensor node sends the Sensor configuration information of N frames to the CRM (including the image output mode configuration information and camera parameters corresponding to the second image output mode, etc.), and the IFE node sends the IFE configuration information of N frames to the CRM. The CRM obtains the Sensor configuration information and IFE configuration information corresponding to N frames (for specific data flow, please refer to Figure 6b , which will not be described in detail here). Optionally, as described above, the camera parameters and image output configuration information in the Sensor configuration information can be sent to the CRM at different times. The specific sending process can refer to the existing technical embodiments, which is not limited in this application.

[0213] In this scenario, the camera parameter effective point is different from the output configuration effective point. The camera parameter effective point is the exposure duration starting point (it can also be any time after the EOF of the N-1 frame and before the exposure starting point, which is not limited in this application), that is, at time T2-1, the Sensor exposes the N-1 frame based on the camera parameters (such as frame length, etc.).

[0214] At time T2-2 (which can be any time after the EOF of frame N-1 and before the SOF of frame N, not limited in this application), the Sensor detects whether the Sensor output configuration information is received at this time based on the effective point information. Among them, the output mode configuration information in the current configuration information of the Sensor indicates the first output mode, that is, mode A. Correspondingly, at time T3, the Sensor outputs N frames according to the first output mode.

[0215] For the configuration information of frame N, it is written in frame N and takes effect in frame N+1. Specifically, at time T3-1 within frame N (generally any time between SOR and EOF, which can be set according to actual needs and is not limited in this application), CRM sends the Sensor configuration information corresponding to frame N (including camera parameters and image output mode configuration information corresponding to the second image output mode) to the Sensor driver, and CRM sends the IFE configuration information corresponding to frame N (including image output mode configuration information (the image output mode configuration information indicates the second image output mode)) to the IFE driver.

[0216] At time T3-1, the Sensor driver and the IFE driver write corresponding configuration information to the Sensor and the IFE, respectively. It should be noted that there may be a delay between the above two writing actions, which is not limited in this application.

[0217] In an embodiment of the present application, as described above, the output mode configuration information of the Sensor may further include switching configuration information and effective configuration information. Exemplarily, the Sensor driver writes the switching configuration information to the register corresponding to the Sensor (recorded as the output mode register). After the Sensor driver writes all the switching configuration information to the register, the Sensor writes the effective configuration information to the corresponding register (recorded as the effective register). The writing method of other configuration information (such as camera parameters) can refer to the existing technology, and this application is not limited.

[0218] Exemplarily, the IFE driver writes N frames of IFE configuration information into a register corresponding to the IFE.

[0219] Optionally, the number of registers written by the IFE is less than the number of registers that the Sensor needs to write. Usually, the configuration information of the Sensor is long and the number of registers that need to be written is large.

[0220] At time T4-1, the Sensor exposes N frames based on camera parameters (such as frame length, etc.). At time T4-2 (which can be any time after the EOF of frame N and before the SOF of frame N+1, not limited in this application), the Sensor detects whether the Sensor output configuration information is received at this time based on the effective point information. In this example, the Sensor has obtained the Sensor output configuration information, that is, the Sensor output configuration information of frame N is effective at this point. Correspondingly, at time T5, the Sensor outputs N frames according to the second output mode.

[0221] For other parts not described, please refer to Figure 7 and Figure 8 The relevant description will not be repeated here.

[0222] Combination Fig. 9 , Fig.13 For an exemplary image frame exposure timing diagram, please refer to Fig.13 ,

[0223] At time T1-1 (which can be any time after receiving the image output mode information from the decision module and before T2-1, which is not limited in this application), the Sensor node responds to the instruction of the decision module and determines that the image output mode needs to be switched in frame N. The Sensor node generates configuration information for frame N-1. The configuration information of frame N-1 includes but is not limited to camera parameters, wherein the frame length in the camera parameters is the second frame length. The current default frame length is the first frame length, for example, 33ms, which can be set according to actual needs, which is not limited in this application. , the Sensor node sends the configuration information of frame N-1 to CRM. CRM obtains the Sensor configuration information of frame N-1 (including the second frame length).

[0224] At time T2-1, which is the effective point of the camera parameters, the Sensor exposes frame N-1 based on the camera parameters of frame N-2. At time T2-2, which is the effective point of the image output configuration, the Sensor detects whether it has received the image output configuration information, and the Sensor still uses the old image output configuration information. Correspondingly, at time T3, the Sensor outputs frame N-1 based on the first image output mode.

[0225] At time T3-1 (which can be after the Sensor sends the Sensor configuration information of frame N-1 and before time T5 (i.e., frame N), which is not limited in this application), the Sensor node generates configuration information for frame N. As described above, the Sensor node has obtained from the decision module that the image output mode of frame N will be switched to the second image output mode. Accordingly, the configuration information of frame N includes image output mode configuration information, which includes but is not limited to switching configuration information and effective configuration information. The switching configuration information is used to instruct the Sensor to switch to the second image output mode (i.e., mode B). Furthermore, it is expected that the frame length of frame N+1 will be restored to the first frame length, and accordingly, the frame length in the camera parameters in the configuration information of frame N is the first frame length.

[0226] The Sensor node sends N frames of Sensor configuration information to the CRM. For the specific sending process, please refer to Figure 6b , I will not go into details here.

[0227] Exemplarily, the IFE node also generates IFE configuration information of frame N. The IFE configuration information includes but is not limited to preprocessing parameters required for the second image output mode.

[0228] The IFE node sends N frames of IFE configuration information to the CRM.

[0229] The CRM obtains the Sensor configuration information and IFE configuration information of N frames.

[0230] At time T3-2 (which can be any time between the SOF of frame N-1 (i.e., time T3) and before the exposure of frame N (i.e., time T4-1), this application does not limit it), CRM sends the configuration information of frame N-1 (including the second frame length) to the Sensor driver. The Sensor driver writes the configuration information of frame N-1 into the register corresponding to the Sensor.

[0231] At time T4-1, which is the effective point of the camera parameters of frame N, the Sensor exposes frame N based on the camera parameters of frame N-1 (i.e., the second frame length). At time T4-2, which is the effective point of the image output configuration, the Sensor detects whether the image output configuration information has been received, and the Sensor still uses the old image output configuration information. Correspondingly, at time T5, the Sensor outputs frame N based on the first image output mode.

[0232] At time T5-2 (which can be any time before time T6-1), CRM sends the Sensor configuration information of N frames acquired at time T3-1 to the Sensor driver, and sends the IFE configuration information of N frames to the IFE driver. Among them, the Sensor configuration information includes but is not limited to camera parameters (where the frame length is the first frame length) and image output mode configuration information. The image output mode configuration information includes but is not limited to switching configuration information and effective configuration information. The switching configuration information indicates that the image output mode is the second image output mode (i.e., mode B).

[0233] After the Sensor driver writes all the switching configuration information into the corresponding register, the Sensor writes the effective configuration information into the effective register. In addition, the IFE driver writes the IFE configuration information of N frames into the corresponding register.

[0234] At time T7-1, which is the effective point of the camera parameters of frame N+1, the Sensor exposes frame N+1 based on the camera parameters of frame N (i.e., the first frame length). At time T7-2, which is the effective point of the image output configuration, the Sensor detects whether the image output configuration information is received. The Sensor has received the image output configuration information of frame N (i.e., the second image output mode). Correspondingly, at time T8, the Sensor outputs frame N+1 based on the second image output mode. The Sensor outputs the image frame to IFE. IFE also takes effect on frame N+1 of the IFE configuration information, that is, IFE processes the image captured by the Sensor based on the second image output mode based on the preprocessing parameters corresponding to the second image output mode. For other undescribed parts, please refer to Fig. 9 , I will not go into details here.

[0235] Combination Fig.10 , Fig.14 For an example of an image frame exposure timing diagram, please refer to Fig.14 At time T1-1 (which may be any time before time T3, not limited in this application), the decision module decides to switch the first image output mode to the second image output mode based on the scene environment information. Figure 6b , the decision module indicates the second output mode to the IFE node and the Sensor node to trigger the IFE node and the Sensor node to generate corresponding configuration information.

[0236] The Sensor node generates N frames of Sensor configuration information in response to the instruction of the decision module. The Sensor configuration information includes but is not limited to camera parameters and image output mode configuration information. The camera parameters include but are not limited to the frame length of N frames (for example, the first frame length). The Sensor image output mode configuration information includes but is not limited to switching configuration information and effective configuration information. The switching configuration information includes configuration parameters corresponding to the second image output mode.

[0237] The IFE node generates N frames of IFE configuration information in response to the instruction of the decision module, wherein the IFE configuration information includes but is not limited to preprocessing parameters corresponding to the second image output mode, so that the IFE can preprocess the preview stream collected by the target camera according to the target image output mode (i.e., the second image output mode).

[0238] The Sensor node sends N frames of Sensor configuration information to the CRM, and the IFE node sends N frames of IFE configuration information to the CRM. The CRM obtains N frames of Sensor configuration information and IFE configuration information. For specific data transmission flow, refer to Figure 6b , I will not go into details here.

[0239] Exemplarily, as described above, based on the timing of the configuration information of N frames taking effect in the Sensor at the N+2 frame, and the timing of the configuration information of N frames taking effect in the IFE at the N+1 frame, the CRM controls the timing of the switching configuration information and the effective configuration information of the Sensor, as well as the sending of the IFE configuration information, so as to achieve data synchronization between the IFE configuration information and the Sensor configuration information in the N+2 frame.

[0240] At time T2-1, the sensor exposes frame N based on the camera parameters (such as frame length, etc.) of frame N-2. Optionally, the camera parameters can be effective at frame N+1, that is, the camera parameters of frame N are still effective at frame N+1. This application does not limit this.

[0241] At time T2-2 (which can be any time after the EOF of frame N and before the SOF of frame N, not limited in this application), the Sensor detects whether the Sensor output configuration information is received at this time based on the effective point information. Among them, the output mode configuration information in the current configuration information of the Sensor indicates the first output mode, that is, mode A. Correspondingly, at time T3, the Sensor outputs N frames according to the first output mode.

[0242] At time T3-1 (which can be any time after the SOF of frame N and before the effective point of frame N+1, not limited in this application), the Sensor driver writes the Sensor configuration information of frame N into the image output mode register corresponding to the Sensor, so that the Sensor outputs images according to the specified image output mode (i.e., the second image output mode). Exemplarily, the Sensor configuration information includes but is not limited to camera parameters and switching configuration information. Camera parameters include but are not limited to the frame length of frame N (for example, the first frame length).

[0243] At time T4-1, the Sensor exposes frame N+1 based on the camera parameters (such as frame length, etc.) of frame N-1.

[0244] At time T4-2 (which can be any time after the EOF of frame N and before the SOF of frame N+1, not limited in this application), the Sensor detects whether the Sensor output configuration information is received at this time based on the effective point information. Among them, the output mode configuration information in the current configuration information of the Sensor indicates the first output mode, that is, mode A. Correspondingly, at time T5, the Sensor outputs the N+1 frame according to the first output mode.

[0245] At time T5-1 (it can be at time T6-1, that is, any time before the effective point of frame N+2, which is not limited in this application), if Fig.11b As shown, the CRM sends N frames of effective configuration information to the Sensor driver, and the Sensor driver writes the N frames of effective configuration information into the effective register of the Sensor.

[0246] At time T6-1, the Sensor exposes frame N+2 based on the camera parameters of frame N (such as frame length, etc.).

[0247] At time T6-2 (which can be any time after the EOF of frame N+1 and before the SOF of frame N+2, and this application does not limit this), the Sensor detects whether the Sensor output configuration information is received at this time based on the effective point information. At time T6-2, that is, the effective point of the image output configuration, the Sensor detects whether the image output configuration information is received, and the Sensor has received the image output configuration information of frame N (that is, the second image output mode). Correspondingly, at time T7, the Sensor outputs the N+2 frame based on the second image output mode. The Sensor outputs the image frame to IFE. IFE takes effect on the IFE configuration information of frame N at frame N+1, that is, IFE processes the image captured by the Sensor based on the second image output mode based on the preprocessing parameters corresponding to the second image output mode. For other undescribed parts, please refer to Fig.10 , I will not go into details here.

[0248] It is understandable that, in order to realize the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0249] In one example, Fig.15 A schematic block diagram of a device 1500 according to an embodiment of the present application is shown. The device 1500 may include: a processor 1501 and a transceiver / transceiver pin 1502 , and optionally, a memory 1503 .

[0250] The components of the device 1500 are coupled together via a bus 1504, wherein the bus 1504 includes a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, all buses are referred to as bus 1504 in the figure.

[0251] Optionally, the memory 1503 may be used for the instructions in the aforementioned method embodiment. The processor 1501 may be used to execute the instructions in the memory 1503, and control the receiving pin to receive a signal, and control the sending pin to send a signal.

[0252] The apparatus 1500 may be the electronic device or a chip of the electronic device in the above method embodiment.

[0253] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0254] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

[0255] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the method in the above-mentioned embodiment.

[0256] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the chip executes the methods in the above-mentioned method embodiments.

[0257] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0258] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for setting photographing parameters. It is characterized in that Applied to electronic equipment, the method comprises: At a first moment, obtaining first camera parameters of the N-1th frame, where the first camera parameters include a first frame length; At a second moment, obtaining second image output mode configuration information and second camera parameters of the Nth frame; wherein the second image output mode configuration information includes configuration parameters corresponding to the second image output mode, and the second camera parameters include a second frame length, and the second frame length is less than the first frame length; At a third moment, writing the first camera parameter into the camera sensor; The camera sensor exposes and outputs the Nth frame based on the first camera parameter; the frame length of the Nth frame is the first frame length, and the output mode of the Nth frame is the first output mode; At a fourth moment, writing the second image output mode configuration information and the second camera parameters into the camera sensor; The camera sensor exposes and outputs the N+1th frame based on the second camera parameters and the second image output mode configuration information; wherein, at the fourth moment in the Nth frame, the frame length of the N+1th frame is the second frame length, and the image output mode of the N+1th frame is the second image output mode.

2. The method according to claim 1, It is characterized in that The duration between the effective point of the image output mode of the Nth frame and the start of frame delimiter SOF of the Nth frame is the first duration, and the duration between the effective point of the image output mode of the N+1th frame and the SOF of the N+1th frame is the first duration; The camera sensor exposes and outputs the N+1th frame based on the second camera parameter and the second image output mode configuration information, including: The camera sensor obtains the second image output mode configuration information at the image output mode effective point of the N+1th frame, The camera sensor outputs the N+1th frame at the SOF of the N+1th frame.

3. The method according to claim 1, It is characterized in that Before writing the first camera parameter into the camera sensor, the method further includes: At a fifth moment, obtaining IFE configuration information of the Nth frame, wherein the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; The step of writing the second image output mode configuration information and the second camera parameter into the camera sensor further includes: The IFE configuration information is written into the IFE.

4. The method according to claim 3, It is characterized in that The method further comprises: The IFE performs image processing on the N+1th frame input by the camera sensor based on the IFE configuration information.

5. The method according to any one of claims 1 to 4, It is characterized in that The first frame length is n times the second frame length, where n is greater than or equal to 1 and less than 2.

6. A method for setting photographing parameters, It is characterized in that Applied to electronic equipment, the method comprises: At a first moment, obtaining second image output mode configuration information of the Nth frame and IFE configuration information of the Nth frame, wherein the second image output mode configuration information includes effective configuration information and configuration parameters corresponding to the second image output mode, and the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; At a second moment, the Nth frame is output; wherein the output mode of the Nth frame is the first output mode; At a third moment, writing the configuration parameters corresponding to the second image output mode into the camera sensor; wherein the third moment is within the Nth frame; At the fourth moment, the N+1th frame is output; wherein the output mode of the N+1th frame is the first output mode; At a fifth moment in the N+1th frame, writing the effective configuration information into the camera sensor, and writing the IFE configuration information into the IFE; The camera sensor outputs an image for the N+2th frame based on the second image output mode configuration information, and the image output mode of the N+2th frame is the second image output mode.

7. The method according to claim 6, It is characterized in that The duration between the effective point of the image output mode of the N+1th frame and the start of frame delimiter SOF of the N+1th frame is the first duration, and the duration between the effective point of the image output mode of the N+2th frame and the SOF of the N+2th frame is the first duration; The camera sensor outputs an image for the N+2th frame based on the second image output mode configuration information, including: The camera sensor obtains the second image output mode configuration information at the image output mode effective point of the N+2th frame, The camera sensor outputs the N+2th frame at the SOF of the N+2th frame.

8. The method according to claim 7, It is characterized in that The fifth moment is before the effective point of the image output mode of the N+2th frame.

9. The method according to claim 6, It is characterized in that The method further comprises: The IFE performs image processing on the N+2th frame input by the camera sensor based on the IFE configuration information.

10. An electronic device, It is characterized in that include: One or more processors, memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the following steps: At a first moment, obtaining first camera parameters of the N-1th frame, wherein the first camera parameters include a first frame length, At a second moment, obtaining second image output mode configuration information and second camera parameters of the Nth frame; wherein the second image output mode configuration information includes configuration parameters corresponding to the second image output mode, and the second camera parameters include a second frame length, and the second frame length is less than the first frame length; At a third moment, writing the first camera parameter into the camera sensor; The camera sensor exposes and outputs the Nth frame based on the first camera parameter; the frame length of the Nth frame is the first frame length, and the output mode of the Nth frame is the first output mode; At a fourth moment, writing the second image output mode configuration information and the second camera parameters into the camera sensor; The camera sensor exposes and outputs the N+1th frame based on the second camera parameters and the second image output mode configuration information; wherein, at the fourth moment in the Nth frame, the frame length of the N+1th frame is the second frame length, and the image output mode of the N+1th frame is the second image output mode.

11. The electronic device according to claim 10, It is characterized in that The duration between the effective point of the image output mode of the Nth frame and the start of frame delimiter SOF of the Nth frame is the first duration, and the duration between the effective point of the image output mode of the N+1th frame and the SOF of the N+1th frame is the first duration; when the computer program is executed by the one or more processors, the electronic device performs the following steps: The camera sensor obtains the second image output mode configuration information at the image output mode effective point of the N+1th frame, The camera sensor outputs the N+1th frame at the SOF of the N+1th frame.

12. The electronic device according to claim 10, It is characterized in that When the computer program is executed by the one or more processors, the electronic device performs the following steps: At a fifth moment, obtaining IFE configuration information of the Nth frame, wherein the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; When the computer program is executed by the one or more processors, the electronic device performs the following steps: The IFE configuration information is written into the IFE.

13. The electronic device according to claim 12, It is characterized in that When the computer program is executed by the one or more processors, the electronic device performs the following steps: The IFE performs image processing on the N+1th frame input by the camera sensor based on the IFE configuration information.

14. The electronic device according to any one of claims 10 to 13, It is characterized in that The first frame length is n times the second frame length, where n is greater than or equal to 1 and less than 2.

15. An electronic device, It is characterized in that include: One or more processors, memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the following steps: At a first moment, obtaining second image output mode configuration information of the Nth frame and IFE configuration information of the Nth frame, wherein the second image output mode configuration information includes effective configuration information and configuration parameters corresponding to the second image output mode, and the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; At a second moment, the Nth frame is output; wherein the output mode of the Nth frame is the first output mode; At a third moment, writing the configuration parameters corresponding to the second image output mode into the camera sensor; wherein the third moment is within the Nth frame; At the fourth moment, the N+1th frame is output; wherein the output mode of the N+1th frame is the first output mode; At a fifth moment in the N+1th frame, writing the effective configuration information into the camera sensor, and writing the IFE configuration information into the IFE; The camera sensor outputs an image for the N+2th frame based on the second image output mode configuration information, and the image output mode of the N+2th frame is the second image output mode.

16. The electronic device according to claim 15, It is characterized in that The duration between the effective point of the image output mode of the N+1th frame and the start of frame delimiter SOF of the N+1th frame is the first duration, and the duration between the effective point of the image output mode of the N+2th frame and the SOF of the N+2th frame is the first duration; when the computer program is executed by the one or more processors, the electronic device performs the following steps: The camera sensor obtains the second image output mode configuration information at the image output mode effective point of the N+2th frame, The camera sensor outputs the N+2th frame at the SOF of the N+2th frame.

17. The electronic device according to claim 16, It is characterized in that The fifth moment is before the effective point of the image output mode of the N+2th frame.

18. The electronic device according to claim 15, It is characterized in that When the computer program is executed by the one or more processors, the electronic device performs the following steps: The IFE performs image processing on the N+2th frame input by the camera sensor based on the IFE configuration information.

19. A computer storage medium, It is characterized in that The method comprises computer instructions, which, when executed on an electronic device, enable the electronic device to execute the method according to any one of claims 1 to 5 or the method according to any one of claims 6 to 9.

20. A computer program product, It is characterized in that When the computer program product is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 5 or the method according to any one of claims 6 to 9.

21. A chip, It is characterized in that The electronic device comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of the electronic device and send the signal to the processor, wherein the signal includes a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the method described in any one of claims 1 to 5 or executes the method described in any one of claims 6 to 9.

Images