Photographing parameter setting method, and electronic device

Abstract

Provided in the present application are a photographing parameter setting method, and an electronic device. The method comprises: an electronic device increasing the frame length of a frame N, such that there is enough time for writing configuration information of the frame N before outputting a frame N+1. Thus, a camera sensor can output the frame N+1 on the basis of the written configuration information of the frame N. In this way, the problem of frame loss caused by configuration information not being completely written is avoided.

Description

Photographing parameter setting method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on November 30, 2023, with application number 202311626683.7 and application name “Photographing parameter setting method and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal devices, and in particular to a method and device for setting photographing parameters. Background Art

[0003] With the development of terminal technology, the application scenarios of terminal devices are becoming more and more extensive. For example, users can use mobile phones to take photos to meet their daily needs. During the photo shooting process, the terminal can automatically switch between different image output modes according to the current environment to meet the current needs, thereby improving the image quality.

[0004] Summary of the Invention

[0005] The present application provides a method for setting photographing parameters and an electronic device. In this method, the electronic device can avoid the problem of frame loss caused by a long writing time of the output mode configuration information by delaying the effective timing of the output mode configuration information.

[0006] In a first aspect, the present application provides a method for setting photographic parameters. The method is applied to an electronic device and includes: at a first moment, obtaining first camera parameters for frame N-1, the first camera parameters including a first frame length. At a second moment, obtaining second image output mode configuration information and second camera parameters for frame N; the second image output mode configuration information includes configuration parameters corresponding to the second image output mode, the second camera parameters including a second frame length, and the second frame length being less than the first frame length. At a third moment, writing the first camera parameters to a camera sensor, which exposes and outputs frame N based on the first camera parameters; the frame length of frame N is the first frame length, and the image output mode of frame N is the first image output mode. At a fourth moment, writing the second image output mode configuration information and the second camera parameters to the camera sensor, which exposes and outputs frame N+1 based on the second camera parameters and the rth image output configuration information; at the fourth moment, within frame N, the frame length of frame N+1 is the second frame length, and the image output mode of frame N+1 is the second image output mode. In this way, the electronic device delays the effective time of the image output mode configuration information by extending the frame length of frame N, so that the electronic device has sufficient time to write the image output mode configuration information to the camera sensor. The effective time of the image output mode configuration information is the exposure time of frame N+1.

[0007] Exemplarily, the first moment may be time T1-1 in FIG. 9 , i.e., the sensor node generates sensor configuration information for frame N-1 and sends the sensor configuration information for frame N-1 to the CRM. The second moment may be time T3-1 in FIG. 9 , i.e., the sensor node generates sensor configuration information for frame N and sends the sensor configuration information for frame N to the CRM. The third moment may be time T3-2 in FIG. 9 , i.e., the CRM sends the sensor configuration information for frame N-1 to the sensor driver, and the sensor driver writes the configuration information for frame N-1 into the sensor register. The fourth moment may be time T5-2 in FIG. 9 , i.e., the CRM sends the sensor configuration information for frame N to the sensor driver, and the sensor driver writes the configuration information for frame N into the sensor register.

[0008] In one possible implementation, the duration between the effective point of the image output mode of the Nth frame and the start of frame delimiter (SOF) of the Nth frame is a first duration, and the duration between the effective point of the image output mode of the N+1th frame and the SOF of the N+1th frame is the first duration; the camera sensor exposes and outputs the N+1th frame based on the second camera parameters and the second image output mode configuration information, including: the camera sensor obtains the second image output mode configuration information at the effective point of the image output mode of the N+1th frame, and the camera sensor outputs the N+1th frame at the SOF of the N+1th frame. In this way, while the effective points of each frame remain unchanged in relative positions within each frame, the effective point of the N+1th frame can be delayed by extending the frame length of the Nth frame, thereby allowing the electronic device sufficient time to write the image output mode configuration information to the camera sensor.

[0009] Exemplarily, the first effective point may optionally be time T4-1 in FIG. 9 , that is, the sensor exposes the Nth frame based on the first frame length.

[0010] Exemplarily, the second effective point may optionally be time T7-1 in FIG. 9 , that is, the sensor exposes the N+1 frame based on the sensor configuration information of the N frame.

[0011] In one possible implementation, before writing the first camera parameters to the camera sensor, the method further includes: at a fifth moment, obtaining IFE configuration information for the Nth frame, the IFE configuration information including preprocessing parameters corresponding to the second image output mode; writing the second image output mode configuration information and the second camera parameters to the camera sensor; and further including: writing the IFE configuration information to the IFE. In this manner, the IFE synchronously writes the IFE configuration information to take effect, thereby achieving synchronization with the sensor configuration information.

[0012] Exemplarily, the fifth moment may optionally be the moment T3-1 in FIG. 9 .

[0013] In one possible implementation, the method further includes: the IFE performing image processing on the N+1th frame input by the camera sensor based on the IFE configuration information. Thus, by delaying the frame length of frame N, the effective time of the image output mode configuration information is postponed, allowing the electronic device sufficient time to write the image output mode configuration information to the camera sensor. The sensor can obtain the complete sensor configuration information for frame N before exposing frame N+1. Based on the sensor configuration information for frame N, frame N+1 is exposed. The IFE also writes the IFE configuration information into frame N, and it takes effect on frame N+1. This ensures that the IFE configuration information and the sensor configuration information are synchronized and effective, avoiding frame loss caused by configuration information asynchrony.

[0014] In a possible implementation, the first frame length is n times the second frame length, where n is greater than or equal to 1 and less than 2.

[0015] In a second aspect, the present application provides a method for setting photographing parameters, which is applied to an electronic device, and the method includes: at a first moment, obtaining the second image output mode configuration information of the Nth frame and the IFE configuration information of the Nth frame, wherein the second image output mode configuration information includes the effective configuration information and the configuration parameters corresponding to the second image output mode, and the IFE configuration information includes the preprocessing parameters corresponding to the second image output mode; at a second moment, outputting the Nth frame; wherein the image output mode of the Nth frame is the first image output mode; at a third moment, writing the configuration parameters corresponding to the second image output mode into the camera sensor; wherein the third moment is within the Nth frame; at a fourth moment, outputting the N+1th frame; wherein the image output mode of the N+1th frame is the first image output mode; at a fifth moment within the N+1th frame, writing the effective configuration information into the camera sensor, and writing the IFE configuration information into the IFE, the camera sensor is based on the second image output mode configuration information

[0016] The electronic device outputs the image mode configuration information for frame N+2 in the second output mode. By delaying the effective timing of the configuration information for frame N, the electronic device makes the effective timing of the configuration information for frame N take effect at frame N+2. This allows the electronic device to have sufficient time to write the image output mode configuration information before exposing frame N+2, thereby avoiding the problem of frame loss.

[0017] For example, the first moment may be time T1-1 in FIG10 . The second moment may be time T2-1 in FIG10 . The third moment may be time T3-1 in FIG10 . The fourth moment may be time T4-1 in FIG10 . The fifth moment may be time T5-1 in FIG10 .

[0018] In one possible implementation, the duration between the effective point of the image output mode of the N+1th frame and the start of frame delimiter (SOF) of the N+1th frame is a first duration, and the duration between the effective point of the image output mode of the N+2th frame and the SOF of the N+2th frame is the first duration; the camera sensor outputs the N+2th frame based on the second image output mode configuration information, including: the camera sensor obtains the second image output mode configuration information at the effective point of the image output mode of the N+2th frame, and the camera sensor outputs the N+2th frame at the SOF of the N+2th frame. In this way, the present application sets the effective timing of the sensor configuration of the electronic device to take effect at the N+2th frame, so that the electronic device has enough time to write the image output mode configuration information to the camera sensor, thereby avoiding the frame loss problem caused by the IFE configuration information and the sensor configuration information being out of sync.

[0019] In a possible implementation, the fifth moment is before the effective point of the image output mode of the N+2th frame.

[0020] In a possible implementation, the IFE performs image processing on the N+2th frame input by the camera sensor based on the IFE configuration information.

[0021] In a third aspect, the present application provides an electronic device, characterized in that it includes: one or more processors, a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the following steps: at a first moment, obtaining first camera parameters of the N-1th frame, the first camera parameters including a first frame length; at a second moment, obtaining second image output mode configuration information and second camera parameters of the Nth frame; wherein the second image output mode configuration information includes configuration parameters corresponding to the second image output mode, the second camera parameters including a second frame length, and the second frame length is less than the first frame length; at a third moment, writing the first camera parameters to a camera sensor, and the camera sensor exposes the Nth frame based on the first camera parameters; the frame length of the Nth frame is the first frame length, and the image output mode of the Nth frame is the first image output mode; at a fourth moment, writing the second image output mode configuration information and the second camera parameters to the camera sensor, and the camera sensor exposes the N+1th frame based on the second camera parameters and the second image output mode configuration information; wherein, at the fourth moment in the Nth frame, the frame length of the N+1th frame is the second frame length, and the image output mode of the N+1th frame is the second image output mode.

[0022] In one possible implementation, when the computer program is executed by one or more processors, the electronic device performs the following steps: the camera sensor exposes the Nth frame based on the first camera parameters at the first effective point; the camera sensor exposes the N+1th frame based on the second camera parameters and the second output mode configuration information at the second effective point; the duration between the second effective point and the start of frame delimiter SOF of the N+1th frame is equal to the duration between the first effective point and the SOF of the Nth frame.

[0023] In one possible implementation, when the computer program is executed by one or more processors, the electronic device performs the following steps: at the fifth moment, obtains the IFE configuration information of the Nth frame, the IFE configuration information includes preprocessing parameters corresponding to the second output mode; when the computer program is executed by the one or more processors, the electronic device performs the following steps: writes the IFE configuration information into the IFE.

[0024] In one possible implementation, when the computer program is executed by one or more processors, the electronic device executes the following steps: the IFE performs image processing on the N+1th frame input by the camera sensor based on the IFE configuration information.

[0025] In a possible implementation, the first frame length is n times the second frame length, where n is greater than or equal to 1 and less than 2.

[0026] In a fourth aspect, the present application provides an electronic device, characterized in that it includes: one or more processors, a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the following steps: at a first moment, obtaining the second image output mode configuration information of the Nth frame and the IFE configuration information of the Nth frame, wherein the second image output mode configuration information includes effective configuration information and configuration parameters corresponding to the second image output mode, and the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; at the ...; At the second moment, the Nth frame is exposed; wherein, the image output mode of the Nth frame is the first image output mode; at the third moment, the configuration parameters corresponding to the second image output mode are written into the camera sensor; wherein, the third moment is within the Nth frame; at the fourth moment, the N+1th frame is exposed; wherein, the image output mode of the N+1th frame is the first image output mode; at the fifth moment within the N+1th frame, the effective configuration information is written into the camera sensor, and the IFE configuration information is written into the IFE, and the camera sensor exposes the N+2th frame based on the second image output mode configuration information, and the image output mode of the N+2th frame is the second image output mode.

[0027] In one possible implementation, when the computer program is executed by one or more processors, the electronic device performs the following steps: the camera sensor exposes the N+1th frame at a first effective point; the camera sensor exposes the N+2th frame at a second effective point; the duration between the second effective point and the start of frame delimiter SOF of the N+1th frame is equal to the duration between the first effective point and the SOF of the N+2th frame.

[0028] In a possible implementation, the fifth moment is before the second effective point.

[0029] In a possible implementation, when the computer program is executed by one or more processors, the electronic device executes the following steps: the IFE performs image processing on the N+2th frame input by the camera sensor based on the IFE configuration information.

[0030] In a fifth aspect, the present application provides a computer-readable medium for storing a computer program, wherein the computer program includes instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0031] In a sixth aspect, the present application provides a computer-readable medium for storing a computer program, wherein the computer program includes instructions for executing the method in the second aspect or any possible implementation of the second aspect.

[0032] In a seventh aspect, the present application provides a computer program comprising instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0033] In an eighth aspect, the present application provides a computer program comprising instructions for executing the method in the second aspect or any possible implementation of the second aspect.

[0034] In a ninth aspect, the present application provides a chip comprising a processing circuit and a transceiver pin. The transceiver pin and the processing circuit communicate with each other via an internal connection path, and the processing circuit executes the method of the first aspect or any possible implementation of the first aspect to control the receive pin to receive a signal and to control the transmit pin to send a signal.

[0035] In a tenth aspect, the present application provides a chip comprising a processing circuit and a transceiver pin. The transceiver pin and the processing circuit communicate with each other via an internal connection path, and the processing circuit executes the method of the second aspect or any possible implementation of the second aspect to control the receive pin to receive a signal and to control the transmit pin to send a signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] FIG1 is a schematic diagram showing a hardware structure of an electronic device;

[0037] FIG2 is a schematic diagram of an exemplary camera module;

[0038] FIG3 is a schematic diagram illustrating a software structure of an electronic device;

[0039] FIG4 is a schematic diagram of an exemplary user interface;

[0040] FIG5 is a schematic diagram showing exemplary module interactions;

[0041] Figures 6a and 6b are schematic diagrams illustrating exemplary module interactions;

[0042] FIG7 is an exemplary image frame exposure timing diagram;

[0043] FIG8 is an exemplary image frame exposure timing diagram;

[0044] FIG9 is an exemplary image frame exposure timing diagram;

[0045] FIG10 is an exemplary timing diagram of image frame exposure;

[0046] FIG11a and FIG11b are schematic diagrams showing exemplary module interactions;

[0047] FIG12 is an exemplary image frame exposure timing diagram;

[0048] FIG13 is an exemplary image frame exposure timing diagram;

[0049] FIG14 is an exemplary image frame exposure timing diagram;

[0050] FIG15 is a schematic structural diagram of an exemplary device. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0052] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0053] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first target object" and "second target object" are used to distinguish different objects, rather than to describe a specific order of objects.

[0054] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0055] In the description of the embodiments of this application, unless otherwise specified, "multiple" means two or more. For example, "multiple processing units" means two or more processing units; "multiple systems" means two or more systems.

[0056] To facilitate understanding, the technical terms involved in the embodiments of the present application are first explained.

[0057] Dynamic range (DRR) is the ratio of the maximum brightness to the minimum brightness in a video or image signal. In many fields, dynamic range is used to express the ratio of the maximum to the minimum value of a variable. In digital images, dynamic range represents the ratio between the maximum grayscale value and the minimum grayscale value within the displayable range of the image. The dynamic range in nature is quite large. The brightness of a night scene under the starry sky is about 0.001cd / m2, and the brightness of the sun itself is as high as 1,000,000,000cd / m2. The dynamic range reaches 1,000,000,000 / 0.001=10 13 However, in real scenes in nature, the brightness of the sun and the brightness of the stars will not be obtained at the same time. For natural scenes in the real world, the dynamic range is

[0058] 10 -3 to 10 6 In most current color digital images, each R, G, and B channel is stored using an 8-bit byte. That is, the representation range of each channel is 0 to 255 grayscale levels. Here, 0 to 255 is the dynamic range of the image. Since the dynamic range of the same scene in the real world is in the range of 10-3 to 106, we call it High Dynamic Range (HDR). The dynamic range of ordinary pictures is relatively low dynamic range (LDR). The imaging process of a digital camera is actually a mapping from the high dynamic range of the real world to the low dynamic range of the photo. The mapping from the high dynamic range in the real world to the low dynamic range of the photo is often a nonlinear process.

[0059] Gain: The process of amplifying or reducing the electrical or digital signal after photoelectric conversion. Increasing the gain makes the captured image brighter than the actual scene; decreasing the gain makes the captured image darker than the actual scene.

[0060] Seamless switching: Seamless switching is a fast, uninterrupted switching process. During the camera's image output, the software automatically switches between different output modes based on the current scene. This switching process is called seamless switching. Functionally, there must be no delay during the switching process, otherwise it will cause lag for the user.

[0061] Image output mode: In an embodiment of the present application, the terminal can switch the image output mode based on the current environmental information. The specific switching method will be described in detail in the following embodiments.

[0062] Exemplarily, the image output mode includes but is not limited to Binning image output mode and HDR image output mode, wherein Binning mode is the default image output mode of the camera sensor.

[0063] HDR output modes include but are not limited to the following:

[0064] 1. SHDR (stagger HDR) mode: The camera sensor outputs two frames of images: a long-exposure frame focusing on dark areas, followed by a short-exposure frame focusing on bright areas. Both frames are simultaneously input into the chip platform's image processing module and processed by a specific algorithm to generate a single image.

[0065] 2. DCG (dual conversion gain) mode: Each pixel of the camera sensor can independently control the gain. When working in DCG mode, only one exposure is performed, but the data is read out in two steps: one using HCG (high conversion gain, also known as High CG) to capture dark information (also known as low-brightness information, that is, brightness less than or equal to the preset threshold), and the other using LCG (low conversion gain, also known as Low CG) to capture bright information (also known as high-brightness information, that is, brightness greater than or equal to the preset threshold). The HCG and LCG images are simultaneously input into the chip platform image processing module and processed by a specific algorithm to generate a single image frame.

[0066] 3. IDCG (intra dual conversion gain) mode: Unlike DCG mode, in DCG, the camera sensor simultaneously inputs two frames of HCG and LCG images into the chip platform, which then fuses them into a single image. In IDCG mode, the HCG and LCG images are fused inside the camera sensor into a single image, which is then input into the chip platform.

[0067] 4. IDCG combo + VS (very short) mode: VS is a special mode with a very short exposure time. IDCG and VS follow the SHDR framing method, first exposing a long-exposure frame (IDCG, a fusion of HCG and LCG), followed by a short-exposure frame (VS) to capture dark areas. Both frames are simultaneously input into the chip platform's image processing module, where they are processed by a specific algorithm to generate a single image.

[0068] In the embodiments of the present application, the terminal device may be a mobile terminal with a shooting function, such as a mobile phone, a tablet computer, a wearable device, an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), or other professional shooting equipment such as a digital camera, a SLR camera / micro-single camera, a sports camera, a gimbal camera, and a drone. The embodiments of the present application do not limit the specific type of the terminal device.

[0069] FIG1 shows a schematic diagram of the structure of an electronic device 100. It should be understood that the electronic device 100 shown in FIG1 is merely an example of an electronic device, and that the electronic device 100 may have more or fewer components than shown, may combine two or more components, or may have a different component configuration. The various components shown in FIG1 may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.

[0070] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0071] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0072] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0073] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0074] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0075] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the electronic device 100.

[0076] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0077] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0078] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0079] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0080] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0081] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0082] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0083] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0084] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0085] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0086] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0087] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0088] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0089] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0090] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0091] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0092] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0093] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0094] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0095] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0096] FIG2 is a schematic diagram of an exemplary camera module. Referring to FIG2 , FIG2 (a) and (b) schematically illustrate the front and back of the electronic device 100, respectively. The front of the electronic device 100 can be understood as the side facing the user when the user is using the electronic device 100, and the back of the electronic device 100 can be understood as the side facing away from the user when the user is using the electronic device 100.

[0097] The camera module 102 is used to capture still images or videos. The camera module 102 can be set on the front and / or back of the electronic device 100. When the camera module 102 is set on the front of the electronic device 100, the front camera 102-1 can be used to shoot the scene on the front side of the electronic device 100, such as for selfies, and in some embodiments it can be referred to as a front camera. When the camera module 102 is set on the back of the electronic device 100, the rear camera 102-2 can be used to shoot the scene on the back side of the electronic device 100, and in some embodiments it can be referred to as a rear camera. When shooting, the user can select the corresponding camera module according to the shooting requirements.

[0098] It should be noted that the embodiment of the present application does not limit the number of camera modules 102 provided, and can be one, two, four, or even more. For example, one or more camera modules 102 can be provided on the front of the electronic device 100, and / or one or more camera modules 102 can be provided on the back of the electronic device 100. When multiple camera modules 102 are provided, the multiple camera modules 102 can be completely identical or different, for example, the multiple camera modules 102 have different lens optical parameters, different lens installation positions, different lens shapes, etc. The embodiment of the present application also does not impose any restrictions on the relative positions of the multiple camera modules when they are provided.

[0099] The image sensor assembly is mainly used for imaging. Specifically, the image sensor assembly captures images and exposes image frames.

[0100] Exemplarily, the image sensor is further configured to output a start of frame delimiter (SOF) signal to identify the beginning of the current frame, and an end of frame delimiter (EOF) signal to identify the end of the current frame. Exemplarily, the frame time of each frame is the length from the SOF to the next SOF, including the effective frame length between the SOF and EOF and the non-exposure blank lines (vblank, also known as blank inactive line time, vertical blanking, or field blanking) in each frame. The specific corresponding relationship is illustrated in FIG7 .

[0101] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0102] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0103] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0104] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0105] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0106] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0107] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0108] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0109] FIG3 is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0110] The layered architecture of the electronic device 100 divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided from top to bottom into an application layer, a framework layer, a hardware abstraction layer (HAL), and a kernel layer. It will be understood that Figure 2 is only an example. That is, the layers divided in the electronic device are not limited to the layers shown in Figure 3. For example, between the application framework layer and the HAL layer, an Android runtime layer and a system library layer may also be included.

[0111] The application layer can include a series of application packages.

[0112] As shown in FIG3 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0113] The framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0114] As shown in Figure 3, the application framework layer may include a window manager, a content provider, a view system, a camera service, a resource manager, a notification manager, and the like.

[0115] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0116] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0117] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0118] The camera service may also be referred to as a camera service, etc., which is not limited in this application. The service is used to call a camera (including a front camera and / or a rear camera) in response to an application request.

[0119] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0120] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0121] The HAL layer is used to abstract the hardware and provide a virtual hardware platform for the operating system. The HAL layer may include but is not limited to: decision modules, CAMX architecture, etc.

[0122] The decision module may be a multi-camera decision module that can determine the camera in the camera group that actually performs image acquisition based on scene information (including information such as ambient brightness and ambient dynamic range). This camera can be referred to as the target camera. The target camera can be, for example, the front camera or rear camera of the electronic device.

[0123] In addition, the decision module can also determine the target image output mode that the target camera needs to enable. The image output mode can be described above and will not be repeated here. In addition, the method for selecting the target image output mode can be combined with the shooting scene, shooting parameters (zoom ratio), etc., and the implementation details can be referred to the description in the subsequent embodiments and will not be repeated here.

[0124] The CAMX architecture is a logical architecture layer in the HAL layer. The CAMX architecture includes sensor nodes and image front end (IFE) nodes (IFE node). Among them, the sensor nodes in the CAMX architecture correspond one-to-one to the camera sensors in the electronic device, and the sensor nodes can configure the camera sensors to enable the specified image output mode. For example, the sensor node can pass the camera parameters (also called configuration parameters) required to enable the target image output mode to the target camera, so that the target camera sensor uses the target image output mode to output images. The above-mentioned IFE node is used to configure the preprocessing parameters required for IFE operation, so that IFE can preprocess the preview stream collected by the target camera according to the target image output mode. For example, the preview stream and the image frames of the video stream are subjected to color correction, downsampling, de-mosaicing and other processing.

[0125] The CAMX architecture also includes a CAMX conversion interface (Camera Serial Interface Decoder, CSL). The CAMX CSL receives configuration parameters from sensor nodes and converts them into I / O control instructions recognizable by the kernel layer. Furthermore, the CAMX CSL instructs the CRM, through V4L2 in the kernel layer, to pass the I / O control instructions corresponding to the camera parameters to the corresponding camera driver. The camera driver then writes the I / O control instructions corresponding to the camera parameters into the camera sensor, causing the camera sensor to output images according to the specified output mode.

[0126] The CAMX CSL receives preprocessing parameters from the IFE node and converts them into I / O control instructions recognizable by the kernel layer. Then, through V4L2, it instructs the CRM to pass the I / O control instructions corresponding to these preprocessing parameters to the corresponding ISP driver. The ISP driver then writes the I / O control instructions corresponding to the preprocessing parameters into the IFE, enabling the IFE to preprocess image frames captured according to the target output mode.

[0127] The kernel layer contains at least the Linux video device driver (Video for Linux2, V4L2) and the camera driver.

[0128] The camera driver includes but is not limited to: a camera request manager (CRM), a sensor driver, an IFE driver, and the like.

[0129] The V4L2 can be called by the HAL layer. The CRM is used to manage the drivers corresponding to camera-related components in the kernel layer, such as the IFE driver and sensor driver. In some examples, the HAL can instruct the CRM to manage the drivers corresponding to camera-related components through V4L2.

[0130] The camera driver can be used to drive a hardware module with a camera function, such as a camera sensor. In other words, the camera driver is responsible for exchanging data with the camera sensor. Of course, the kernel layer may also include driver software such as an audio driver and a sensor driver, and this embodiment of the application does not impose any restrictions on this.

[0131] The hardware includes a camera group (described above and not repeated here) and a display screen, etc. The camera group includes but is not limited to physical components such as IFE, sensors, and lenses.

[0132] It is understood that the layers in the software structure shown in FIG3 and the components contained in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown, and each layer may include more or fewer components, and this application does not limit this.

[0133] FIG4 is a schematic diagram of an exemplary user interface. Referring to FIG4 , a user clicks on the camera application. In response to the received user operation, the camera application displays an image preview interface 201 on display interface 200. Image preview interface 201 displays a preview image captured by the camera. The present embodiment of the present application only uses the processing of preview images as an example for illustration. In other embodiments, the solutions in the present embodiment of the present application can also be applied to other shooting scenarios, such as video recording, and this application does not limit this.

[0134] Figure 5 is an exemplary diagram illustrating module interaction. Specifically, after a user clicks on the camera application, the camera application sends a trigger instruction to the camera service to instruct it to call the camera service. Exemplarily, the trigger instruction includes, for example, shooting mode information. In this embodiment of the present application, after the camera application is started, it enters the photo mode by default, as shown in Figure 4. Accordingly, the trigger instruction includes, but is not limited to, photo mode information.

[0135] In response to the trigger instruction, the camera service sends a call instruction to the decision module for calling the camera, wherein the call instruction includes but is not limited to shooting mode information, such as photo mode.

[0136] The decision module can determine the camera in the camera group that actually performs image acquisition based on scene information (including information such as ambient brightness and ambient dynamic range). The target camera can be the front camera or rear camera in the electronic device.

[0137] The decision module can also determine the target output mode that the target camera needs to enable. In the embodiment of the present application, the decision module can first select the default output mode, that is, the binning mode. During the subsequent shooting process, the decision module can select the corresponding output mode based on each image frame captured by the camera and other parameters, such as the shooting scene, shooting parameters (zoom ratio), etc., to achieve output mode switching.

[0138] Still referring to Figure 5, after the decision module decides on the target camera, it can send information such as the target output mode to the IFE node and Sensor node corresponding to the target camera in the CAMX architecture to instruct the Sensor node and IFE node to generate corresponding output mode configuration information based on the target output mode.

[0139] Exemplarily, the Sensor node configures the camera sensor to enable the specified output mode in response to the received target output mode. Specifically, the Sensor node generates Sensor configuration information. Among them, the Sensor configuration information includes but is not limited to: the output mode configuration information corresponding to the target output mode (also referred to as Sensor output configuration information), which can also be understood as the output mode configuration information required to enable the target output mode. Optionally, the Sensor configuration information also includes other camera parameters (also referred to as configuration parameters or exposure parameters), such as frame length, etc., which are not limited in this application. In some examples, the camera parameters and the Sensor output configuration information can be issued at the same time or at different times. Among them, the camera parameters are used for the Sensor to expose the image frame, and accordingly, the camera parameters need to be issued before the image frame is exposed. The output mode configuration parameters are used to output the image frame, and accordingly, the output configuration parameters need to be issued before the image is output, which is not limited in this application.

[0140] Still referring to FIG. 5 , illustratively, the IFE generates IFE configuration information (also referred to as IFE output configuration information) in response to instructions from the decision module. The IFE configuration information includes, but is not limited to, preprocessing parameters required for IFE operation, enabling the IFE to preprocess the preview stream captured by the target camera according to the target output mode. For example, color correction, downsampling, and demosaicing may be performed on the preview stream and image frames of the video stream.

[0141] In the embodiments of the present application, the IFE and Sensor configuration information must correspond. For example, when the Sensor operates in binning output mode, the preprocessing parameters required for IFE operation also correspond to binning output mode. If the Sensor operates in another mode, such as HDR mode, and the preprocessing parameters required for IFE operation are those for binning output mode, the IFE cannot process the image input by the Sensor and will discard the image frame. A specific example is described below (Figure 8).

[0142] Continuing with Figure 5, for example, the IFE node sends IFE configuration information to the conversion interface. The sensor node sends sensor configuration information to the conversion interface. The conversion interface, through V4L2 in the kernel layer, instructs the CRM to pass the I / O control instructions corresponding to the camera parameters to the corresponding camera driver. The camera driver writes the I / O control instructions corresponding to the camera parameters to the camera sensor, causing the camera sensor to output images according to the specified output mode.

[0143] Specifically, V4L2 sends the IFE configuration information and the Sensor configuration information to the CRM. The CRM sends the IFE configuration information to the IFE driver at an appropriate timing (the specific timing will be described below in conjunction with the accompanying drawings), and sends the Sensor configuration information to the sensor driver. After the IFE driver receives the IFE configuration information, it writes the IFE configuration information to the IFE. After the sensor driver receives the Sensor configuration information, it writes the Sensor configuration information to the Sensor. The Sensor will capture images according to the Sensor configuration information. For example, the Sensor will capture images according to the camera parameters and output mode configuration information in the Sensor configuration information. The Sensor outputs the captured image to the IFE. The IFE can process the image based on the IFE configuration information. For example, the IFE can pre-process the image based on the pre-processing parameters corresponding to the target output mode.

[0144] Figures 6a and 6b are schematic diagrams of module interactions shown as examples. Please refer to Figure 6a. For example, during the photo-taking process, the decision module will make a decision for each frame. If it is determined that the output mode remains unchanged, there is no need to send information such as the output mode. In this example, the Sensor node will send the corresponding Sensor configuration information to the CRM in advance (for example, one frame in advance, or multiple frames in advance, which is not limited in this application) for each frame (the specific data transmission process can be referred to the description in Figure 4, which will not be repeated here). Among them, since the decision module has not switched the output mode, the Sensor configuration information includes camera parameters such as frame length, but does not include Sensor output mode configuration information. In other words, when the Sensor exposes the image frame corresponding to the Sensor configuration information, it is exposed according to the current (i.e., the most recent) output mode.

[0145] For example, after the CRM obtains the sensor configuration information, it can send the sensor configuration information to the sensor driver at an appropriate time, so that the sensor driver writes the sensor configuration information into the register corresponding to the sensor. The specific implementation method will be described in Figure 9.

[0146] Please refer to Figure 6b. If the decision module determines to switch the output mode, the decision module sends the output mode to the IFE node and the Sensor node to indicate the switching of the target output mode. In this example, the Sensor node generates Sensor configuration information. Among them, the Sensor configuration information includes but is not limited to: the output mode configuration information corresponding to the target output mode, which can also be understood as the output mode configuration information required to enable the target output mode. Optionally, the Sensor configuration information also includes other camera parameters (also referred to as configuration parameters), such as frame length, etc., which are not limited in this application.

[0147] The IFE generates IFE configuration information in response to the decision module's instructions. This information includes, but is not limited to, preprocessing parameters required for IFE operation, enabling the IFE to preprocess the preview stream captured by the target camera according to the target image output mode. For example, this includes color correction, downsampling, and de-mosaicing of image frames in the preview stream and video stream.

[0148] FIG7 is an exemplary image frame exposure timing diagram. Please refer to FIG7. Before frame N-1 (including frame N-1), the Sensor uses mode A (also referred to as the first output mode) to output each image frame. In this example, the configurations of the Sensor and IFE (including frame length, output mode, etc.) are configured to take effect in the new configuration at frame N+1. That is, for the configuration information of frame N, the Sensor node and the IFE node send the Sensor configuration information and IFE configuration information of N frames to the CRM before frame N (it can be at least one frame in advance, which is not limited in this application). At frame N, the CRM sends the Sensor configuration information and IFE configuration information of N frames to the Sensor driver and IFE driver, so that the Sensor driver and IFE driver write the Sensor configuration information and IFE configuration information to the Sensor and IFE respectively within frame N. The Sensor and IFE are configured to take effect on frame N of the configuration information at frame N+1. Accordingly, the Sensor and IFE will take effect on frame N of the sensor and IFE at frame N+1.

[0149] For example, still referring to Figure 7, at frame N-1, the Sensor uses mode A (also called the first output mode) for image exposure. Before moment T1-1, the decision module decides the target output mode and the target camera based on environmental information, wherein the target output mode is the second output mode (for example, mode B), that is, the current target output mode (i.e., the first output mode) is switched to the second output mode (mode B). Optionally, the decision module can make a decision at any time before frame N (i.e., moment T3), for example, it can be at frame N-1 or at frame N-2, and this application does not limit it. The time for sending each configuration information and the time for writing, etc. in the embodiments of this application are only illustrative examples and can be set according to actual needs, and this application does not limit it.

[0150] For example, in combination with Figure 6b, after the decision module decides on the target camera, it can send information such as the target output mode to the IFE node and Sensor node corresponding to the target camera in the CAMX architecture to instruct the Sensor node and IFE node to generate corresponding output mode configuration information based on the target output mode.

[0151] Exemplarily, the Sensor node configures the camera sensor to enable the specified output mode in response to the received target output mode. Specifically, the Sensor node generates N frames of Sensor configuration information. The Sensor configuration information includes but is not limited to: the output mode configuration information corresponding to the target output mode, which can also be understood as the output mode configuration parameters required to enable the target output mode. Exemplarily, the output mode configuration information includes but is not limited to switching configuration information and effective configuration information. The switching configuration information includes the output mode configuration parameters required for the target output mode, and the effective configuration information is used to indicate that the switching configuration information has been completely written.

[0152] Optionally, the sensor configuration information also includes other camera parameters (also referred to as configuration parameters), such as frame length, etc., which are not limited in this application.

[0153] Still referring to Figure 5 , illustratively, the IFE node generates IFE configuration information in response to instructions from the decision module. This IFE configuration information includes, but is not limited to, preprocessing parameters required for IFE operation, enabling the IFE to preprocess the preview stream captured by the target camera according to the target image output mode. For example, this includes color correction, downsampling, and de-mosaicing of image frames in the preview stream and video stream.

[0154] Still referring to Figure 7, at time T1-1 (which can be any time before frame N, not limited in this application), the Sensor node sends the Sensor configuration information of N frames to the CRM (including the output mode configuration information and camera parameters corresponding to the second output mode, etc.), and the IFE node sends the IFE configuration information of N frames to the CRM. The CRM obtains the Sensor configuration information and IFE configuration information corresponding to the N frames (the specific data flow can be referred to Figure 6b, which will not be repeated here). Optionally, as described above, the camera parameters and output configuration information in the Sensor configuration information can be sent to the CRM at different times. The specific sending process can refer to the existing technical embodiments, which is not limited in this application.

[0155] In an embodiment of the present application, the sensor is pre-configured with effective point information. Based on the effective point information, the sensor can obtain the sensor configuration information at a specified time in each frame (this time is called the effective point) and start image exposure based on the sensor configuration information. The effective point is the starting point of the exposure duration of the image frame, and the EOF of the image frame is the end point of the exposure duration. Optionally, in some examples, the effective point of the image output configuration information may be different from the effective point of the camera parameters. Specifically, as described above, the camera parameters are mainly used when the sensor exposes the image frame. Therefore, the camera parameters are effective at the exposure start point, and the sensor can expose the image frame based on the camera parameters. The sensor image output configuration information is mainly used by the sensor to output the image frame (for example, the time T3 to T5 in Figure 7, that is, within the frame length of N frames). Among them, the image output duration corresponds to the image frame length, for example, the time T3 to T5 in Figure 7, that is, the frame length of N frames. The sensor outputs N frames within this image output duration. Optionally, during the image output duration, the sensor actually performs the image output action (i.e., outputs the exposed image to the IFE) between the SOF and EOF of frame N. At other times within frame N (e.g., T4 to T5), the sensor performs other configurations. Accordingly, the sensor's image output configuration information takes effect before the image frame is output. That is, the sensor's image output configuration information only takes effect before the image frame is output. Specific examples are illustrated in Figures 12 to 14.

[0156] Optionally, the effective point information is used to indicate the relative duration between the effective point and the SOF. For example, the effective point information indicates that the relative duration is X. Accordingly, the sensor starts exposing the image frame X time before the SOF of each frame.

[0157] For example, refer to Figure 7. At time T2-1, the sensor determines the time corresponding to the effective point of frame N based on the effective point information. In this example, this is the exposure start point of frame N. The sensor obtains the written sensor configuration information and begins exposing frame N based on the camera parameters in the sensor configuration information. The interval between time T2-1 and the end of frame N (i.e., time T3) is X time.

[0158] For example, at time T3, the sensor outputs N frames based on the sensor output configuration information. The actual output duration is from time T3 to time T4, that is, from the SOF to the EOF of N frames. In this embodiment of the application, the duration of N frames (i.e., time T3 to time T4) is referred to as the output duration of N frames.

[0159] Exemplarily, the sensor obtains the already written sensor configuration information at the effective point. The output mode configuration information in the current sensor configuration information indicates the first output mode, namely mode A. Accordingly, the sensor exposes and outputs N frames according to the camera parameters in the configuration information and the first output mode.

[0160] Please continue to refer to Figure 7. As mentioned above, for the configuration information of frame N, it is written in frame N and takes effect in frame N+1. Specifically, at the T3-1 moment in frame N (generally any moment between SOR and EOF, which can be set according to actual needs and is not limited in this application), the CRM sends the Sensor configuration information corresponding to frame N (including camera parameters and the output mode configuration information corresponding to the second output mode) to the Sensor driver, and the CRM sends the IFE configuration information corresponding to frame N (including the output mode configuration information (the output mode configuration information indicates the second output mode)) to the IFE driver.

[0161] At time T3-1, the sensor driver and the IFE driver write corresponding configuration information to the sensor and the IFE, respectively. It should be noted that there may be a delay between the two writing actions, which is not limited in this application.

[0162] In an embodiment of the present application, as described above, the output mode configuration information of the Sensor may further include switching configuration information and effective configuration information. Exemplarily, the Sensor driver writes the switching configuration information to the register corresponding to the Sensor (recorded as the output mode register). After the Sensor driver writes all the switching configuration information to the register, the Sensor writes the effective configuration information to the corresponding register (recorded as the effective register). The writing method of other configuration information (such as camera parameters) can refer to the existing technology, and this application does not limit it.

[0163] Exemplarily, the IFE driver writes N frames of IFE configuration information into a register corresponding to the IFE.

[0164] Optionally, the number of registers written by the IFE is smaller than the number of registers that the sensor needs to write. Typically, the configuration information of the sensor is longer and the number of registers that need to be written is larger.

[0165] At time T4-1, the sensor determines the time corresponding to the effective point of frame N+1 based on the effective point information. In this example, this is the exposure time of frame N+1. The sensor obtains the already written sensor configuration information (that is, the configuration information of frame N written at T3-1) and begins exposing frame N+1 based on this sensor configuration information. The interval between time T4-1 and the end of the frame N (that is, time T3) is X time.

[0166] For example, at time T5, the sensor outputs the N+1 frame based on the sensor output configuration information. The actual output duration is from time T5 to T6, that is, from the SOF to the EOF of the N+1 frame. In this embodiment of the application, the duration of the N+1 frame (i.e., time T5 to T7) is referred to as the output duration of the N+1 frame.

[0167] For example, the Sensor obtains the configuration of the effective register at the effective point. In an example, if the Sensor has completely written the output mode configuration information of N frames before T4-1, that is, the Sensor has written all the switching configuration information into the corresponding output mode register, and written the effective configuration information into the corresponding effective register. Accordingly, the Sensor determines to start taking effect the configuration of the output mode register corresponding to the effective register based on the configuration of the effective register. In other words, the Sensor outputs the N+1 frame based on the output mode configuration information of the N frame (that is, modeB, which can also be called the second output mode).

[0168] In another example, if the Sensor fails to completely write the output mode configuration information of N frames before T4-1, that is, the Sensor fails to write the effective configuration information to the corresponding effective register. Accordingly, the Sensor determines at the effective point that the effective register has not been written completely, then the Sensor will not take into effect the configuration information being written, but will take into effect the old configuration information. For example, Figure 8 is an exemplary image frame exposure timing diagram, please refer to Figure 8, at the T3-1 moment in the N frame (generally any moment between SOR and EOF, which can be set according to actual needs, and is not limited in this application), the CRM sends the Sensor configuration information corresponding to the N frames (including camera parameters and output mode configuration information (the output mode configuration information indicates the second output mode)) to the Sensor driver, and sends the IFE configuration information corresponding to the N frames (including output mode configuration information (the output mode configuration information indicates the second output mode)) to the IFE driver.

[0169] At time T3-1, the sensor driver and the IFE driver write corresponding configuration information to the sensor and the IFE, respectively. It should be noted that there may be a delay between the two writing actions, which is not limited in this application.

[0170] Exemplarily, the sensor driver writes the switching configuration information to the corresponding register (referred to as the output mode register). After all the switching configuration information is written to the register, the sensor driver writes the effective configuration information to the corresponding register (referred to as the effective register). The writing method of other configuration information (such as camera parameters) can refer to the existing technology and is not limited in this application.

[0171] Exemplarily, the IFE driver writes N frames of IFE configuration information into a register corresponding to the IFE.

[0172] As shown in Figure 8, in this example, the sensor's switching configuration information is long, and many output mode registers need to be written. As a result, the sensor fails to fully write the switching configuration information before the effective point 1 (i.e., T4-1) of frame N+1. At effective point 1, since the sensor fails to detect the effective configuration information being written to the effective register, it can also be understood that the sensor has not received the effective configuration information of frame N. Accordingly, the sensor still outputs the image of frame N+1 according to the old output mode configuration information (i.e., modeA).

[0173] The Sersor outputs the exposed N+1 frame to the IFE. The IFE's current configuration information is the new image output mode configuration information, indicating the second image output mode. Accordingly, the IFE detects that the IFE's image output mode (i.e., the second image output mode, mode B) does not match the image output mode used by the sensor (i.e., the first image output mode, mode A). The IFE will discard the N+1 frame. Accordingly, the old image frame, i.e., frame N, will be displayed on the display. In other words, during the time that the N+1 frame should have been displayed, the N frame is displayed on the display. From the user's perspective, the user will perceive the image as stuck.

[0174] Assume that at time T5-1, after the sensor has completely written the switching configuration information, it writes the effective configuration information to the effective register. At time T6-1, the sensor obtains the configuration information at the effective point 2. The sensor detects that the effective configuration information has been written to the effective register and accordingly outputs the N+2 frame according to the new switching configuration information (i.e., the second output mode, Mode B).

[0175] The sensor outputs frame N+2 to the IFE. The IFE's current configuration information is the new image output mode configuration information, indicating the second image output mode. Accordingly, the IFE detects that the IFE's image output mode (i.e., the second image output mode, mode B) matches the image output mode used by the sensor (i.e., the second image output mode, mode B). The IFE can further process frame N+2 based on the preprocessing parameters (i.e., configuration information) corresponding to this image output mode.

[0176] This application proposes a method for setting photographing parameters, which avoids the frame loss problem by delaying the effective point of N-frame configuration information.

[0177] FIG9 is an exemplary image frame exposure timing diagram. Referring to FIG9 , before time T-1, the decision module decides to switch the current first image output mode (i.e., mode A) to the second image output mode (mode B). Alternatively, the decision module can determine the frame at which the image output mode is switched, or the sensor node and IFE node can determine the frame at which the image output mode is switched, which is not limited in this application.

[0178] 6b, the decision module outputs image output mode information to the Sensor node and IFE node, indicating that the first image output mode configuration information should be switched to the second image output mode. The Sensor and IFE are still configured to take effect on the Nth frame configuration information at the N+1th frame.

[0179] As shown in Figure 9, in this example, the purpose of the solution is to achieve the effect of delaying the effective point by lengthening the frame length of frame N, specifically lengthening the duration of vblank (the exposure duration remains unchanged). As mentioned above, the configuration of the Sensor takes effect at frame N+1. Therefore, if the frame length of frame N needs to be lengthened, that is, the new frame length parameter takes effect at frame N, then the Sensor needs to write the new frame length at frame N-1 to obtain the new frame length before the effective point of frame N (that is, time T4-1, which can also be understood as the exposure starting point), so that the Sensor can expose and output the N frames based on the new frame length.

[0180] For this purpose, at time T1-1 (which can be any time after receiving the output mode information from the decision module and before T2-1, and this application does not limit it), the Sensor node responds to the instruction of the decision module and determines that the output mode needs to be switched in frame N. The Sensor node generates configuration information for the N-1 frame. The configuration information of the N-1 frame includes but is not limited to camera parameters, wherein the frame length in the camera parameters is the second frame length. The current default frame length is the first frame length, for example, 33ms, which can be set according to actual needs and is not limited by this application.

[0181] Optionally, the second frame length is set to the first frame length * Ratio (ratio). The first frame length is the default frame length, which is usually set to 33ms. The sensor node can determine the value of Ratio based on the value of the product, device, and switching scenario configuration. In an embodiment of the present application, the value of Ratio is usually greater than 1 and less than 2. That is, as shown in Figure 9, the extended duration of the N frame (i.e., T7 to T8) is less than the original frame length of the N frame (i.e., the first frame length) (i.e., T5 to T7).

[0182] Alternatively, since the output mode of frame N-1 remains unchanged, as shown in FIG6a , the IFE node does not need to send the image mode configuration information in frame N-1. It should be noted that other configuration information except the frame length and image output mode configuration information is implemented according to existing technical processes and will not be described in detail in this application.

[0183] For example, as shown in Figure 6a, the sensor node sends the configuration information of frame N-1 to the CRM. The CRM obtains the sensor configuration information of frame N-1 (including the second frame length).

[0184] At time T2-1, it is the effective point of frame N-1. The Sensor still exposes frame N-1 based on the configuration information such as the first frame length. For example, at time T3, the Sensor outputs frame N-1 based on the Sensor output configuration information. The actual output duration is from time T3 to time T4, that is, from the SOF to the EOF of frame N. In the embodiment of the present application, the duration of frame N-1 (i.e., time T3 to time T4) is referred to as the output duration of frame N.

[0185] At time T3-1 (which can be after the Sensor sends the Sensor configuration information of frame N-1 and before time T5 (i.e., frame N), this application does not limit it), the Sensor node generates the configuration information of frame N. As mentioned above, the Sensor node has obtained from the decision module that the output mode of frame N will switch to the second output mode. Accordingly, the configuration information of frame N includes output mode configuration information, and the output mode configuration information includes but is not limited to switching configuration information and effective configuration information. The switching configuration information is used to instruct the Sensor to switch to the second output mode (i.e., mode B). In addition, it is expected that the frame length of frame N+1 will be restored to the first frame length, and accordingly, the frame length in the camera parameters in the configuration information of frame N is the first frame length.

[0186] The Sensor node sends N frames of Sensor configuration information to the CRM. The specific sending process can be found in Figure 6b and will not be repeated here.

[0187] Exemplarily, the IFE node also generates IFE configuration information of frame N. The IFE configuration information includes but is not limited to pre-processing parameters required for the second image output mode.

[0188] The IFE node sends N frames of IFE configuration information to the CRM.

[0189] The CRM obtains N frames of sensor configuration information and IFE configuration information.

[0190] It should be noted that there may be a certain delay between the generation and issuance of configuration information, which is not limited in this application. Optionally, there may also be a delay between the issuance of configuration information of the IFE node and the issuance of configuration information of the Sensor node, which is not limited in this application.

[0191] At time T3-2 (which can be any time between the SOF of frame N-1 (i.e., time T3) and before the exposure of frame N (i.e., time T4-1), this application does not limit this), the CRM sends the configuration information of frame N-1 (including the second frame length) to the sensor driver. The sensor driver writes the configuration information of frame N-1 to the corresponding register of the sensor.

[0192] At time T4-1, which is the effective point for frame N (in this example, the exposure start point for frame N), the sensor configuration for frame N-1 takes effect. That is, the sensor exposes frame N based on the already written configuration for frame N-1 (including the second frame length and first image output mode information). The frame length of frame N is the second frame length, which is times T5 to T8. The image output mode for frame N is the first image output mode, mode A.

[0193] At time T5-1 (which can be any time between the time when the Sensor node sends the N frame configuration information and the time before the N+1 frame, and is not limited in this application), the Sensor node generates the Sensor configuration information for the N+1 frame. Since the decision module did not instruct to switch the image output mode in the N+1 frame, the Sensor configuration information for the N+1 frame includes but is not limited to the camera parameters, but does not include the image output mode configuration information. The frame length in the camera parameters of the N+1 frame is the first frame length.

[0194] For example, the sensor node sends the sensor configuration information of frame N+1 to the CRM. The CRM obtains the sensor configuration information of frame N+1. The specific transmission process can be seen in Figure 6a and will not be repeated here.

[0195] At time T5-2 (which can be any time before time T6-1), the CRM sends the Sensor configuration information of N frames acquired at time T3-1 to the Sensor driver, and sends the IFE configuration information of N frames to the IFE driver. The Sensor configuration information includes but is not limited to camera parameters (where the frame length is the first frame length) and image output mode configuration information. The image output mode configuration information includes but is not limited to switching configuration information and effective configuration information. The switching configuration information indicates that the image output mode is the second image output mode (i.e., mode B).

[0196] After the sensor driver writes all the switching configuration information to the corresponding registers, the sensor writes the effective configuration information to the effective registers. In addition, the IFE driver writes the IFE configuration information of N frames to the corresponding registers.

[0197] Exemplarily, as described above, the Sensor is configured with effective point information to indicate the position of the effective point within the frame. In an embodiment of the present application, the effective point is spaced X time lengths apart from the SOF of the image frame. As shown in FIG9 , since the frame length of the N frame is the second frame length, it increases the extended frame length. Therefore, the original effective point of the N+1 frame (i.e., the T6-1 moment) is moved backward, and the actual effective point of the N+1 frame (i.e., the T7-1 moment) is spaced "extended time length" from the original effective point (i.e., the T6-1 moment). Moreover, the effective point of the N+1 frame (i.e., the T7-1 moment) and the SOF of the N+1 frame are spaced X time lengths apart.

[0198] Still referring to Figure 9, at time T7-1, the effective point of frame N+1 (which can also be understood as the exposure start point of frame N+1), the sensor detects that the effective configuration information has been written to the effective register. In other words, the configuration information for frame N has been fully written. The sensor exposes and outputs frame N+1 based on the sensor configuration information for frame N. That is, the frame length of frame N+1 is the first frame length, and the output mode is the second output mode (i.e., mode B).

[0199] The sensor outputs the image frame to the IFE. The IFE also takes effect on the IFE configuration information of frame N at frame N+1. That is, the IFE processes the image captured by the sensor in the second image output mode based on the preprocessing parameters corresponding to the second image output mode.

[0200] As shown in Figure 9, even if the sensor takes a long time to write the configuration information (for example, the time point of writing the effective register has exceeded the original effective point), the delayed effective point of frame N+1 gives the sensor enough time to write the configuration information and complete the writing of the configuration information of frame N before the effective point of frame N+1. This further enables the sensor to capture image frames based on the new output mode in a timely manner at frame N+1.

[0201] In one possible implementation, if the Sensor node predicts that the time it takes to write the configuration information to the sensor may not exceed the original effective point based on factors such as the length of the sensor configuration information and the device's capabilities, the Sensor node can also configure the Ratio to 1, meaning that the Sensor still exposes and outputs N frames according to the original duration.

[0202] The embodiment of the present application also provides another shooting parameter setting method, in which the Sensor is configured to take effect on the configuration information of N frames in the N+2 frame. The IFE is still configured to take effect on the configuration information of N frames in the N+1 frame. Correspondingly, the CRM performs data synchronization based on the N+1 frame effectiveness timing of the IFE and the N+2 frame effectiveness timing of the Sensor. That is, the CRM sends the Sensor configuration information of N frames to the Sensor driver in the N frame, so that the Sensor writes the Sensor configuration information of N frames in the N frame, and the Sensor configuration information takes effect in the N+2 frame, that is, the CRM takes effect on the configuration information in the N+2 frame to the CRM. In addition, the CRM sends the IFE configuration information of N frames to the IFE driver in the N+1 frame, so that the IFE writes the IFE configuration information of N frames in the N+1 frame, and the IFE configuration information takes effect in the N+2 frame. In this way, in the scenario where the sensor writes the sensor configuration information of frame N in frame N, the sensor may complete writing the sensor configuration information before the effective point of frame N+1, or it may complete writing the sensor configuration information before the effective point of frame N+2. By writing the effective configuration information of the sensor in frame N+1, the present application can ensure that the IFE configuration information and the sensor configuration information are synchronized in frame N+2, that is, the second output mode can be correctly switched to in frame N+2.

[0203] The above method is described in detail below with reference to Figure 10. Figure 10 is an exemplary image frame exposure timing diagram. Referring to Figure 10, at time T1-1 (which can be any time before time T3, not limited in this application), the decision module, based on the scene environment information, decides to switch from the first image output mode to the second image output mode. With reference to Figure 6b, the decision module indicates the second image output mode to the IFE node and the Sensor node, triggering the IFE node and the Sensor node to generate corresponding configuration information.

[0204] The sensor node generates N frames of sensor configuration information in response to the instruction of the decision module. The sensor configuration information includes, but is not limited to, camera parameters and image output mode configuration information. The camera parameters include, but are not limited to, the frame length of the N frames (e.g., the first frame length). The sensor image output mode configuration information includes, but is not limited to, switching configuration information and effective configuration information. The switching configuration information includes configuration parameters corresponding to the second image output mode.

[0205] The IFE node generates N frames of IFE configuration information in response to an instruction from the decision module. The IFE configuration information includes, but is not limited to, preprocessing parameters corresponding to the second image output mode, so that the IFE can preprocess the preview stream captured by the target camera according to the target image output mode (i.e., the second image output mode).

[0206] The sensor node sends N frames of sensor configuration information to the CRM, and the IFE node sends N frames of IFE configuration information to the CRM. The CRM receives N frames of sensor and IFE configuration information. The specific data transmission flow is shown in Figure 6b and is not detailed here.

[0207] Exemplarily, as described above, based on the timing of the configuration information of N frames taking effect on the Sensor at frame N+2, and the timing of the configuration information of N frames taking effect on the IFE at frame N+1, the CRM controls the timing of sending the Sensor's switching configuration information, effective configuration information, and IFE configuration information, so as to achieve data synchronization between the IFE configuration information and the Sensor configuration information at frame N+2. Accordingly, in the example shown in FIG10 , at time T3-1 (which can be any time within frame N, not limited in this application), as shown in FIG11a , the CRM drives the output of other configuration information except the effective configuration information to the Sensor, such as switching configuration information and camera parameters (such as frame length, etc.).

[0208] Still referring to Figure 10, at time T2-1, the Sensor exposes N frames and outputs N frames at time T3. Among them, the exposure of N frames uses the configuration information of N-1 frame. For example, the frame length of N frame is the first frame length, and the output mode is the first output mode (i.e. Mode A). At time T3-1 (which can be any time after the SOF of N frame and before the effective point of N+1 frame, not limited in this application), the Sensor driver writes the Sensor configuration information of N frames into the output mode register corresponding to the Sensor, so that the Sensor outputs the image according to the specified output mode (i.e. the second output mode). Exemplarily, the Sensor configuration information includes but is not limited to camera parameters and switching configuration information. Camera parameters include but are not limited to the frame length of N frames (for example, the first frame length).

[0209] At time T4-1, the sensor determines that this time point is the effective point of frame N+1 based on the effective point configuration information (for example, the effective point is X time length away from the SOF of frame N). In this example, it is the exposure start point of frame N+1. Since the effective configuration information of frame N has not yet been written to the corresponding register, that is, the sensor has not yet obtained the effective configuration information of frame N. Therefore, the sensor still outputs the image according to the old configuration information. In other words, the frame length of frame N is the first frame length, and the output mode of frame N is the first output mode, that is, mode A.

[0210] At time T5-1 (it can be at time T6-1, that is, any time before the effective point of frame N+2, which is not limited in this application), as shown in Figure 11b, the CRM sends the effective configuration information of frame N to the Sensor driver, and the Sensor driver writes the effective configuration information of frame N into the effective register of the Sensor.

[0211] For example, at time T5-1 (which can be time T6-1, that is, any time before the effective point of frame N+2, not limited in this application), CRM sends IFE configuration information of N frames to the IFE driver, and the IFE driver writes the IFE configuration information of N frames into the register of IFE.

[0212] Optionally, the time when the CRM sends N frames of IFE configuration information to the IFE driver and the time when the CRM sends N frames of effective configuration information to the Sensor driver may be different, which is not limited in this application.

[0213] At time T6-1, the Sensor determines that this time point is the effective point of frame N+2 based on the effective point configuration information (for example, the effective point is X time length away from the SOF of frame N+1). In this example, it is the exposure start point of frame N+2. As mentioned above, before this moment, the Sensor has written the effective configuration information. Accordingly, at this effective point, the Sensor can output the image based on the configuration information of frame N that has been written (including the second output mode and the first frame length, etc.), that is, it can expose and output the N+2 frame. In other words, the output mode of frame N+2 is the second output mode, that is, mode B.

[0214] For example, the sensor outputs frame N+2 to the IFE. As mentioned above, the IFE takes effect on frame N+1. Therefore, if the IFE writes IFE configuration information in frame N+1, this IFE configuration information will take effect on frame N+2. Accordingly, the IFE processes frame N+1 based on the IFE configuration information in frame N. For example, the IFE performs image processing on frame N+1 based on the preprocessing parameters corresponding to the second output mode.

[0215] The IFE outputs the processed image to the camera application, which displays frame N+2. Due to the switching of the image output mode, the display effect of frame N+2 is different from that of frame N+1, frame N, and so on.

[0216] Exemplarily, as described above, the effective point of the camera parameters and the effective point of the image output configuration may be different. The following is a detailed description of this scenario in conjunction with Figures 7, 9 and 10 above. In conjunction with Figure 7, Figure 12 is an exemplary image frame exposure timing diagram. Please refer to Figure 12. At time T1-1 (which can be any time before N frames, this application is not limited), the Sensor node sends the Sensor configuration information of N frames to the CRM (including the image output mode configuration information and camera parameters corresponding to the second image output mode, etc.), and the IFE node sends the IFE configuration information of N frames to the CRM. The CRM obtains the Sensor configuration information and IFE configuration information corresponding to N frames (the specific data flow can be referred to Figure 6b, which will not be repeated here). Optionally, as described above, the camera parameters and image output configuration information in the Sensor configuration information can be sent to the CRM at different times. The specific sending process can refer to the existing technical embodiments, which is not limited in this application.

[0217] In this scenario, the camera parameter effective point is different from the output configuration effective point. The camera parameter effective point is the exposure duration starting point (it can also be any time after the end of frame N-1 and before the exposure starting point, which is not limited in this application). That is, at time T2-1, the sensor exposes frame N-1 based on the camera parameters (such as frame length, etc.).

[0218] At time T2-2 (which can be any time between the EOF of frame N-1 and the SOF of frame N, not limited in this application), the Sensor detects whether the Sensor output configuration information is received at this time based on the effective point information. Among them, the output mode configuration information in the current configuration information of the Sensor indicates the first output mode, that is, mode A. Correspondingly, at time T3, the Sensor outputs frame N according to the first output mode.

[0219] For the configuration information of frame N, it is written in frame N and takes effect in frame N+1. Specifically, at the T3-1 moment in frame N (generally any moment between SOR and EOF, which can be set according to actual needs and is not limited in this application), the CRM sends the Sensor configuration information corresponding to frame N (including camera parameters and the output mode configuration information corresponding to the second output mode) to the Sensor driver, and the CRM sends the IFE configuration information corresponding to frame N (including the output mode configuration information (the output mode configuration information indicates the second output mode)) to the IFE driver.

[0220] At time T3-1, the sensor driver and the IFE driver write corresponding configuration information to the sensor and the IFE, respectively. It should be noted that there may be a delay between the two writing actions, which is not limited in this application.

[0221] In an embodiment of the present application, as described above, the output mode configuration information of the Sensor may further include switching configuration information and effective configuration information. Exemplarily, the Sensor driver writes the switching configuration information to the register corresponding to the Sensor (recorded as the output mode register). After the Sensor driver writes all the switching configuration information to the register, the Sensor writes the effective configuration information to the corresponding register (recorded as the effective register). The writing method of other configuration information (such as camera parameters) can refer to the existing technology, and this application does not limit it.

[0222] Exemplarily, the IFE driver writes N frames of IFE configuration information into a register corresponding to the IFE.

[0223] Optionally, the number of registers written by the IFE is smaller than the number of registers that the sensor needs to write. Typically, the configuration information of the sensor is longer and the number of registers that need to be written is larger.

[0224] At time T4-1, the Sensor exposes N frames based on camera parameters (such as frame length, etc.). At time T4-2 (which can be any time after the EOF of frame N and before the SOF of frame N+1, not limited in this application), the Sensor detects whether the Sensor output configuration information is received at this moment based on the effective point information. In this example, the Sensor has obtained the Sensor output configuration information, that is, the Sensor output configuration information of frame N takes effect at this point. Correspondingly, at time T5, the Sensor outputs N frames according to the second output mode.

[0225] For other undescribed parts, please refer to the relevant descriptions of Figures 7 and 8, which will not be repeated here.

[0226] In conjunction with FIG9 , FIG13 is an exemplary image frame exposure timing diagram.

[0227] At time T1-1 (which can be any time after receiving the output mode information from the decision module and before T2-1, and is not limited in this application), the Sensor node responds to the instruction of the decision module and determines that the output mode needs to be switched in frame N. The Sensor node generates configuration information for frame N-1. The configuration information of frame N-1 includes but is not limited to camera parameters, wherein the frame length in the camera parameters is the second frame length. The current default frame length is the first frame length, for example, 33ms, which can be set according to actual needs, and is not limited in this application. , the Sensor node sends the configuration information of frame N-1 to CRM. CRM obtains the Sensor configuration information of frame N-1 (including the second frame length).

[0228] At time T2-1, the camera parameters take effect. The sensor exposes frame N-1 based on the camera parameters of frame N-2. At time T2-2, the image output configuration takes effect, the sensor checks whether it has received the image output configuration information. The sensor still uses the old image output configuration information. Accordingly, at time T3, the sensor outputs frame N-1 based on the first image output mode.

[0229] At time T3-1 (which can be after the Sensor sends the Sensor configuration information of frame N-1 and before time T5 (i.e., frame N), this application does not limit it), the Sensor node generates the configuration information of frame N. As mentioned above, the Sensor node has obtained from the decision module that the output mode of frame N will switch to the second output mode. Accordingly, the configuration information of frame N includes output mode configuration information, and the output mode configuration information includes but is not limited to switching configuration information and effective configuration information. The switching configuration information is used to instruct the Sensor to switch to the second output mode (i.e., mode B). In addition, it is expected that the frame length of frame N+1 will be restored to the first frame length, and accordingly, the frame length in the camera parameters in the configuration information of frame N is the first frame length.

[0230] The Sensor node sends N frames of Sensor configuration information to the CRM. The specific sending process can be found in Figure 6b and will not be repeated here.

[0231] Exemplarily, the IFE node also generates IFE configuration information of frame N. The IFE configuration information includes but is not limited to pre-processing parameters required for the second image output mode.

[0232] The IFE node sends N frames of IFE configuration information to the CRM.

[0233] The CRM obtains N frames of sensor configuration information and IFE configuration information.

[0234] At time T3-2 (which can be any time between the SOF of frame N-1 (i.e., time T3) and before the exposure of frame N (i.e., time T4-1), this application does not limit this), the CRM sends the configuration information of frame N-1 (including the second frame length) to the sensor driver. The sensor driver writes the configuration information of frame N-1 to the corresponding register of the sensor.

[0235] At time T4-1, the camera parameters for frame N take effect. The sensor exposes frame N based on the camera parameters for frame N-1 (i.e., the second frame length). At time T4-2, the output configuration takes effect, the sensor checks whether it has received the output configuration information. The sensor still uses the old output configuration information. Accordingly, at time T5, the sensor outputs frame N based on the first output mode.

[0236] At time T5-2 (which can be any time before time T6-1), the CRM sends the Sensor configuration information of N frames acquired at time T3-1 to the Sensor driver, and sends the IFE configuration information of N frames to the IFE driver. The Sensor configuration information includes but is not limited to camera parameters (where the frame length is the first frame length) and image output mode configuration information. The image output mode configuration information includes but is not limited to switching configuration information and effective configuration information. The switching configuration information indicates that the image output mode is the second image output mode (i.e., mode B).

[0237] After the sensor driver writes all the switching configuration information to the corresponding registers, the sensor writes the effective configuration information to the effective registers. In addition, the IFE driver writes the IFE configuration information of N frames to the corresponding registers.

[0238] At time T7-1, which is the effective point of the camera parameters of frame N+1, the Sensor exposes frame N+1 based on the camera parameters of frame N (i.e., the first frame length). At time T7-2, which is the effective point of the image output configuration, the Sensor detects whether the image output configuration information is received. The Sensor has received the image output configuration information of frame N (i.e., the second image output mode). Correspondingly, at time T8, the Sensor outputs frame N+1 based on the second image output mode. The Sensor outputs the image frame to IFE. IFE also takes effect on frame N+1 of the IFE configuration information, that is, IFE processes the image captured by the Sensor based on the second image output mode based on the pre-processing parameters corresponding to the second image output mode. For other undescribed parts, please refer to Figure 9, which will not be repeated here.

[0239] In conjunction with Figure 10, Figure 14 is an example of an image frame exposure timing diagram. Referring to Figure 14, at time T1-1 (which can be any time before time T3, not limited in this application), the decision module, based on the scene environment information, decides to switch from the first image output mode to the second image output mode. In conjunction with Figure 6b, the decision module indicates the second image output mode to the IFE node and the sensor node, triggering the IFE node and the sensor node to generate corresponding configuration information.

[0240] The sensor node generates N frames of sensor configuration information in response to the instruction of the decision module. The sensor configuration information includes, but is not limited to, camera parameters and image output mode configuration information. The camera parameters include, but are not limited to, the frame length of the N frames (e.g., the first frame length). The sensor image output mode configuration information includes, but is not limited to, switching configuration information and effective configuration information. The switching configuration information includes configuration parameters corresponding to the second image output mode.

[0241] The IFE node generates N frames of IFE configuration information in response to an instruction from the decision module. The IFE configuration information includes, but is not limited to, preprocessing parameters corresponding to the second image output mode, so that the IFE can preprocess the preview stream captured by the target camera according to the target image output mode (i.e., the second image output mode).

[0242] The sensor node sends N frames of sensor configuration information to the CRM, and the IFE node sends N frames of IFE configuration information to the CRM. The CRM receives N frames of sensor and IFE configuration information. The specific data transmission flow is shown in Figure 6b and is not detailed here.

[0243] For example, as described above, based on the timing of the sensor's configuration information taking effect in frame N+2, and the timing of the IFE's configuration information taking effect in frame N+1, the CRM controls the timing of the sensor's switching configuration information and effective configuration information, as well as the timing of sending the IFE configuration information, so as to achieve data synchronization between the IFE configuration information and the Sensor configuration information in frame N+2.

[0244] At time T2-1, the sensor exposes frame N based on the camera parameters (e.g., frame length, etc.) of frame N-2. Optionally, the camera parameters can be effective at frame N+1, that is, the camera parameters of frame N still take effect at frame N+1. This is not limited in this application.

[0245] At time T2-2 (which can be any time between the EOF of frame N and the SOF of frame N, not limited in this application), the Sensor detects whether the Sensor output configuration information is received at this time based on the effective point information. Among them, the output mode configuration information in the current configuration information of the Sensor indicates the first output mode, that is, mode A. Correspondingly, at time T3, the Sensor outputs N frames according to the first output mode.

[0246] At time T3-1 (which can be any time after the SOF of frame N and before the effective point of frame N+1, not limited in this application), the sensor driver writes the sensor configuration information of frame N into the image output mode register corresponding to the sensor, so that the sensor outputs the image according to the specified image output mode (i.e., the second image output mode). Exemplarily, the sensor configuration information includes but is not limited to camera parameters and switching configuration information. The camera parameters include but are not limited to the frame length of frame N (for example, the first frame length).

[0247] At time T4-1, the sensor exposes frame N+1 based on the camera parameters (such as frame length, etc.) of frame N-1.

[0248] At time T4-2 (which can be any time between the EOF of frame N and the SOF of frame N+1, not limited in this application), the sensor detects whether the sensor output configuration information is received at this time based on the effective point information. Among them, the output mode configuration information in the current configuration information of the sensor indicates the first output mode, that is, mode A. Correspondingly, at time T5, the sensor outputs the N+1 frame according to the first output mode.

[0249] At time T5-1 (it can be at time T6-1, that is, any time before the effective point of frame N+2, which is not limited in this application), as shown in Figure 11b, the CRM sends the effective configuration information of frame N to the Sensor driver, and the Sensor driver writes the effective configuration information of frame N into the effective register of the Sensor.

[0250] At time T6-1, the sensor exposes frame N+2 based on the camera parameters of frame N (such as frame length, etc.).

[0251] At time T6-2 (which can be any time after the EOF of frame N+1 and before the SOF of frame N+2, and is not limited in this application), the Sensor detects whether the Sensor output configuration information is received at this moment based on the effective point information. At time T6-2, which is the effective point of the output configuration, the Sensor detects whether the output configuration information is received, and the Sensor has received the output configuration information of frame N (i.e., the second output mode). Correspondingly, at time T7, the Sensor outputs the N+2 frame based on the second output mode. The Sensor outputs the image frame to IFE. IFE takes effect on the IFE configuration information of frame N at frame N+1, that is, IFE processes the image captured by the Sensor based on the second output mode based on the preprocessing parameters corresponding to the second output mode. For other undescribed parts, please refer to Figure 10, which will not be repeated here.

[0252] It is understandable that, in order to implement the above functions, the electronic device includes hardware and / or software modules that perform the corresponding functions. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0253] In an example, FIG15 shows a schematic block diagram of a device 1500 according to an embodiment of the present application. The device 1500 may include: a processor 1501 and a transceiver / transceiver pin 1502 , and optionally, a memory 1503 .

[0254] The various components of the device 1500 are coupled together via a bus 1504, wherein the bus 1504 includes not only a data bus but also a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all buses are referred to as bus 1504 in the figure.

[0255] Optionally, the memory 1503 may be used for instructions in the aforementioned method embodiment. The processor 1501 may be used to execute instructions in the memory 1503 and control the receiving pin to receive a signal and control the transmitting pin to send a signal.

[0256] The apparatus 1500 may be the electronic device or a chip of the electronic device in the above method embodiment.

[0257] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0258] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

[0259] This embodiment further provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the above-mentioned related steps to implement the method in the above-mentioned embodiment.

[0260] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the methods in the above-mentioned method embodiments.

[0261] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0262] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for setting photographing parameters, characterized in that: Applied to electronic equipment, the method comprises: At a first moment, obtaining first camera parameters of the N-1th frame, where the first camera parameters include a first frame length; At a second moment, obtaining second image output mode configuration information and second camera parameters of the Nth frame; wherein the second image output mode configuration information includes configuration parameters corresponding to the second image output mode, and the second camera parameters include a second frame length, and the second frame length is less than the first frame length; At a third moment, writing the first camera parameter into the camera sensor; The camera sensor exposes and outputs the Nth frame based on the first camera parameter; the frame length of the Nth frame is the first frame length, and the output mode of the Nth frame is the first output mode; At a fourth moment, writing the second image output mode configuration information and the second camera parameters into the camera sensor; The camera sensor exposes and outputs the N+1th frame based on the second camera parameters and the second image output mode configuration information; wherein, at the fourth moment in the Nth frame, the frame length of the N+1th frame is the second frame length, and the image output mode of the N+1th frame is the second image output mode.

2. The method according to claim 1, characterized in that The duration between the effective point of the image output mode of the Nth frame and the start of frame delimiter SOF of the Nth frame is the first duration, and the duration between the effective point of the image output mode of the N+1th frame and the SOF of the N+1th frame is the first duration; The camera sensor exposes and outputs the N+1th frame based on the second camera parameter and the second image output mode configuration information, including: The camera sensor obtains the second image output mode configuration information at the image output mode effective point of the N+1th frame, The camera sensor outputs the N+1th frame at the SOF of the N+1th frame.

3. The method according to claim 1, characterized in that Before writing the first camera parameter into the camera sensor, the method further includes: At a fifth moment, obtaining IFE configuration information of the Nth frame, wherein the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; The step of writing the second image output mode configuration information and the second camera parameter into the camera sensor further includes: The IFE configuration information is written into the IFE.

4. The method according to claim 3, characterized in that The method further comprises: The IFE performs image processing on the N+1th frame input by the camera sensor based on the IFE configuration information.

5. The method according to any one of claims 1 to 4, characterized in that: The first frame length is n times the second frame length, where n is greater than or equal to 1 and less than 2.

6. A method for setting photographing parameters, characterized in that: Applied to electronic equipment, the method comprises: At a first moment, obtaining second image output mode configuration information of the Nth frame and IFE configuration information of the Nth frame, wherein the second image output mode configuration information includes effective configuration information and configuration parameters corresponding to the second image output mode, and the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; At a second moment, the Nth frame is output; wherein the output mode of the Nth frame is the first output mode; At the third moment, the configuration parameters corresponding to the second image output mode are written into the camera sensor; wherein the configuration parameters corresponding to the second image output mode are written into the camera sensor; The three moments are within the Nth frame; At the fourth moment, the N+1th frame is output; wherein the output mode of the N+1th frame is the first output mode; At a fifth moment in the N+1th frame, writing the effective configuration information into the camera sensor, and writing the IFE configuration information into the IFE; The camera sensor outputs an image for the N+2th frame based on the second image output mode configuration information, and the image output mode of the N+2th frame is the second image output mode.

7. The method according to claim 6, characterized in that The duration between the effective point of the image output mode of the N+1th frame and the start of frame delimiter SOF of the N+1th frame is the first duration, and the duration between the effective point of the image output mode of the N+2th frame and the SOF of the N+2th frame is the first duration; The camera sensor outputs an image for the N+2th frame based on the second image output mode configuration information, including: The camera sensor obtains the second image output mode configuration information at the image output mode effective point of the N+2th frame, The camera sensor outputs the N+2th frame at the SOF of the N+2th frame.

8. The method according to claim 7, characterized in that The fifth moment is before the effective point of the image output mode of the N+2th frame.

9. The method according to claim 6, characterized in that The method further comprises: The IFE performs image processing on the N+2th frame input by the camera sensor based on the IFE configuration information.

10. An electronic device, characterized in that: include: One or more processors, memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the following steps: At a first moment, obtaining first camera parameters of the N-1th frame, wherein the first camera parameters include a first frame length, At a second moment, obtaining second image output mode configuration information and second camera parameters of the Nth frame; wherein the second image output mode configuration information includes configuration parameters corresponding to the second image output mode, and the second camera parameters include a second frame length, and the second frame length is less than the first frame length; At a third moment, writing the first camera parameter into the camera sensor; The camera sensor exposes and outputs the Nth frame based on the first camera parameter; the frame length of the Nth frame is the first frame length, and the output mode of the Nth frame is the first output mode; At a fourth moment, writing the second image output mode configuration information and the second camera parameters into the camera sensor; The camera sensor exposes and outputs the N+1th frame based on the second camera parameters and the second image output mode configuration information; wherein, at the fourth moment in the Nth frame, the frame length of the N+1th frame is the second frame length, and the image output mode of the N+1th frame is the second image output mode.

11. The electronic device according to claim 10, characterized in that: The duration between the effective point of the image output mode of the Nth frame and the start of frame delimiter SOF of the Nth frame is the first duration, and the duration between the effective point of the image output mode of the N+1th frame and the SOF of the N+1th frame is the first duration; when the computer program is executed by the one or more processors, the electronic device performs the following steps: The camera sensor obtains the second image output mode configuration information at the image output mode effective point of the N+1th frame, The camera sensor outputs the N+1th frame at the SOF of the N+1th frame.

12. The electronic device according to claim 10, characterized in that: When the computer program is executed by the one or more processors, the electronic device performs the following steps: At a fifth moment, obtaining IFE configuration information of the Nth frame, wherein the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; When the computer program is executed by the one or more processors, the electronic device performs the following steps: The IFE configuration information is written into the IFE.

13. The electronic device according to claim 12, characterized in that: When the computer program is executed by the one or more processors, the electronic device performs the following steps: The IFE performs image processing on the N+1th frame input by the camera sensor based on the IFE configuration information.

14. The electronic device according to any one of claims 10 to 13, characterized in that: The first frame length is n times the second frame length, where n is greater than or equal to 1 and less than 2.

15. An electronic device, characterized in that: include: One or more processors, memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the following steps: At a first moment, obtaining second image output mode configuration information of the Nth frame and IFE configuration information of the Nth frame, wherein the second image output mode configuration information includes effective configuration information and configuration parameters corresponding to the second image output mode, and the IFE configuration information includes preprocessing parameters corresponding to the second image output mode; At a second moment, the Nth frame is output; wherein the output mode of the Nth frame is the first output mode; At a third moment, writing the configuration parameters corresponding to the second image output mode into the camera sensor; wherein the third moment is within the Nth frame; At the fourth moment, the N+1th frame is output; wherein the output mode of the N+1th frame is the first output mode; At a fifth moment in the N+1th frame, writing the effective configuration information into the camera sensor, and writing the IFE configuration information into the IFE; The camera sensor outputs an image for the N+2th frame based on the second image output mode configuration information, and the image output mode of the N+2th frame is the second image output mode.

16. The electronic device according to claim 15, characterized in that: The duration between the effective point of the image output mode of the N+1th frame and the start of frame delimiter SOF of the N+1th frame is the first duration, and the duration between the effective point of the image output mode of the N+2th frame and the SOF of the N+2th frame is the first duration; when the computer program is executed by the one or more processors, the electronic device performs the following steps: The camera sensor obtains the second image output mode configuration information at the image output mode effective point of the N+2th frame, The camera sensor outputs the N+2th frame at the SOF of the N+2th frame.

17. The electronic device according to claim 16, characterized in that: The fifth moment is before the effective point of the image output mode of the N+2th frame.

18. The electronic device according to claim 15, characterized in that: When the computer program is executed by the one or more processors, the electronic device performs the following steps: The IFE performs image processing on the N+2th frame input by the camera sensor based on the IFE configuration information.

19. A computer storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, enable the electronic device to execute the method according to any one of claims 1 to 5 or the method according to any one of claims 6 to 9.

20. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 5 or the method according to any one of claims 6 to 9.

21. A chip, characterized in that: The electronic device comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of the electronic device and send the signal to the processor, wherein the signal includes a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the method described in any one of claims 1 to 5 or executes the method described in any one of claims 6 to 9.

Images