Photographing parameter setting method and electronic equipment

Abstract

The invention provides a photographing parameter setting method and electronic equipment, and relates to the field of terminal equipment. Shooting parameter setting is provided, under the condition that a sensor is switched to an image output mode, a sensor driver detects whether effective configuration information of an Nth frame is issued or not, and when the sensor driver determines that the effective configuration information of the Nth frame is issued, a first data packet of the Nth frame is issued to a camera sensor, and when the sensor driver determines that the effective configuration information of the Nth frame is not issued, issuing the effective configuration information of the Nth frame to the camera sensor. By adopting the method provided by the embodiment of the invention, when the sensor driver in the electronic equipment is indicated to repeatedly send the graph mode configuration information, whether the graph mode configuration information has been issued or not can be firstly judged, and different information is issued to the camera sensor based on the result; therefore, the problem of abnormal image generation caused by repeated issuing of the effective configuration information of the camera sensor can be avoided.

Description

Technical Field

[0001] This application relates to the field of terminal equipment, and in particular to a method for setting photographic parameters and an electronic device. Background Technology

[0002] With the development of terminal technology, the application scenarios of terminal devices are becoming increasingly widespread. For example, users can use their mobile phones to take photos to meet their needs for recording daily life. During the photo-taking process, the terminal can automatically switch between different image output modes according to the current environment to meet the needs of the current environment, thereby improving image quality.

[0003] However, when the terminal device has insufficient performance, frame drops are likely to occur, resulting in choppy and unsmooth images when taking pictures. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a method for setting photographic parameters and an electronic device. In this method, when the sensor driver in the electronic device is instructed to issue image mode configuration information, it can first determine whether the image mode configuration information has already been issued. Based on the determination result, different information is issued to the camera sensor to avoid the problem of abnormal image output caused by repeatedly issuing the effective configuration information of the camera sensor.

[0005] In a first aspect, embodiments of this application provide a camera parameter setting and electronic device, applied to an electronic device. The method includes: a sensor driver acquiring image output mode configuration information of the Nth frame from a sensor node, the image output mode configuration information of the Nth frame including switching configuration information for switching from a first image output mode to a second image output mode, effective configuration information corresponding to the switching configuration information, and a first data packet, the first data packet including first exposure information and first gain information; at a first moment, the sensor driver sends the switching configuration information of the Nth frame to the camera sensor, the first moment being within the N-2th frame; the sensor driver determines whether the effective configuration information of the Nth frame has been sent to the camera sensor. The sensor driver detects that the effective configuration information of the Nth frame has been sent to the camera sensor, and sends the first data packet of the Nth frame to the camera sensor at a second time. The first time is before the second time. When the sensor driver determines that the effective configuration information of the Nth frame has not been sent to the camera sensor, it sends the effective configuration information of the Nth frame to the camera sensor at a second time. The camera sensor outputs the image of the Nth frame based on the image output mode configuration information of the Nth frame. The image output mode of the Nth frame is the second image output mode. Among them, the image output modes of the N-3rd frame, the N-2nd frame, and the N-1st frame are the first image output modes, and N is an integer greater than or equal to 3.

[0006] For example, electronic devices can be mobile phones, tablets, smart bracelets, in-vehicle computers, etc.

[0007] For example, the initial output mode of an electronic device is usually the Binning output mode.

[0008] For example, the Nth frame can be as follows: Figure 6 As shown, the output mode of frame N-3 is the first output mode (e.g., Figure 6 As shown in mode A), the output mode of the Nth frame is the second output mode (e.g., mode A). Figure 6 (Mode B shown). When the decision module in the electronic device determines that the Nth frame is switched to the second image output mode, the image output mode configuration information for the Nth frame includes switching configuration information, effective configuration information, and a first data packet. The first and second image output modes are different; that is, when the decision module changes the image output mode of the camera sensor (i.e., the Sensor), the image output mode configuration information includes switching configuration information, effective configuration information, and a first data packet. When the image output mode does not change, the image output mode configuration information includes camera parameters (such as frame length).

[0009] For example, refer to Figure 5d The first moment can be a moment within the (N-2)th frame.

[0010] For example, continue to refer to Figure 5d The second moment can be a moment within the (N-1)th frame.

[0011] For example, the sensor driver obtains the image output mode configuration information for the Nth frame at least two frames in advance. That is, the sensor driver obtains the image output mode configuration information for the Nth frame in frame N-2, and the configuration takes effect in frame N. In some examples, such as when the sensor driver is triggered to perform a rollback operation, the image output mode configuration information for the Nth frame will be reissued after frame N-2.

[0012] Thus, when switching image output modes, the switching configuration information for frame N needs to be paired with the effective configuration information for frame N. If the electronic device experiences performance issues that prevent the IFE driver from properly sending the IFE configuration information, the sensor driver will be triggered to perform a rollback operation. This can result in the same effective configuration information for the same frame being sent repeatedly to the camera sensor. In other words, the camera sensor will have one image output configuration information for frame N and two effective configuration information for frame N, leading to image output failure and frame loss. In this example, the sensor driver first checks whether the effective configuration information for frame N has already been sent. If it is confirmed that the effective configuration information for frame N has been sent, the first data packet of frame N is sent to the camera sensor. This avoids the problem of one switching configuration information for frame N and two effective configuration information for frame N appearing in the camera sensor due to repeated sending of the effective configuration information for frame N, thereby preventing frame loss caused by abnormal image output of frame N. In addition, the first data packet of the Nth frame includes exposure information and gain information, which can also ensure that the output mode configuration information of the Nth frame can take effect normally and avoid problems with abnormal activation.

[0013] According to the first aspect, the sensor driver determines whether the effective configuration information of the Nth frame has been sent to the camera sensor, including: the sensor driver acquiring the first identifier of the Nth frame, the first identifier of the Nth frame corresponding to the output mode configuration information of the Nth frame, the first identifier of the Nth frame being used to record whether the sensor driver has sent the effective configuration information of the Nth frame to the camera sensor; when the sensor driver detects that the first identifier of the Nth frame is a first state value, it determines that the sensor driver has sent the effective configuration information of the Nth frame to the camera sensor; when the sensor driver detects that the first identifier of the Nth frame is a second state value, it determines that the sensor driver has not sent the effective configuration information of the Nth frame to the camera sensor, and the first state value is different from the second state value.

[0014] For example, the image output mode configuration information for each frame can be set with a corresponding first identifier. The value of the first identifier includes a first state value (such as "true") and a second state value (such as "false"). If the image output mode configuration information for the current frame does not include switching configuration information or effective configuration information, the first identifier can default to the second state value.

[0015] In this way, the sensor driver stores the first identifier of each frame. When the sensor driver sends the effective configuration information of the Nth frame to the camera sensor, it can update the first identifier of the Nth frame from the second state value to the first state value. The sensor driver can quickly determine whether the effective configuration information of the Nth frame has been sent through the first identifier of the Nth frame.

[0016] According to the first aspect, the sensor driver obtains the image output mode configuration information for the Nth frame from the sensor node, including: within the (N-3)th frame, the sensor node sends the switching configuration information for the Nth frame to the sensor driver; within the (N-2)th frame, the sensor node sends the activation configuration information for the Nth frame and a first data packet to the sensor driver. In this way, the sensor node begins sending the image output mode configuration information for the Nth frame 3 frames in advance, ensuring that the sensor driver can send the image output mode configuration information for the Nth frame to the camera sensor at least 2 frames in advance.

[0017] According to the first aspect, before the sensor driver sends the switching configuration information of the Nth frame to the camera sensor, the method further includes: when the sensor driver receives the effective configuration information and the first data packet of the Nth frame sent by the sensor node, the sensor driver sends a first notification of the Nth frame to the Camera Request Manager (CRM), the first notification of the Nth frame being used to indicate that the sensor driver has received the effective configuration information and the first data packet of the Nth frame; when the IFE driver receives the frame header delimiter (SOF) signal of the (N-2)th frame output by the camera sensor, the IFE sends a first indication message to the CRM; in response to the first indication message, the CRM sends a second notification to the camera sensor, the second notification being used to instruct the sensor driver to send the switching configuration information of the Nth frame to the camera sensor; when the sensor driver receives the second notification, it triggers the sensor driver to execute the sending of the switching configuration information of the Nth frame to the camera sensor. The method further includes the following steps before the sensor driver determines whether the effective configuration information of the Nth frame has been sent to the camera sensor: In the (N-1)th frame, when the sensor driver receives the camera parameters of the N+1th frame sent by the sensor node, it sends a third notification to the Camera Request Manager (CRM). The third notification indicates that the sensor driver has received the camera parameters of the N+1th frame. When the IFE driver receives the frame header delimiter (SOF) signal of the N-1th frame output by the camera sensor, it sends a second indication message to the CRM. In response to the second indication message, the CRM sends a fourth notification to the camera sensor. The fourth notification indicates that the sensor driver has sent the effective configuration information of the Nth frame to the camera sensor. When the sensor driver receives the fourth notification, it triggers the sensor driver to perform the operation of determining whether the effective configuration information of the Nth frame has been sent to the camera sensor.

[0018] For example, you can refer to Figure 5d The sensor driver receives the handover configuration information of the Nth frame within the (N-3)th frame (i.e. Figure 5d (The request for the Nth frame in the process). The effective configuration information for the Nth frame is received within the (N-2)th frame of the sensor driver.

[0019] In this way, the switching configuration information for frame N is sent to the sensor driver before the effective configuration information and the first data packet for frame N. When the CRM determines that the sensor driver has received the sensor configuration information for frame N+1 (i.e., the camera parameters for frame N+1) and detects the SOF interrupt signal for frame N-2, it instructs the sensor driver to send the effective configuration information for frame N, thereby accurately controlling the timing of the sensor driver sending information. According to the first aspect, when the second time step is within the (N-1)th frame, after the effective configuration information of the Nth frame is sent to the camera sensor at the second time step, the method further includes: within the Nth frame, the sensor driver sends the camera parameters of the N+1th frame to the camera sensor and the IFE driver does not send the IFE configuration information of the N+1th frame to the IFE, where the IFE configuration information of the N+1th frame is sent from the IFE node to the IFE driver within the (N-1)th frame; when the CRM determines that the sensor driver has sent the camera parameters of the N+1th frame to the camera sensor within the Nth frame and the IFE driver has not sent the IFE configuration information of the N+1th frame to the IFE within the Nth frame, it instructs the camera sensor to discard the information within the N+1th frame and instructs the IFE to discard the information within the N+1th frame; the camera sensor discards the information within the N+1th frame and the IFE discards the information within the N+1th frame; the CRM sends the effective configuration information of the Nth frame to the sensor driver within the N+1th frame. A first rollback command and a second rollback command are sent to the IFE driver. The first rollback command instructs the sensor driver to resend the effective configuration information of frame N within frame N+1, and the second rollback command instructs the IFE driver to resend the IFE configuration information of frame N within frame N+1. In response to the first rollback command, the sensor driver determines whether the effective configuration information of frame N has been sent to the camera sensor. When the sensor driver detects that the effective configuration information of frame N has been sent, it sends the first data packet of frame N to the camera sensor within frame N+1. In response to the second rollback command, the IFE driver sends the IFE configuration information of frame N to the IFE within frame N+1. The camera sensor outputs the image of frame N+2 based on the image output mode configuration information of frame N. The IFE performs image processing on the image of frame N+2 output by the camera sensor based on the IFE configuration information of frame N.

[0020] In this way, if the sensor driver sends the effective configuration information for frame N to the sensor within frame N-1, and the image output mode for frame N+1 remains unchanged (i.e., frame N+1 is also in the second image output mode), the image output mode configuration information for frame N+1 includes camera parameters but does not include the effective configuration information or the switching configuration information. This image output mode configuration information for frame N+1 is sent to the sensor in frame N and takes effect in frame N+1. Since the IFE does not send the IFE configuration information for frame N+1 within frame N, a mismatch occurs between the sensor's image output and the IFE's processing of the image frame in frame N+1. The CRM instructs the sensor driver to resend the effective configuration information for frame N, and the IFE driver to resend the IFE configuration information for frame N. Responding to the CRM's instruction, the sensor driver detects that the effective configuration information for frame N has already been sent and sends the first data packet of frame N to the sensor, thus avoiding the problem of repeatedly sending the effective configuration information for frame N. The IFE driver also reissued the IFE configuration information, so that frames N+1 and N+2 could be displayed normally.

[0021] According to the first aspect, the method further includes: within the (N+2)th frame, the sensor driver sends the camera parameters of the (N+1)th frame to the camera sensor; the camera sensor outputs the image for the (N+3)th frame based on the camera parameters of the (N+1)th frame. Thus, after the IFE driver performs the rollback operation, the IFE driver sends the IFE configuration information for the (N+1)th frame within the (N+2)th frame, ensuring that subsequent IFE processing operations on image frames match the sensor's output mode, so that the image can be displayed correctly.

[0022] According to the first aspect, the method further includes: within the (N+2)th frame, the IFE driver sends the IFE configuration information of the (N+1)th frame to the IFE; the IFE performs image processing on the (N+3)th frame image output by the camera sensor based on the configuration information of the (N+1)th frame. In this way, the IFE driver successfully sends the IFE configuration information of the (N+1)th frame within the (N+2)th frame, ensuring that subsequent processing of the image output by the sensor is possible.

[0023] According to the first aspect, the effective point of the image output mode of the Nth frame is before the Nth frame, and the duration between the effective point of the image output mode of the Nth frame and the SOF of the Nth frame is a first duration; the camera sensor outputs an image of the Nth frame based on the second image output mode configuration information of the Nth frame, including: the camera sensor acquiring the second image output mode configuration information of the Nth frame at the effective point of the image output mode of the Nth frame; and the camera sensor outputting an image of the Nth frame at the SOF of the Nth frame.

[0024] In this way, by setting the activation timing of the sensor configuration of the electronic device to take effect in frame N+2, the electronic device has enough time to write the image output mode configuration information to the camera sensor, thus avoiding the frame loss problem caused by the asynchrony between the IFE configuration information and the sensor configuration information.

[0025] Secondly, this application provides an electronic device, including: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the first aspect and the photographing parameter setting method corresponding to any implementation of the first aspect.

[0026] The second aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the second aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0027] Thirdly, this application provides a computer-readable medium for storing a computer program that, when run on an electronic device, causes the electronic device to execute the photographing parameter setting method corresponding to the first aspect and any implementation thereof. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A schematic diagram of a user interface is shown as an example;

[0030] Figure 2 A schematic diagram of the hardware structure of an electronic device as an example;

[0031] Figure 3 A schematic diagram of the software structure of an electronic device as an example;

[0032] Figures 4a-4b This is a schematic diagram illustrating module interaction as an example.

[0033] Figures 5a-5b This is a schematic diagram illustrating module interaction as an example.

[0034] Figure 5cAn exemplary image frame exposure timing diagram is shown.

[0035] Figure 5d A timing diagram illustrating the distribution of output mode configuration information for an exemplary sensor;

[0036] Figure 6 This is a sequence diagram illustrating an example of an IFE failing to properly distribute IFE configuration information.

[0037] Figure 7 A flowchart illustrating an exemplary method for setting photographic parameters;

[0038] Figures 8a-8b A module interaction diagram illustrating an exemplary method for setting camera parameters;

[0039] Figure 9 This is a sequence diagram illustrating an example of an IFE failing to properly distribute IFE configuration information.

[0040] Figure 10 This is a schematic diagram of the structure of an exemplary device. Detailed Implementation

[0041] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0043] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0044] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0045] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0046] To facilitate understanding, the technical terms involved in the embodiments of this application will be explained first.

[0047] Dynamic range (VR) is the ratio of the maximum to the minimum brightness in a video or image signal. It's used in many fields to represent the ratio of a variable's maximum to its minimum value. In digital images, VR represents the ratio between the maximum and minimum grayscale values ​​within the displayable range of the image. The dynamic range in nature is quite large; the brightness of a starry night sky is approximately 0.001 cd / m², while the sun's brightness reaches 1,000,000,000 cd / m², resulting in a VR on the order of 1,000,000,000 / 0.001 = 10¹³. However, in real-world natural scenes, the brightness of the sun and starlight are not simultaneously present. For real-world natural scenes, the VR is typically around 10... -3 Up to 10 6 Within this range, most color digital images currently use one byte (8 bits) to store each of the R, G, and B channels. This means each channel represents a grayscale range of 0 to 255 levels. This 0-255 range is the image's dynamic range. Since the dynamic range of the same scene in the real world is between 10⁻³ and 10⁶, we call this High Dynamic Range (HDR). The dynamic range of ordinary images is, in contrast, Low Dynamic Range (LDR). The imaging process of a digital camera is essentially a mapping from the high dynamic range of the real world to the low dynamic range of the photograph. This mapping from high dynamic range in the real world to low dynamic range in the photograph is often a non-linear process.

[0048] Gain: The process of amplifying or reducing the electrical or digital signal after photoelectric conversion. Increasing the gain results in a brighter image than the actual scene; decreasing the gain results in a darker image than the actual scene.

[0049] Seamless switching: Seamless switching is a fast, uninterrupted transition. During camera image output, the software automatically switches between different output modes based on the current environment. This switching process is called seamless switching. Functionally, there needs to be no delay during the switching process; otherwise, it will cause a lag-like experience for the user.

[0050] Image output mode: In this embodiment of the application, the terminal can switch the image output mode based on the current environment information. The specific switching method will be described in detail in the following embodiments.

[0051] For example, the output modes include, but are not limited to, Binning output mode and HDR output mode. Among them, Binning mode is the default output mode of the camera sensor.

[0052] HDR output modes include, but are not limited to, the following:

[0053] 1. SHDR (stagger HDR) mode: The camera sensor outputs two frames of images. First, a long exposure frame is exposed to focus on capturing dark information, and then a short exposure frame is exposed to focus on capturing bright information. The two frames are simultaneously input into the chip platform's image processing module, and after processing by a certain algorithm, a single frame of image is generated.

[0054] 2. DCG (Dual Conversion Gain) Mode: Each pixel of the camera sensor can have its gain controlled individually. When working in DCG mode, only one exposure is performed, but it is read out twice. HCG (High Conversion Gain, also known as High CG) is used to capture dark information (also known as low brightness information, i.e., brightness less than or equal to a preset threshold), and LCG (Low Conversion Gain, also known as Low CG) is used to capture bright information (also known as high brightness information, i.e. brightness greater than or equal to a preset threshold). The two frames of HCG and LCG images are simultaneously input into the chip platform's image processing module, and after processing by a certain algorithm, a single frame of image is generated.

[0055] 3. IDCG (intra dual conversion gain) mode: The difference between IDCG and DCG modes is that DCG involves the camera sensor simultaneously inputting two frames of HCG and LCG images into the chip platform, which then fuses them into a single frame. In IDCG mode, HCG and LCG are fused into a single frame internally within the camera sensor before being input into the chip platform.

[0056] 4. IDCG comb+VS (very short) mode: VS refers to a special mode where the exposure time is very short. IDCG and VS follow the SHDR frame output method, first exposing a long exposure frame (IDCG, which is a fusion of HCG and LCG), and then exposing a short exposure frame (VS). This short exposure frame is used to capture dark area information. The two frames are simultaneously input into the chip platform's image processing module, and after processing by a certain algorithm, a single frame image is generated.

[0057] In this application embodiment, the terminal device can be a mobile terminal with shooting function, such as a mobile phone, tablet computer, wearable device, vehicle-mounted device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. Alternatively, it can be a professional shooting device such as a digital camera, SLR camera / mirrorless camera, action camera, gimbal camera, drone, etc. This application embodiment does not limit the specific type of terminal device.

[0058] Figure 1 This is a schematic diagram of an exemplary user interface. Please refer to... Figure 1 The user clicks the camera application. In response to the received user action, the camera application displays an image preview interface 101 on the display interface 100. The image preview interface 101 displays a preview image captured by the camera. This embodiment only illustrates the processing method of the preview image as an example. In other embodiments, the solution in this embodiment can also be applied to other shooting scenarios such as video recording, and this application does not limit it.

[0059] Figure 2 A schematic diagram of the structure of the electronic device 100 is shown. It should be understood that... Figure 2 The electronic device 100 shown is merely an example of an electronic device, and the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. Figure 2 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits. In this example, electronic device 100 is exemplified by a mobile phone.

[0060] Electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0061] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0062] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193.

[0063] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0064] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0065] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. In some embodiments, electronic device 100 may include one or N displays screens 194, where N is a positive integer greater than 1.

[0066] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0067] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0068] Camera 193 is used to capture still images or videos. In some embodiments, electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1. Cameras 193 are disposed on the front and / or back of electronic device 100. The front of electronic device 100 is the side with a display screen, and the back of electronic device 100 is the side opposite to the display screen. At least one camera (e.g., one) may be disposed on the front of electronic device 100, and at least one camera (e.g., two, three, etc.) may be disposed on the back of electronic device 100. Different shooting modes in electronic device 100 correspond to different camera combinations. For example, camera 1 is mounted on the front of electronic device, and cameras 2 and 3 are mounted on the back; the camera combination corresponding to shooting mode 1 includes cameras 2 and 3; shooting mode 2 corresponds to camera 1.

[0069] Image sensor components are primarily used for imaging. Specifically, image sensor components acquire images and expose image frames.

[0070] For example, the image sensor also outputs a start of frame delimiter (SOF) signal, which identifies the beginning of the current frame, and an end of frame delimiter (EOF) signal, which identifies the end of the current frame. For example, the frame time of each frame is the length between the SOF and the next SOF, including the effective frame length between the SOF and EOF and the non-exposure blank lines (vertical blank, vblank, also known as blank invalid line time, vertical blanking, or field blanking) in each frame. Specific correspondences will be discussed later. Figure 4b The explanation is as follows.

[0071] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0072] Figure 3 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0073] The layered architecture of the electronic device 100 divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided from top to bottom into the application layer, framework layer, hardware abstraction layer (HAL), and kernel layer. It is understandable that... Figure 3 This is just an example; that is, the layers in an electronic device are not limited to... Figure 3 The layers shown, for example, between the application framework layer and the HAL layer, may also include the Android runtime and system library layers.

[0074] The application layer can include a series of application packages.

[0075] like Figure 3 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0076] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions.

[0077] like Figure 3As shown, the application framework layer may include a window manager, content provider, view system, camera service, resource manager, notification manager, etc.

[0078] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0079] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0080] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0081] The camera service, also known as a webcam service, is not limited to this application. This service is used to invoke the camera (including the front camera and / or the rear camera) in response to the application's request.

[0082] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0083] The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short time without user interaction.

[0084] The HAL layer is used to abstract hardware and provide a virtual hardware platform for the operating system. The HAL layer may include, but is not limited to, decision modules, CAMX architecture, etc.

[0085] The decision-making module can be a multi-camera decision-making module. This module can determine the camera in the camera group that actually performs image acquisition based on scene information (including ambient brightness and dynamic range). This target camera can be a front-facing camera or a rear-facing camera in an electronic device.

[0086] In addition, the decision module can also determine the target image output mode that the target camera needs to use. The image output mode can be referred to the description above, and will not be repeated here. Alternatively, the method of selecting the target image output mode can be based on the shooting scene, shooting parameters (zoom ratio), etc. The implementation details can be referred to the description in the subsequent embodiments, and will not be repeated here.

[0087] The CAMX architecture is a logical layer within the HAL layer. It includes sensor nodes and image front-end (IFE) nodes. Each sensor node in the CAMX architecture corresponds one-to-one with a camera sensor in an electronic device. The sensor node can configure the camera sensor to enable a specified image output mode. For example, a sensor node can pass configuration parameters required to enable a target image output mode to the target camera, causing the target camera sensor to output images in that mode. The aforementioned IFE nodes are used to configure the preprocessing parameters required for IFE operation, enabling the IFE to preprocess the preview stream captured by the target camera according to the target image output mode. This includes performing color correction, downsampling, and depixelation on image frames in the preview and video streams.

[0088] The CAMX architecture also includes a CAMX conversion interface (camera serial interface decoder, CSL). This CAMX CSL receives configuration parameters from sensor nodes and translates them into I / O control commands recognizable by the kernel layer. Furthermore, the CAMX CSL can also transmit the I / O control commands corresponding to the camera parameters to the corresponding camera driver via V4L2 in the kernel layer. The camera driver then writes these I / O control commands into the camera sensor, enabling the camera sensor to output images according to the specified image output mode.

[0089] The aforementioned CAMX CSL receives preprocessing parameters from the IFE node and converts them into kernel-level recognizable I / O control instructions. Then, via V4L2, the I / O control instructions corresponding to the preprocessing parameters are passed to the corresponding ISP driver. The ISP driver then writes the I / O control instructions corresponding to the preprocessing parameters into the IFE, enabling the IFE to preprocess image frames acquired according to the target output mode.

[0090] The kernel layer includes at least the Linux video device driver (Video for Linux2, V4L2) and the camera driver.

[0091] The camera driver includes, but is not limited to: camera request manager (CRM), sensor driver, IFE driver, etc.

[0092] This camera driver can be used to drive hardware modules with shooting capabilities, such as camera sensors. In other words, the aforementioned camera driver is responsible for data interaction with the camera sensor. Of course, the kernel layer may also include driver software such as audio drivers and sensor drivers; this application embodiment does not impose any limitations on this.

[0093] The hardware includes a camera assembly (described above, and will not be repeated here) and a display screen. The camera assembly includes, but is not limited to, physical components such as IFE, sensors, and lenses.

[0094] Understandable Figure 3 The layers in the illustrated software structure and the components contained in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer layers than illustrated, and each layer may include more or fewer components; this application does not impose any limitations.

[0095] Figure 4a This is a schematic diagram illustrating module interaction as an example. Please refer to... Figure 4a Specifically, after the user clicks the camera application, the camera application sends a trigger command to the camera service to instruct it to be invoked. For example, the trigger command includes shooting mode information, etc. In this embodiment, after the camera application starts, it defaults to shooting mode, such as... Figure 4a As shown. Accordingly, the triggering command includes, but is not limited to, information about the shooting mode.

[0096] In response to the trigger command, the camera service sends a call command to the decision module to invoke the camera. This call command includes, but is not limited to, shooting mode information, such as the photo mode.

[0097] The decision-making module can determine the actual camera in the camera group that is capturing images based on scene information (including ambient brightness and dynamic range). This target camera can be a front-facing camera or a rear-facing camera in an electronic device.

[0098] The decision module can also determine the target image output mode that the target camera needs to use. In this embodiment, the decision module can first select the default image output mode, namely Binning mode. During subsequent shooting, the decision module can select the corresponding image output mode based on each image frame captured by the camera and other parameters, such as the shooting scene and shooting parameters (zoom ratio), to achieve image output mode switching.

[0099] Still refer to Figure 4a After the decision module identifies the target camera, it can send information such as the target image output mode to the IFE node and Sensor node corresponding to the target camera in the CAMX architecture, so as to instruct the Sensor node and IFE node to generate the corresponding image output mode configuration information based on the target image output mode.

[0100] For example, in response to the received target image output mode, the Sensor node configures the camera sensor to enable the specified image output mode. Specifically, the Sensor node generates Sensor configuration information. This Sensor configuration information includes, but is not limited to, image output mode configuration information corresponding to the target image output mode (also referred to as Sensor image output configuration information), which can be understood as the image output mode configuration information required to enable the target image output mode. Optionally, the Sensor configuration information also includes other camera parameters (also referred to as configuration parameters), such as second exposure information, frame length, second gain information, etc., which are not limited in this application. In some examples, the camera parameters and the Sensor image output configuration information can be sent at the same time or at different times. The camera parameters are used for the Sensor to expose the image frame; therefore, the camera parameters need to be sent before the image frame is exposed. The image output mode configuration information is used to output the image frame; therefore, the image output mode configuration parameters need to be sent before the Sensor outputs the image; which is not limited in this application.

[0101] Still refer to Figure 4a For example, in response to the instructions of the decision module, the IFE generates IFE configuration information (also known as IFE output configuration information). This IFE configuration information includes, but is not limited to, preprocessing parameters required for IFE operation, enabling the IFE to preprocess the preview stream captured by the target camera according to the target output mode. For example, performing color correction, downsampling, and depixelation on image frames of the preview stream and video stream.

[0102] In this embodiment, the configuration information of the IFE and the sensor must correspond. For example, when the sensor is operating in Binning output mode, the preprocessing parameters required for the IFE to run also correspond to Binning output mode. If the sensor is operating in other modes, such as HDR mode, and the preprocessing parameters required for the IFE to run are Binning output mode, then the IFE cannot process the image input from the sensor, and the IFE will discard the image frame.

[0103] Please continue to refer to Figure 4a For example, the IFE node sends its IFE configuration information to the conversion interface. The Sensor node sends its Sensor configuration information to the conversion interface. The conversion interface passes the camera parameters to the corresponding camera driver via V4L2 in the kernel layer. After receiving the camera parameters, the camera driver notifies the CRM that it has received the camera parameters. The CRM then controls the camera driver to write the camera parameters into the camera sensor, enabling the camera sensor to output images according to the specified image output mode.

[0104] Specifically, V4L2 sends the IFE configuration information to the IFE driver, and V4L2 sends the Sensor configuration information to the Sensor driver. After receiving the IFE configuration information, the IFE driver notifies the CRM that it has received the IFE configuration information (i.e., as shown in the image). Figure 4a As shown, feedback information is sent to the CRM (1). After receiving notification from the IFE driver that the Sensor's SOF interrupt signal has been received, the CRM controls the IFE driver to send the IFE configuration information to the IFE according to the IFE configuration information activation timing, so that the IFE can write the IFE configuration information into its register. After receiving the Sensor configuration information, the sensor driver notifies the CRM that it has received the Sensor configuration information (e.g., ...). Figure 4a 2) The Sensor driver sends feedback information to the CRM; After receiving notification from the IFE driver that it has received the Sensor's SOF interrupt signal, the CRM controls the Sensor driver to send the Sensor configuration information to the Sensor according to the Sensor configuration information activation timing, so that the Sensor can write the Sensor configuration information into the Sensor's register. The Sensor will acquire images according to the Sensor configuration information. For example, the Sensor will acquire images according to the camera parameters and image output mode configuration information in the Sensor configuration information. The Sensor outputs the acquired images to the IFE. The IFE can process the images based on the IFE configuration information. For example, the IFE can preprocess the images based on the preprocessing parameters corresponding to the target image output mode.

[0105] Figure 4b This is an exemplary image frame exposure timing diagram; please refer to it. Figure 4b Before frame N-1 (including frame N-1), the sensor uses mode A to output images for each image frame. In this example, in frame N-1, the sensor also uses mode A (also known as the first output mode) for image exposure. Before time T1-1, the decision module determines the target output mode and target camera for frame N+1 based on environmental information. The target output mode is the second output mode (e.g., mode B), meaning that the target output mode (i.e., the first output mode) for frame N+1 switches to the second output mode (mode B). Optionally, the decision module can make the decision at any time before frame N (i.e., time T3), such as at frame N-1 or frame N-2; this application does not limit this. The configuration information distribution time and writing time in this embodiment are only illustrative examples and can be set according to actual needs; this application does not limit this.

[0106] For example, in combination Figure 4aAfter the decision module identifies the target camera, it can send information such as the target image output mode to the IFE node and Sensor node corresponding to the target camera in the CAMX architecture, so as to instruct the Sensor node and IFE node to generate the corresponding image output mode configuration information based on the target image output mode.

[0107] For example, in response to the received target image mode, the Sensor node configures the camera sensor to enable the specified image mode. Specifically, the Sensor node generates N frames of Sensor configuration information. This Sensor configuration information includes, but is not limited to, image mode configuration information corresponding to the target image mode, which can also be understood as the image mode configuration parameters required to enable the target image mode. For example, the image mode configuration information includes the image mode configuration parameters required for the target image mode, indicating that the switching configuration information has been fully written into the effective information.

[0108] Optionally, the sensor configuration information may also include other camera parameters (also known as configuration parameters), such as frame length, which are not limited in this application.

[0109] Reference Figure 4a For example, the IFE node, in response to an instruction from the decision module, generates IFE configuration information. This IFE configuration information includes, but is not limited to, preprocessing parameters required for IFE operation, enabling the IFE to preprocess the preview stream captured by the target camera according to the target image output mode. For instance, this may involve color correction, downsampling, and depixelation processing of image frames in the preview and video streams.

[0110] Reference Figure 4b At time T1-1, the Sensor node sends the Nth frame's Sensor configuration information (including the image output mode configuration information corresponding to the second image output mode and camera parameters, etc.) to the Sensor driver, and the IFE node sends the Nth frame's IFE configuration information to the IFE driver. Optionally, as described above, the camera parameters and image output configuration information in the Sensor configuration information can be sent to the Sensor driver at different times. The Sensor driver sends feedback information 1 to the CRM to notify the CRM that the Sensor driver has received the Nth frame's Sensor configuration information. The IFE driver sends feedback information 2 to the CRM to notify the CRM that the IFE driver has received the Nth frame's IFE configuration information. The specific sending process can be referred to existing technical embodiments, and this application does not limit it.

[0111] In this embodiment, the sensor is pre-configured with activation point information. Based on this activation point information, the sensor can obtain sensor configuration information at a specified time within each frame (this time is called the activation point) and begin image exposure based on the sensor configuration information. This activation point is the starting point of the exposure duration of the image frame, and the EOF of the image frame is the ending point of the exposure duration. Optionally, in some examples, the activation point of the image output configuration information may be different from the activation point of the camera parameters. Specifically, as mentioned above, the camera parameters are mainly used by the sensor when exposing the image frame; therefore, the camera parameters take effect at the exposure start point, and the sensor can expose the image frame based on the camera parameters. The sensor image output configuration information is mainly used by the sensor to output the image frame (e.g., ...). Figure 4b The time intervals from T3 to T5 (within the frame length of N frames) are defined as follows. The image output time corresponds to the image frame length, for example... Figure 4b The time intervals T3 to T5 represent the frame length N. The sensor outputs images from N frames within this output time. Optionally, within the output time, the sensor actually performs the image output action (i.e., outputs the exposed image to the IFE) between SOF and EOF of the N frames. During other times within the N frames (e.g., T4 to T5), the sensor performs other configurations. Correspondingly, the sensor's image output configuration information takes effect before the image frames are output; that is, the sensor's image output configuration information only needs to take effect before the image frames are output.

[0112] Optionally, the effective point information is used to indicate the relative duration between the effective point and the SOF. For example, the effective point information indicates a relative duration of X. Accordingly, the sensor begins exposing the image frame X duration before the SOF of each frame.

[0113] For example, please refer to Figure 4b At time T2-1, the sensor determines the time corresponding to the effective point of N frames based on the effective point information. In this example, time T2-1 is the exposure start point of N frames. The sensor obtains the written sensor configuration information (this sensor configuration information is the configuration information corresponding to the first output mode) and starts exposing N frames based on the camera parameters in the sensor configuration information. The interval X between time T2-1 and the SOF of N frames (i.e., time T3) is a duration of X.

[0114] For example, at time T3, the sensor outputs the image for the Nth frame based on the sensor output configuration information. The actual output duration is from time T3 to T4, which is the SOF to EOF of the Nth frame. In this embodiment, the entire duration of the Nth frame (i.e., from time T3 to T4) is referred to as the output duration of the Nth frame.

[0115] For example, at this point of application, the sensor retrieves the written sensor configuration information. The current sensor configuration information indicates the first output mode, i.e., mode A. Accordingly, the sensor exposes and outputs N frames according to the camera parameters in the configuration information and the first output mode.

[0116] Please continue to refer to Figure 4b As mentioned above, the configuration information for the N+1th frame is written in the Nth frame and takes effect in the N+1th frame. Specifically, at time T3-1 within the Nth frame (generally any time between SOR and EOF, which can be set according to actual needs, and this application does not limit it), the CRM instructs the Sensor driver to send the Sensor configuration information for the N+1th frame to the Sensor (including camera parameters and the image output mode configuration information corresponding to the second image output mode). In addition, the CRM instructs the IFE driver to send the IFE configuration information for the Nth frame to the IFE (including the image output mode configuration information (which indicates the second image output mode)).

[0117] At time T3-1, the Sensor driver and IFE driver write the corresponding configuration information to the Sensor and IFE, respectively. It should be noted that there may be a delay between the two write operations, which is not limited in this application.

[0118] In this embodiment, as described above, the sensor's output mode configuration information may further include switching configuration information and effective configuration information. For example, the sensor driver writes the switching configuration information to the corresponding register of the sensor (denoted as the output mode register). After the sensor driver writes all the switching configuration information to the register, the sensor writes the effective configuration information to the corresponding register (denoted as the effective register). The writing methods for other configuration information (such as camera parameters) can refer to existing technologies, and this application does not limit them.

[0119] For example, the IFE driver writes the IFE configuration information of N frames into the register corresponding to the IFE.

[0120] Optionally, the number of registers written by the IFE is less than the number of registers that the sensor needs to write. Typically, the sensor's configuration information is quite long, requiring the writing of a large number of registers.

[0121] At time T4-1, the sensor determines the time corresponding to the activation point of frame N+1 based on the activation point information. In this example, time T4-1 is the exposure time of frame N+1. The sensor obtains the written sensor configuration information (i.e., the configuration information of frame N written at T3-1) and starts exposing frame N+1 based on the sensor configuration information. The interval X between time T4-1 and the SOF of frame N (i.e., time T3) is a duration of X.

[0122] For example, at time T5, the sensor outputs an image for frame N+1 based on the sensor output configuration information. The actual output duration is from time T5 to T6, which is the SOF to EOF of frame N+1. In this embodiment, the entire duration of frame N+1 (i.e., from time T5 to T7) is referred to as the output duration of frame N+1.

[0123] For example, at the activation point, the sensor retrieves the configuration of the activation register. In one example, if the sensor has completely written the output mode configuration information for N frames before T4-1, that is, if the sensor has written all the switching configuration information to the corresponding output mode register and the activation configuration information to the corresponding activation register, then the sensor determines to activate the configuration of the output mode register corresponding to that activation register based on the configuration of the activation register. In other words, the sensor outputs the N+1 frames based on the output mode configuration information of N frames (i.e., mode B, also known as the second output mode).

[0124] In another example, if the sensor fails to completely write the output mode configuration information for frame N before T4-1, that is, the sensor fails to write the effective configuration information to the corresponding effective register, then the sensor will not apply the configuration information being written, but will instead apply the old configuration information.

[0125] For example, suppose the sensor only fully writes the N-frame output mode configuration information after T4-1. At T4-1, the sensor cannot obtain the new configuration information, only the configuration information of the first output mode. This causes the sensor to expose according to the first output mode configuration. Simultaneously, the IFE (In-Frame Exposure) applies the new output mode configuration information at this time. The sensor exposes the N+1th frame according to the first output mode configuration and outputs the exposed image to the IFE. Correspondingly, the IFE detects that its output mode (i.e., the second output mode, mode B) does not match the output mode used by the sensor (i.e., the first output mode, mode A). The IFE will discard the N+1th frame. Consequently, the display will show the old image frame, i.e., frame N. In other words, within the time it should have shown N+1 frames, the display shows N frames. From the user's perspective, this will result in a noticeable stuttering or lag.

[0126] To address the issue of frame loss caused by the sensor taking a long time to write new configuration information, in one embodiment, the sensor is configured to write the sensor configuration information of frame N in frame N-2 and make the configuration information of frame N effective in frame N (i.e., write the sensor configuration information two frames in advance). The IFE (Integrated Function Exchange) remains the same: it writes the IFE configuration information of frame N in frame N-1 and makes the IFE configuration information of frame N effective in frame N (i.e., write the IFE configuration information one frame in advance). Accordingly, the CRM performs data synchronization based on the effective timing of IFE in frame N+1 and the effective timing of the sensor in frame N+2. That is, the CRM controls the sensor driver to send the sensor configuration information of frame N to the sensor in frame N-2, causing the sensor to write the sensor configuration information of frame N in frame N-2 and make the sensor configuration information effective in frame N. Furthermore, the CRM controls the IFE driver to send the IFE configuration information for frame N to the IFE in frame N-1, causing the IFE to write the IFE configuration information for frame N in frame N-1 and make the IFE configuration information effective in frame N. Thus, in a scenario where the sensor writes the sensor configuration information for frame N in frame N-2, the sensor may complete writing the sensor configuration information before the effective point in frame N-2, or it may complete writing the sensor configuration information before the effective point in frame N-1. This application, by writing the effective sensor configuration information in frame N-2, can ensure that the IFE configuration information and the sensor configuration information are synchronized in frame N, that is, it can correctly switch to the second output mode in frame N.

[0127] The following is combined with Figure 5c Please refer to the following for a detailed explanation of the process of switching the image display mode at frame N+2. Figure 5cSpecifically, at time T-1 (which can be any time before time T3, this application does not limit this), the decision module determines, based on the scene environment information, to switch the output mode from the first output mode to the second output mode. Combined with... Figure 5a The decision module indicates the second output mode to the IFE node and Sensor node to trigger the IFE node and Sensor node to generate the corresponding configuration information.

[0128] Please refer to Figure 5c In response to the instruction from the decision module, the Sensor node generates Sensor configuration information for the (N+2)th frame. This Sensor configuration information includes, but is not limited to, camera parameters and switching configuration information within the image output mode configuration information. Camera parameters include, but are not limited to, the frame length of the (N+2)th frame (e.g., the first frame length, which is the default frame length and is typically set to 33ms). The Sensor image output mode configuration information includes, but is not limited to, switching configuration information and activation configuration information. The switching configuration information includes configuration parameters corresponding to the second image output mode.

[0129] In response to the instructions of the decision module, the IFE node generates IFE configuration information for the N+2th frame. The IFE configuration information includes, but is not limited to, preprocessing parameters corresponding to the second image output mode, enabling the IFE to preprocess the preview stream captured by the target camera according to the target image output mode (i.e., the second image output mode).

[0130] Please refer to Figure 5a The Sensor node sends the Sensor configuration information (including camera parameters and switching configuration information) of frame N+2 to the Sensor driver via the conversion interface and V4L2. The IFE node sends the IFE configuration information of frame N+2 to the IFE driver via the conversion interface and V4L2. After receiving the IFE configuration information, the IFE driver notifies the CRM that it has received the IFE configuration information (i.e., sends feedback information 4 to the CRM). After receiving the Sensor configuration information, the sensor driver notifies the CRM that it has received the Sensor configuration information (i.e., sends feedback information 5 to the CRM). Accordingly, combined with... Figure 5c As shown, at time T-1, the Sensor node sends the Sensor configuration information of the N+2th frame to the Sensor driver, and the IFE node sends the IFE configuration information of the N+2th frame to the IFE driver.

[0131] Still refer to Figure 5c At time T2-1, the sensor exposes the Nth frame, and at time T3, it outputs the image of the Nth frame. The exposure of the Nth frame uses the sensor configuration information for the Nth frame written at time N-1. For example, the frame length of the Nth frame is the same as the first frame length, and the output mode is the first output mode (i.e., Mode A).

[0132] For example, as described above, CRM synchronizes data based on the N+1 frame activation sequence of the IFE and the N+2 frame activation sequence of the Sensor, indicating the timing of the switching configuration information and activation configuration information issued by the Sensor driver and the IFE configuration information issued by the IFE driver, so as to achieve data synchronization between the IFE configuration information and the Sensor configuration information in the N+2 frame. (Referring to...) Figure 5c and Figure 5a At time T3-1 (which can be any time within N frames, and this application does not limit it), the CRM instructs the Sensor driver to send the Sensor configuration information for the Nth frame to the Sensor (i.e., Figure 5a The CRM sends instruction information 3 to the Sensor driver. This Sensor configuration information includes switching configuration information and camera parameters (such as frame length), but does not include effective configuration information. At time T3-1 (which can be any time from after the SOF of the Nth frame to before the effective point of the N+1th frame, this application is not limited), the Sensor driver responds to instruction information 3 and writes the Sensor configuration information of the N+2th frame into the corresponding register of the Sensor. For example, the Sensor configuration information includes, but is not limited to, camera parameters and switching configuration information (the Sensor configuration information currently written to the Sensor does not include effective configuration information). Camera parameters include, but are not limited to, the frame length of the Nth frame (e.g., the first frame length).

[0133] Reference Figure 5b At time T3-1, the Sensor node can send the Sensor activation configuration information (also referred to as activation configuration information) of frame N+2 to the Sensor driver through the conversion interface and V4L2. Upon receiving the activation configuration information of frame N+2, the Sensor driver sends feedback information 5 to the CRM to notify the CRM that the Sensor driver has received the activation configuration information of frame N+2. In other words, the Sensor node generates the Sensor configuration information of frame N+2 in two steps.

[0134] At time T4-1, the sensor, based on the activation point configuration information (e.g., the SOF distance from the activation point to N frames is X seconds), determines that this time point is the activation point of frame N+1, which in this example is the exposure start point of frame N+1. Since the activation configuration information for frame N+2 has not yet been written to the corresponding register, meaning the sensor has not obtained the activation configuration information for frame N+2, the sensor still outputs the image according to the old configuration information. That is, the output mode for frame N+1 is the first output mode, i.e., mode A.

[0135] Reference Figure 5cWhen the IFE driver receives the SOF interrupt signal of the sensor in frame N+1, it notifies the CRM that it has detected the SOF interrupt signal of the sensor in frame N+1. In response to the IFE driver's notification, the CRM instructs the sensor driver to send the effective configuration information for frame N+2 to the sensor (e.g., ...). Figure 5b The CRM sends instruction information to the sensor driver (5). For example... Figure 5c As shown, at time T5-1 (which can be any time before the effective point of frame N+2, i.e., time T6-1, this application does not limit this), the Sensor driver responds to indication information 5 and writes the effective configuration information of frame N+2 into the Sensor's effective register.

[0136] For example, when the IFE driver receives the SOF interrupt signal of the sensor in frame N+1, it notifies the CRM that it has detected the SOF interrupt signal of the sensor in frame N+1. In response to the notification from the IFE driver, the CRM instructs the IFE driver to send the IFE configuration information of frame N+2 (such as...) to the IFE. Figure 5b The CRM sends instruction information to the IFE driver (4). For example... Figure 5c As shown, at time T5-1 (which can be time T6-1, i.e., any time before the effective point of the N+2th frame, this application does not limit it), the IFE driver responds to the indication information 4 and writes the IFE configuration information of the N+2th frame into the IFE register.

[0137] Optionally, the timing of the IFE configuration information sent by the IFE driver in frame N+2 and the timing of the effective configuration information sent by the Sensor driver in frame N+2 can be different, and this application does not impose any restrictions.

[0138] At time T6-1, the sensor, based on the activation point configuration information (e.g., the activation point is X seconds from the SOF of frame N+1), determines that this time point is the activation point of frame N+2, which in this example is the exposure start point of frame N+2. As mentioned above, the sensor has already written the activation configuration information before this time. Accordingly, at this activation point, the sensor can perform image output based on the already written configuration information of frame N+2 (including the second output mode and the first frame length, etc.), that is, perform exposure and image output for frame N+2. In other words, the output mode of frame N+2 is the second output mode, i.e., mode B.

[0139] For example, the sensor outputs the (N+2)th frame to the IFE. As mentioned above, the IFE takes effect at frame N+1. Therefore, the IFE writes the IFE configuration information for frame N+2 at frame N+1, and this IFE configuration information for frame N+2 will take effect at frame N+2. Accordingly, the IFE processes frame N+2 based on the IFE configuration information for frame N+2. For example, the IFE performs image processing on frame N+2 based on the preprocessing parameters corresponding to the second output mode.

[0140] The IFE outputs the processed image to the camera application, which then displays the (N+2)th frame. Due to the switching of the output mode, the display effect of the (N+2)th frame differs from that of the (N+1), (Nth), and other image frames.

[0141] Figure 5d This is a timing diagram illustrating the segmented loading of Sensor configuration information described above. Specifically, referring to 5d, the output mode of the N-3rd frame image is the first output mode (i.e., mode A). The decision module determines, based on the scene environment information, to switch the output mode of the Nth frame image from the first output mode to the second output mode, and instructs the Sensor node and IFE node on the second output mode (i.e., mode A). Figure 5d The mode switching command shown triggers the Sensor node and IFE node to generate corresponding configuration information. The Sensor node can generate Sensor configuration information in frame N-3, which includes the switching configuration information in the Sensor image output configuration information and camera parameters such as... Figure 5d As shown. The sensor node can generate the handover configuration information for frame N in frame N-3. Optionally, the handover configuration information includes: a command start flag (such as...). Figure 5d The configuration information includes: the Group on flag, Seamless control parameter 1, and the first part of the mode parameters. The effective configuration information includes: the second part of the mode parameters, VC switching, first frame AE parameters, Seamless control parameter 2, and the effective flag (such as...). Figure 5d The Group off flag in the configuration sequence can be used to divide the Seamless configuration into two parts, resulting in Seamless control parameter 1 and Seamless control parameter 2. Similarly, the mode parameters in the configuration sequence can be divided into two parts, serving as the first and second parts of the mode parameters. It is understood that the conditions for dividing the Sensor configuration information include: the switching configuration information must include at least a command start flag and some parameter information; the effective configuration information must include at least an effective flag. In other examples, the division of Sensor configuration information only needs to meet the conditions for dividing Sensor configuration information. For example, the switching configuration information may include: a command start flag (such as...) Figure 5dThe configuration includes: Group on flag, Seamless control parameter 1, Mode parameter first part, VC switching, and first frame AE parameter; Effective configuration information includes: Mode parameter second part, Seamless control parameter 2, and Effective flag. The VC switching, first frame AE parameter, Seamless control parameter, and Mode parameter can be found in existing Sensor configuration information and will not be elaborated upon here.

[0142] The IFE node generates IFE configuration information, which includes, but is not limited to, preprocessing parameters corresponding to the second output mode, enabling the IFE to preprocess the preview stream captured by the target camera according to the target output mode (i.e., the second output mode).

[0143] Continue to refer to Figure 5d Assume that the Sensor node sends the Nth frame request to the Sensor driver within the frame length of the N-3th frame (i.e., the Sensor node sends the Nth frame request to the Sensor driver through the conversion interface and V4L2). The Nth frame request includes the Sensor switching configuration information for the Nth frame. After receiving the Sensor switching configuration information for the Nth frame, the Sensor driver sends a first notification to the CRM. The first notification is used to indicate that the Sensor driver has received the notification of the Nth frame request.

[0144] When the IFE driver receives the SOF interrupt signal of the Sensor in frame N-3, it notifies the CRM that the Sensor has detected the SOF interrupt signal in frame N-3. Responding to the IFE driver's notification, the CRM, based on the Sensor's activation timing information, instructs the Sensor driver to retrieve the next frame request (e.g., frame N-1 in this example, which includes the Sensor configuration information for frame N-1) from the processing queue and send it to the Sensor. In this example, within the frame length of frame N-3, the Sensor driver, responding to the CRM's instruction, retrieves the Sensor configuration information for frame N-1 (i.e., mode A's configuration information) from the processing queue and sends it to the Sensor. The Sensor then writes the Sensor configuration information for frame N-1 into its register. Understandably, the speed at which the sensor driver sends information differs from the speed at which the sensor node sends information. Therefore, there will be a delay when the sensor driver sends a request to the sensor compared to when the sensor node sends a request to the sensor driver. For example, ... Figure 5d As shown, the Sensor node sends the Nth frame request to the Sensor driver. After receiving the Nth frame request, the Sensor driver sends the N-1th frame request within the frame length of the N-3th frame.

[0145] It should be noted that after receiving the sensor configuration information from the sensor node, the sensor driver can cache the received configuration information in a pending queue. When the sensor receives a notification (or instruction) from the CRM, the sensor driver retrieves the information to be sent from this pending queue according to the processing order.

[0146] Continue to refer to Figure 5d Within the frame length of frame N-2, the Sensor node generates the Sensor activation configuration information for frame N. The Sensor node uses the Sensor activation configuration information for frame N as a request for frame N+1. The Sensor node sends the frame N+1 request to the Sensor driver. Upon receiving the frame N+1 request, the Sensor driver notifies the CRM that it has received the frame N+1 request.

[0147] When the IFE driver receives the SOF interrupt signal of the sensor in frame N-2, it notifies the CRM that the sensor has detected the SOF interrupt signal in frame N-2. In response to the notification from the IFE driver, the CRM, based on the sensor's activation timing information, instructs the sensor driver to retrieve the sensor switching configuration information for frame N from the processing queue and send it to the sensor. The sensor then writes the sensor switching configuration information for frame N into the sensor's register.

[0148] Within the frame length of N-1 frames, the decision module determines that the image output mode will not change in the N+1 frame. The Sensor node generates the Sensor configuration information for the N+1 frame and uses it as a request for the N+2 frame (since the image output mode remains unchanged, the Sensor configuration information for the N+1 frame includes camera parameters). When the Sensor driver receives the N+2 frame request, it notifies the CRM that the Sensor driver has received the N+2 frame request.

[0149] When the IFE driver receives the SOF interrupt signal of the sensor in frame N-1, it notifies the CRM that the sensor has detected the SOF interrupt signal in frame N-1. In response to the notification from the IFE driver, the CRM, based on the sensor's activation timing information, instructs the sensor driver to retrieve the request for frame N+1 (i.e., including the sensor activation configuration information for frame N) from the processing queue and send it to the sensor. The sensor writes the sensor activation configuration information for frame N into the sensor's register. This sensor activation configuration information for frame N takes effect at the activation point of frame N (the activation point is X time intervals from the SOF of frame N), and exposure is performed at the activation point according to the sensor configuration information for frame N. The image is then plotted in mode B at the SOF time of frame N.

[0150] Within frame N, if the decision module detects that the image output mode remains unchanged (referred to as normal mode), the Sensor node sends a frame (N+3) request to the Sensor driver. This frame (N+3) request includes the Sensor configuration information from frame N+2 (since the image output mode remains unchanged, the Sensor configuration information in frame N+2 includes camera parameters). Upon receiving this frame (N+3) request, the Sensor driver notifies the CRM that it has received the frame (N+3) request.

[0151] When the IFE driver receives the SOF interrupt signal for the Nth frame of the sensor, it notifies the CRM that the SOF interrupt signal for the Nth frame of the sensor has been detected. In response to the notification from the IFE driver, the CRM, based on the sensor's activation timing information, instructs the sensor driver to retrieve the N+2th frame request (i.e., including the sensor configuration information for the N+1th frame) from the processing queue and send it to the sensor. The sensor then writes the N+2th frame request (i.e., the sensor configuration information for the N+1th frame) into the sensor's register.

[0152] As can be seen, the sensor writes the switching configuration information of frame N in frame N-2 and the activation configuration information of frame N in frame N-1, so that the sensor's activation point in frame N is exposed according to the sensor output mode configuration information of frame N, and the image is output according to mode B in frame N.

[0153] In some embodiments, the configuration information for the Nth frame of the Sensor needs to be sent to the Sensor two frames in advance, while the configuration information for the Nth frame of the IFE needs to be sent to the Sensor one frame in advance. In scenarios where the performance of the electronic device is insufficient, the IFE driver may delay sending the IFE configuration information to the IFE (e.g., delaying the sending by one frame), which will cause the IFE's output mode to mismatch with the Sensor's output mode. The Sensor and IFE will discard the mismatched current frame and the next frame, and the CRM will instruct the Sensor driver and the IFE driver to perform a rollback operation. This scenario is called the Bubble scenario.

[0154] The sensor driver and IFE driver will fall back to the frame before the IFE driver experienced a delay, for example, Figure 6As shown, the IFE driver rolls back to the frame before the IFE was issued. When the frame rolled back by the sensor is the frame that issued the sensor activation configuration information, it will cause the sensor driver to issue the same sensor activation configuration information twice. In other words, the sensor driver's rollback operation causes the sensor driver to issue the same sensor activation configuration information twice, resulting in one sensor switching configuration information and two sensor activation configuration information in the same frame. That is, the sensor configuration information in the frame is not paired (paired sensor configuration information usually means that one sensor switching configuration information and one corresponding sensor activation configuration information in the same frame are paired), which leads to abnormal sensor output and frame loss.

[0155] The following is combined with Figure 6 This section explains in detail the process by which the Sensor driver repeatedly sends the same frame twice to activate the Sensor configuration information.

[0156] Figure 6 The process of the IFE node sending IFE configuration information for each frame to the IFE driver is not shown, nor is the process of the Sensor node sending Sensor configuration information for each frame to the Sensor driver. For details, please refer to [link to relevant documentation]. Figure 5a and Figure 5b The corresponding textual descriptions will not be repeated here.

[0157] Please refer to Figure 6 Assume that in the (N-2)th frame, the Sensor driver sends the Sensor request 0 (referred to as Sensor R0) to the Sensor. Sensor R0 includes the Sensor switching configuration information of the Nth frame. The Sensor writes the Sensor switching configuration information of the Nth frame into the Sensor register.

[0158] The sensor driver sends request 1 (Sensor R1) to the sensor in frame N-1. Sensor R1 includes the sensor activation configuration information for frame N. The IFE driver sends IFE Request 0 (IFE R0) to the IFE in frame N-1. IFE R0 includes the IFE configuration information for frame N. The sensor configuration information for frame N (including sensor switching configuration information and sensor activation configuration information) takes effect in frame N, and the IFE configuration information for frame N also takes effect in frame N. In other words, in frame N, the sensor exposes according to the sensor configuration information for frame N (including sensor R0 and sensor R1) and outputs the exposed image to the IFE. The IFE processes the image frame according to the IFE configuration information for frame N (i.e., IFE R0). At this time, the IFE's output mode matches the sensor's output mode.

[0159] The decision module determines that the image output mode for frame N+1 remains unchanged. The Sensor node sends the Sensor configuration information for frame N+1 to the Sensor driver. This Sensor configuration information includes camera parameters. The IFE node sends the IFE configuration information for frame N+1 to the IFE driver. Since the image output mode for frame N+1 remains unchanged, the Sensor configuration information and IFE configuration information for frame N+1 are written in frame N and take effect in frame N+1.

[0160] Continue to refer to Figure 6 Within frame N, the Sensor driver sends Sensor Request2 (referred to as Sensor R2) to the Sensor. Sensor R2 includes the Sensor configuration information from frame N+1. However, due to insufficient performance of the electronic device, the IFE driver does not send IFE Request1 (referred to as IFE R1) to the IFE normally. Instead, it delays sending IFE R1 to the IFE until frame N+1. This IFE R1 includes the IFE configuration information from frame N+1. Since the IFE configuration information takes effect with a one-frame delay, that is, the IFE writes IFE R1 into the register in frame N+1, and the IFE configuration information from frame N+1 only takes effect in frame N+2.

[0161] The IFE still takes effect according to the IFE configuration information of frame N at the point of activation in frame N+1, while the Sensor takes effect according to the Sensor configuration information of frame N+1. The Sensor exposes according to the Sensor configuration information of frame N+1 and transmits the exposed image to the IFE. At this time, the IFE processes the image output by the Sensor according to the IFE configuration information of frame N, which causes the IFE configuration information to mismatch with the Sensor configuration information, triggering the IFE driver and the Sensor driver to perform a rollback operation.

[0162] Reference Figure 6 The Sensor driver and IFE driver will fall back to the frame before the IFE driver delayed sending the request. The specific process is as follows:

[0163] 1. The IFE driver discards the current frame (e.g.) Figure 6 The (N+1)th frame in the image and the next frame (e.g., the next frame) Figure 6 The content of the (N+2)th frame in the document;

[0164] 2. In the N+1th frame, the Sensor driver rolls back to the frame before the IFE driver sends the R1 of the IFE. That is, the Sensor driver resends the R1 of the Sensor in the N+1th frame, and the IFE driver resends the R0 of the IFE.

[0165] 3. In the N+2th frame, the IFE driver reissues the IFE's R1.

[0166] 4. In the N+2th frame, the Sensor driver reissues the Sensor's R2.

[0167] Since Sensor R1 and R0 need to be paired, the Sensor driver reissued Sensor R1 after rollback, resulting in two R1s that could not match R0, causing the Sensor to malfunction and fail to generate images correctly.

[0168] This application provides a method for setting image capture parameters. Sensor configuration information for frame N is written in frame N-2, and this frame N configuration information is applied in frame N. The frame N sensor configuration information includes image output mode configuration information and camera parameters. The image output mode configuration information includes switching configuration information, effective configuration information, and a first data packet. The first data packet includes, but is not limited to, first exposure information and first gain information. The first exposure information is different from the second exposure information, and the first gain information is different from the second gain information. The sensor driver stores the frame N sensor configuration information (including switching configuration information, effective configuration information, the first data packet, and camera parameters). The sensor driver first sends the sensor switching configuration information to the sensor. When the sensor driver receives a CRM instruction to send the sensor effective configuration information, it checks whether the frame N sensor effective configuration information has already been sent. If it detects that the frame N sensor effective configuration information has already been sent, the sensor driver sends the first data packet to the sensor. When it is detected that the Sensor effective configuration information for frame N has not been sent, the Sensor driver sends the Sensor effective configuration information for frame N to the Sensor.

[0169] In this example, when the sensor driver receives an instruction to send the Nth frame of sensor activation configuration information to the sensor, it checks whether the Nth frame of sensor activation configuration information has already been sent. If the Nth frame of sensor activation configuration information has already been sent, the sensor driver sends the first data packet to the sensor, avoiding the repeated sending of the Nth frame of sensor activation configuration information to the sensor. This avoids the problem of mismatch between the Nth frame of sensor switching configuration information and the sensor activation configuration information, solves the problem of image frame dropping when the electronic device has insufficient performance and a bubble problem occurs, and improves the image output quality.

[0170] Figure 7 A flowchart illustrating a method for setting image capture parameters provided in an embodiment of this application. In this example, the sensor configuration information of the Nth frame is used, where N is an integer greater than 0. The method for setting image capture parameters includes the following steps:

[0171] Step 701: The Sensor node generates the Sensor configuration information for the Nth frame.

[0172] In one example, the Sensor is configured to write the Sensor configuration information for frame N in frame N-2, and the Sensor configuration information for frame N takes effect in frame N. The IFE is still configured to write the IFE configuration information for frame N in frame N-1, and the IFE configuration information takes effect in frame N. When the decision module decides to switch from the first plotting mode to the second plotting mode based on the scene environment information, it indicates the second plotting mode to the Sensor node and the IFE node (i.e., sends a request to switch plotting modes) to trigger the IFE node and the Sensor node to generate the corresponding configuration information.

[0173] The sensor node can generate switching configuration information, effective configuration information, and the first data packet from the sensor configuration information in two consecutive frames. For example, in this example, the sensor node can generate the switching configuration information from the sensor configuration information of frame N in frame N-3. The sensor node generates the first data packet of frame N and the effective configuration information from the sensor configuration information of frame N in frame N-2. The first data packet includes the first exposure information (such as exposure time) and the first gain information (i.e., ISO information) of frame N, and camera parameters include frame length and other information. The IFE configuration information includes, but is not limited to, the preprocessing parameters corresponding to the second output mode, so that the IFE can preprocess the preview stream acquired by the target camera according to the target output mode (i.e., the second output mode).

[0174] In another example, when the decision module determines that the sensor rendering mode remains unchanged, there is no need to send rendering mode information. The sensor node generates sensor configuration information for frame N. Since the decision module has not switched the rendering mode, this sensor configuration information for frame N includes camera parameters such as frame length, but does not include sensor rendering configuration information. In this case, the sensor configuration information for frame N takes effect in frame N+1.

[0175] Step 702: The Sensor node sends the Nth frame of Sensor configuration information to the Sensor driver.

[0176] In one example, when the decision module determines to switch the output mode, the Sensor node sends a frame N request to the Sensor driver in frame N-3. The frame N request includes the Sensor switching configuration information for frame N (see reference). Figure 5d The sensor node sends a request for the N+1th frame to the sensor driver in the N-2th frame. The N+1th request includes the sensor activation configuration information for the Nth frame and the first data packet for the Nth frame. The first data packet for the Nth frame includes the first exposure time and the first gain information for the Nth frame.

[0177] It should be noted that a Sensor node can send frame N and frame N-1 requests to the Sensor driver simultaneously, or it can send them at different times (before frame N-2). This example demonstrates how a Sensor node can send frame N and frame N-1 requests to the Sensor driver at different times.

[0178] Still refer to Figure 5d After receiving the Nth frame request from the Sensor node in frame N-3, the Sensor driver stores the Nth frame request in the processing queue and notifies the CRM that the Sensor driver has received the Nth frame request. In response to this notification (i.e., the first notification), the CRM instructs the Sensor driver to issue the N-1th frame request based on the Sensor's activation information. This N-1th frame request includes the Sensor configuration information from frame N-2. The Sensor driver issues the N-1th frame request to the Sensor within frame N-3. The Sensor then writes the N-1th frame request (i.e., the Sensor configuration information from frame N-2) into its register within frame N-3 and activates the N-2th frame Sensor configuration information in frame N-2.

[0179] Furthermore, the IFE node can send the IFE configuration information for frame N to the IFE driver before frame N-1 (e.g., frame N-2). Upon receiving the IFE configuration information for frame N, the IFE driver will also notify the CRM that the IFE has received the IFE configuration information for frame N. Since the IFE is configured to write the IFE configuration information for frame N in frame N-1, the IFE driver sends the IFE configuration information for frame N within the frame length of frame N, and the IFE writes the IFE configuration information for frame N into its register.

[0180] In another example, if the decision module determines that the sensor's output mode remains unchanged, the sensor node sends a frame N request to the sensor driver in frame N-1. The frame N request includes the sensor's configuration information for frame N (this configuration information includes camera parameters but does not include the sensor's output mode configuration information). The sensor node sends the frame N request to the sensor driver in frame N-1.

[0181] In this example, we take the decision module determining the change in the sensor output mode as an example.

[0182] Step 703: The Sensor driver sends Sensor switching configuration information to the Sensor in frame N-2, so that the Sensor can write the switching configuration information into its register.

[0183] For example, when the IFE driver receives the SOF interrupt signal of the sensor in frame N-2, it notifies the CRM to trigger the CRM to instruct the sensor driver to send the sensor switching configuration information of frame N to the sensor. According to the CRM's instruction (i.e., the first instruction information), the sensor driver sends the sensor switching configuration information of frame N to the sensor within the frame length of frame N-2, so that the sensor writes the sensor switching configuration information of frame N into its register.

[0184] Step 704: The sensor driver determines whether the sensor activation configuration information for frame N has been sent before the request for frame N+1. If the sensor driver detects that the sensor activation configuration information for frame N has been sent before the request for frame N+1, step 705 is executed, that is, the sensor driver sends the first data packet to the sensor so that the sensor can write the first data packet into its register. If the sensor driver detects that the sensor activation configuration information for frame N has not been sent before the request for frame N+1, step 706 is executed, that is, the sensor driver sends the sensor activation configuration information for frame N to the sensor so that the sensor can write the sensor activation configuration information for frame N into its register.

[0185] In one example, the sensor driver can store a first identifier corresponding to the sensor configuration information of each frame (referred to as the first identifier of each frame). The first identifier of each frame is used to record whether the sensor driver has sent the sensor activation configuration information for that frame to the sensor. Optionally, the first identifier of each frame includes a first state value and a second state value. When the first identifier of the frame is the first state value (e.g., "true"), it indicates that the sensor driver has sent the sensor activation configuration information for that frame to the sensor. When the first identifier of the frame is the second state value (e.g., "false"), it indicates that the sensor driver has not sent the sensor activation configuration information for that frame to the sensor.

[0186] For example, when the IFE driver receives the SOF interrupt signal of the sensor's N-1 frame, it notifies the CRM to trigger the CRM to instruct the sensor driver to determine whether the sensor activation configuration information of the Nth frame has been sent before receiving the N+1 frame request. The sensor driver detects whether the first identifier of the N+1 frame is a first state value. When the first identifier of the N+1 frame is detected to be a first state value, it determines that the sensor activation configuration information of the Nth frame has been sent before the sensor driver received the N+1 frame request. When the first identifier of the N+1 frame is detected to be a second state value, it determines that the sensor activation configuration information of the Nth frame has not been sent before the sensor driver received the N+1 frame request.

[0187] When the Sensor driver determines that the Sensor activation configuration information for frame N has been sent before the Sensor driver receives the request for frame N+1, step 705 is executed, that is, the first data packet of frame N is sent to the Sensor. When the Sensor driver determines that the Sensor activation configuration information for frame N has not been sent before the Sensor driver receives the request for frame N+1, step 706 is executed, that is, the Sensor activation configuration information for frame N is sent to the Sensor.

[0188] Step 705: The Sensor driver sends the first data packet to the Sensor so that the Sensor can write the first data packet into its register.

[0189] When the sensor driver detects that the sensor activation configuration information for frame N has already been sent before the sensor driver receives the request for frame N+1, in order to avoid duplicate sending, the sensor driver sends the first data packet of frame N to the sensor. This first data packet of frame N includes the new exposure time and gain information for frame N, so that the sensor can expose frame N according to the new time and the new gain information.

[0190] Step 706: The Sensor driver sends the Sensor activation configuration information for the Nth frame to the Sensor so that the Sensor can write the Sensor activation configuration information for the Nth frame into its register.

[0191] When the sensor driver determines that the sensor activation configuration information for frame N has not been sent before the request for frame N+1, the sensor driver sends the sensor activation configuration information for frame N to the sensor. The sensor then writes the sensor activation configuration information for frame N into the frame N-1. The sensor performs exposure at the activation point of frame N according to the sensor configuration information for frame N, and outputs the image at the SOF time of frame N according to the second image output mode.

[0192] In this example, when insufficient electronic device performance causes the IFE driver to delay sending IFE configuration information, the Sensor driver will perform a rollback operation. This may result in the Sensor driver repeatedly sending the same Sensor effective configuration information for the same frame. When the Sensor driver receives the CRM's instruction to send Sensor effective configuration information, it determines whether the Sensor effective configuration information has already been sent. If the Sensor effective configuration information has already been sent, the first data packet is sent to the Sensor, thus avoiding the problem of the Sensor driver repeatedly sending the Sensor effective configuration information and ensuring that the Sensor can expose the data frame at the correct time.

[0193] Figures 8a-8b This is an interactive diagram illustrating the method for setting image capture parameters on an electronic device. In this example, the method is explained in detail by showing the Sensor node sending a request for the Nth frame to the Sensor driver in frame N-3.

[0194] Step 801: The sensor node generates the sensor switching configuration information for the Nth frame.

[0195] For example, when the decision module determines that the output mode will change in frame N, it instructs the Sensor node and the IFE node to generate their respective configuration information. Upon receiving the instruction from the decision module, the Sensor node generates the Sensor switching configuration information (also called switching configuration information) for frame N, the Sensor activation configuration information for frame N, and the first data packet for frame N in two consecutive frames. In this example, the Sensor node generates the Sensor switching configuration information in frame N-3.

[0196] It is understandable that the camera parameters for frame N can be generated in frame N-3 and sent to the sensor driver along with the switching configuration information for frame N. The sending and writing of the camera parameters for frame N to the sensor register can be referenced... Figure 4a The description in the text will not be repeated here.

[0197] Step 802: The sensor node sends the Nth frame request to the sensor driver.

[0198] Specifically, the sensor node generates a frame N request based on the sensor switching configuration information in frame N. This frame N request includes the sensor switching configuration information in frame N. The sensor node then sends the frame N request to the sensor driver.

[0199] Step 803: The sensor driver receives the sensor switching configuration information of the Nth frame.

[0200] Step 804: When the sensor driver receives the sensor switching configuration information in the Nth frame, it sends the first feedback information to the CRM.

[0201] Specifically, when the sensor driver receives the sensor switching configuration information of the Nth frame, it sends a first feedback message (i.e., a first notification) to the CRM. The first feedback message is used to indicate that the sensor driver has received the sensor switching configuration information of the Nth frame.

[0202] Step 805: The IFE driver notifies the CRM that it has received the SOF interrupt signal of the sensor's N-3rd frame.

[0203] Specifically, when the IFE driver receives the SOF interrupt signal of the sensor in frame N-3, it notifies the CRM that the sensor's SOF interrupt signal in frame N-3 has been detected.

[0204] Step 806: In response to the notification sent by the IFE driver, the CRM sends the first control command to the sensor driver.

[0205] Specifically, in response to the notification sent by the IFE driver (i.e., the notification indicating that the IFE has received the SOF interrupt signal of frame N-3), the CRM sends a first control command to the sensor driver. This first control command instructs the Sensor driver to issue a request for frame N-1. The request for frame N-1 includes the Sensor configuration information for frame N-1, where the image output mode for frame N-1 is mode A (i.e., the first image output mode). In this example, the image output mode for frame N-2 is the same as that for frame N-1; therefore, the Sensor configuration information for frame N-1 includes the camera parameters for frame N-1, and the Sensor node sends this information to the Sensor driver in frame N-2.

[0206] Step 807: The Sensor driver sends the Sensor configuration information of the (N-1)th frame to the Sensor.

[0207] Specifically, in response to the first control command of the CRM, the Sensor driver sends the Sensor configuration information for frame N-1 to the Sensor, and the Sensor writes the Sensor configuration information for frame N-1 into its corresponding register. This Sensor configuration information for frame N-1 takes effect at the effective point of frame N-2, and the N-2 frame is plotted based on the Sensor configuration information for frame N-2.

[0208] Step 808: The Sensor node generates the Sensor activation configuration information and the first data packet for the Nth frame.

[0209] Specifically, within the (N-2)th frame, the sensor can generate the sensor activation configuration information and the first data packet for the Nth frame. This sensor activation configuration information and the first data packet for the Nth frame match the sensor switching configuration information for the Nth frame.

[0210] Step 809: The Sensor node sends the N+1th frame request to the Sensor driver.

[0211] Specifically, the sensor node generates a request for frame N+1 based on the sensor activation configuration information and the first data packet from frame N. This request includes the sensor activation configuration information and the first data packet from frame N. The sensor node then sends the request for frame N+1 to the sensor driver.

[0212] Step 810: The Sensor driver receives the Sensor activation configuration information and the first data packet in the Nth frame.

[0213] Step 811: When the Sensor driver receives the Sensor activation configuration information and the first data packet in the Nth frame, it sends the second feedback information to the CRM.

[0214] Step 812: The IFE driver notifies the CRM that it has received the SOF interrupt signal of the sensor's N-2th frame.

[0215] Specifically, when the IFE receives the SOF interrupt signal of the sensor in the (N-2)th frame, it can send a first indication message to the CRM to notify the CRM that the SOF interrupt signal of the sensor in the (N-2)th frame has been received.

[0216] Step 813: In response to the notification sent by the IFE driver, the CRM sends a second control command to the Sensor driver.

[0217] Specifically, in response to the notification sent by the IFE driver (i.e., the notification indicating that the IFE has received the SOF interrupt signal of the N-2th frame), the CRM sends a second control command to the sensor driver. The second control command is used to instruct the Sensor driver to issue the Nth frame request, which includes the Sensor switching configuration information of the Nth frame.

[0218] Step 814: The sensor driver sends the sensor switching configuration information for the Nth frame to the sensor.

[0219] Specifically, in response to the second control command (i.e. the second notification), the sensor driver sends the sensor switching configuration information of the Nth frame to the sensor, and the sensor's register writes the sensor switching configuration information into the corresponding register.

[0220] Step 815: The sensor node generates the sensor configuration information for the N+1th frame.

[0221] Specifically, the sensor node generates the sensor configuration information for the N+1th frame within the N-1th frame. Since the decision module determines that the image output mode will not change in the N+1th frame, the configuration information for the N+1th frame generated by the sensor node includes the camera parameters for the N+1th frame, such as the frame length, but does not include the switching configuration information, the effective configuration information, or the first data packet.

[0222] In addition, the IFE node also generates IFE configuration information for frame N+1. The IFE configuration information for frame N+1 is written into the IFE register in frame N and takes effect in frame N+1.

[0223] Step 816: The sensor node sends the N+2th frame request to the sensor driver.

[0224] Specifically, the sensor node generates a request for frame N+2 based on the sensor configuration information of frame N+1. The request for frame N+2 includes the sensor configuration information of frame N+1. The sensor node then sends the request for frame N+2 (i.e., the sensor configuration information of frame N+1) to the sensor driver within frame N-1.

[0225] Step 817: The sensor driver receives the request for frame N+2.

[0226] Step 818: When the sensor driver receives the request for the N+2th frame, it sends the third feedback information to the CRM.

[0227] Specifically, when the sensor driver receives the N+2 frame request, it sends a third feedback message (i.e., a third notification) to the CRM. The third feedback message is used to indicate that the sensor driver has received the N+2 frame request.

[0228] In addition, the IFE node can also send the IFE configuration information of the N+1th frame to the IFE driver in the N-1th frame. When the IFE driver receives the IFE configuration information of the N+1th frame, it sends a feedback to the CRM that it has received the IFE configuration information of the N+1th frame (that is, the IFE driver sends the fifth notification).

[0229] Step 819: IFE notifies CRM that it has received the SOF interrupt signal of the (N-1)th frame of the Sensor.

[0230] Specifically, when the IFE driver receives the SOF interrupt signal of the sensor in frame N-1, it notifies the CRM that the sensor's SOF interrupt signal in frame N-1 has been detected.

[0231] Step 820: CRM sends a third control command to the sensor driver.

[0232] Specifically, after receiving the notification from the IFE (i.e., the notification indicating that the IFE has received the SOF interrupt signal of the sensor's (N-1)th frame), the CRM sends a third control instruction (i.e., a fourth notification) to the sensor driver. The third control instruction is used to instruct the sensor driver to send the effective configuration information of the Nth frame to the sensor.

[0233] Furthermore, after receiving the IFE configuration information of frame N+1 within frame N-1, the IFE driver notifies the CRM that the IFE driver has received the IFE configuration information of frame N+1. After receiving the notification from the IFE (i.e., the notification indicating that the IFE has received the SOF interrupt signal of the sensor in frame N-1), the CRM instructs the IFE driver to send the IFE configuration information of frame N to the IFE within frame N-1.

[0234] Step 821: The sensor driver determines whether the Sensor effective configuration information for the Nth frame has been sent.

[0235] In response to the third control command, the sensor driver determines whether it has already sent the sensor activation configuration information for frame N. Specifically, the sensor driver obtains the first identifier of frame N. When it detects that the first identifier of frame N is a second state value, it determines that the sensor driver has not sent the sensor activation configuration information for frame N to the sensor.

[0236] Step 822: The sensor driver sends the effective configuration information of the Nth frame to the sensor so that the sensor can write the effective configuration information of the Nth frame into its register.

[0237] In addition, after receiving the information from the CRM instructing the IFE driver to send the IFE configuration information of the Nth frame to the IFE in the (N-1)th frame, the IFE driver sends the IFE configuration information of the Nth frame to the IFE.

[0238] The Sensor configuration information and IFE configuration information of frame N take effect at the effective point of frame N. The Sensor outputs the image of frame N with reference to the Sensor configuration information of frame N and sends the output image frame to the IFE. The IFE then performs image processing on frame N based on the IFE configuration information of frame N.

[0239] In this example, when the sensor driver detects that the sensor activation configuration information for frame N has not been sent, it sends the activation configuration information for frame N to the sensor in frame N-1. Within frame N-1, the sensor driver sends the sensor activation configuration information for frame N to the sensor. After receiving the sensor activation configuration information for frame N, the sensor writes the entire configuration information into its register before the activation point of frame N, enabling the sensor to activate the sensor configuration information for frame N at the activation point of frame N, expose the image frame, and output the image at the SOF of frame N.

[0240] After the Sensor driver completes sending the Sensor activation configuration information for the Nth frame to the Sensor within N-1 frames, the first identifier of the Nth frame can be updated from the second state value (e.g., "false") to the first state value (e.g., "true"). When the first identifier of the Nth frame is the first state value, it indicates that the Sensor driver has sent the Sensor activation configuration information for the Nth frame. When the first identifier of the Nth frame is the second state value, it indicates that the Sensor driver has not sent the Sensor activation configuration information for the Nth frame.

[0241] The following reference Figure 8b Next, we will explain how to set the camera parameters.

[0242] Step 823: The sensor node generates the sensor configuration information for the N+2th frame.

[0243] Continue to refer to Figure 5dWhen the decision module determines that the image output mode will not change in frame N+1, it does not send image output mode information to the Sensor node. The Sensor node generates Sensor configuration information for frame N+1, which includes Sensor configuration information for frame N+2 (i.e., second exposure information and second gain information). Since the image output mode has not changed, this Sensor configuration information for frame N+2 is written into the Sensor register in frame N+1 and takes effect in frame N+2.

[0244] Step 824: The sensor node sends the N+3rd frame request to the sensor driver.

[0245] Specifically, the Sensor node generates a request for the N+3rd frame based on the Sensor configuration information in the N+2nd frame. The N+3rd frame request includes the Sensor configuration information from the N+2nd frame. Within the Nth frame, the Sensor node sends the N+3rd frame request (i.e., the Sensor configuration information from the N+2nd frame) to the Sensor driver.

[0246] In addition, the IFE node can also send the IFE configuration information of the N+2th frame to the IFE driver within the Nth frame.

[0247] Step 825: The sensor driver receives the request for frame N+3.

[0248] For example, the Sensor driver receives the request for the N+3rd frame and obtains the Sensor configuration information for the N+2nd frame.

[0249] In addition, the IFE driver receives the IFE configuration information of the N+2th frame. It should be noted that after receiving the IFE configuration information of the N+2th frame, the IFE driver will also notify the CRM that the IFE has received the IFE configuration information of the N+2th frame.

[0250] Step 826: The sensor driver sends the fourth feedback to the CRM.

[0251] For example, after receiving the N+3rd frame request, the Sensor driver sends a fourth feedback to the CRM, which indicates that the Sensor driver has received the N+3rd frame request.

[0252] In addition, after receiving the IFE configuration information in frame N+2, the IFE driver notifies the CRM that the IFE driver has received the IFE configuration information in frame N+2.

[0253] Step 827: The IFE driver notifies the CRM that it has received the SOF interrupt signal of the Nth frame of the Sensor.

[0254] For example, when the IFE driver receives the SOF interrupt signal of the Nth frame of the sensor, it notifies the CRM of the information that the IFE driver has received the SOF interrupt signal of the Nth frame of the sensor.

[0255] Step 828: In response to the notification from the IFE driver, the CRM sends a fourth control command to the sensor driver.

[0256] For example, when the CRM receives a notification from the IFE driver (i.e., a notification indicating that the IFE has received the SOF interrupt signal of the Nth frame of the Sensor), it sends a fourth control instruction to the sensor driver. This fourth control instruction is used to instruct the sensor driver to send the next pending request to the Sensor in sequence (i.e., send the Sensor configuration information of the N+1th frame).

[0257] In addition, when CRM receives a notification from the IFE driver (i.e., a notification indicating that the IFE has received the SOF interrupt signal of the Nth frame of the Sensor), it instructs the IFE driver to send the next pending request to the IFE in sequence (i.e., send the IFE configuration information of the N+1th frame).

[0258] Step 829: The sensor driver writes the sensor configuration information of frame N+1 to the sensor's register.

[0259] For example, after receiving the fourth control command, the Sensor driver retrieves the next pending request according to the processing order in the pending queue (the next pending request is the N+2 frame request). The Sensor driver sends the N+2 frame request (i.e., the Sensor configuration information of the N+1 frame) to the Sensor within the N frame. The Sensor writes the Sensor configuration information of the N+1 frame from the N+2 frame request into the Sensor's register. The Sensor then applies the N+1 frame Sensor configuration information at the effective point of the N+1 frame.

[0260] refer to Figure 8b The problem of insufficient transmission performance of the electronic device caused the IFE driver to send the IFE configuration information of the N+1 frame to the IFE in the N+1 frame instead of the N+1 frame. This resulted in a mismatch between the sensor output and the IFE processing in the N+1 frame.

[0261] Step 830: The IFE driver notifies the CRM that it has received the SOF interrupt signal of the N+1th frame of the Sensor.

[0262] Step 831: In response to the notification from the IFE driver, the CRM sends the first rollback command to the sensor driver.

[0263] Specifically, when CRM detects that the IFE driver has not sent the IFE configuration information for frame N+1 in frame N and receives a notification from the IFE driver (a notification indicating that the IFE driver has received the interrupt signal of SOF in frame N+1), it sends a first rollback instruction to the Sensor driver and a second rollback instruction to the IFE driver.

[0264] The first rollback instruction is used to instruct the Sensor driver to resend the Sensor effective configuration information for frame N (i.e., the request for frame N+1), and the second rollback instruction is used to instruct the IFE driver to resend the IFE configuration information for frame N.

[0265] Step 832: The sensor driver determines whether the Sensor effective configuration information for frame N has been sent before the request for frame N+1.

[0266] After receiving the first rollback command from the CRM, the Sensor driver determines whether the Sensor activation configuration information for frame N has been sent. Specifically, the Sensor driver obtains the first identifier of frame N. When it detects that the first identifier of frame N is the first state value, it determines that the Sensor driver has sent the Sensor activation configuration information for frame N to the Sensor, and executes step 833.

[0267] Step 833: The sensor driver sends the first data packet to the sensor's register so that the sensor can write the first data packet into the register.

[0268] Once the sensor driver determines that it has sent the sensor activation configuration information for frame N to the sensor, it sends the first data of frame N to the sensor. The sensor then writes the first data packet of frame N into its register before the activation point of frame N+1. The first data packet of frame N includes the first exposure time and first gain information of the sensor in frame N.

[0269] In addition, after receiving the second rollback instruction, the IFE driver resends the IFE configuration information of the Nth frame to the IFE in the N+1th frame.

[0270] It should be noted that the Sensor writes the effective configuration information of the Nth frame in the (N+1)th frame, and the IFE writes the IFE configuration information of the Nth frame in the (N+1)th frame. The Sensor configuration information of the Nth frame takes effect in the (N+2)th frame, and the IFE configuration information of the Nth frame takes effect in the (N+2)th frame.

[0271] The effective configuration information for frame N written by the sensor in frame N-1 and the IFE configuration information for frame N written by the IFE in frame N-1 have already taken effect at the effective point of frame N. In frame N+1, the effective sensor configuration information for frame N and the IFE configuration information for frame N will continue to be used.

[0272] In this example, due to insufficient performance of the electronic device, the IFE is delayed. The Sensor driver and the IFE driver perform a rollback operation. The Sensor driver is instructed to resend the N+1 frame request (i.e., the Sensor effective configuration information for the Nth frame). Based on this instruction, the Sensor detects that the N+1 frame request has already been sent and sends the first data packet of the Nth frame to the Sensor. This avoids the repeated sending of the effective configuration information for the Nth frame and also ensures that the Sensor configuration information for the Nth frame takes effect at the correct time, avoiding frame loss in the N+1 and N+2 frames.

[0273] Figure 9 A timing diagram is shown for the photo parameter setting method in the embodiment of this application in a Bubble scene.

[0274] Figure 9 The process of the IFE node sending IFE configuration information for each frame to the IFE driver is not shown, nor is the process of the Sensor node sending Sensor configuration information for each frame to the Sensor driver. For details, please refer to [link to relevant documentation]. Figure 5a and Figure 5b The corresponding textual descriptions will not be repeated here.

[0275] Please refer to Figure 9 Assuming that within the effective frame duration of the first frame, the Sensor driver sends the Sensor request 0 (referred to as Sensor R0) to the Sensor. Sensor R0 includes the Sensor switching configuration information in the Sensor configuration information of the Nth frame. The Sensor writes the Sensor switching configuration information of the Nth frame into the corresponding register.

[0276] The Sensor driver sends Request 1 (Sensor R1) to the Sensor in the second frame. Sensor R1 includes the Sensor activation configuration information for frame N and the first data packet for frame N (including the exposure time and gain information for frame N). The IFE driver sends IFE Request 0 (IFE R0) to the IFE in the second frame. IFE R0 includes the IFE configuration information for frame N. Sensor R1 and IFE R0 are activated in frame N. That is, in frame N, the Sensor exposes according to Sensor configuration information 1 (including Sensor R0 and Sensor R1) and outputs the exposed image to the IFE. The IFE processes the image frame according to the IFE configuration information for frame N (i.e., IFE R0). At this time, the IFE's output mode matches the Sensor's output mode.

[0277] Within frame N, the decision module determines that the image output mode for frame N+1 remains unchanged. The Sensor node sends the Sensor configuration information for frame N+1 to the Sensor driver. This Sensor configuration information includes camera parameters. The IFE node sends the IFE configuration information for frame N+1 to the IFE driver. Since the image output mode remains unchanged for frame N+1, the Sensor configuration information and IFE configuration information for frame N+1 are written in frame N and take effect in frame N+1.

[0278] Continue to refer to Figure 9 Within frame N, the Sensor driver sends Sensor Request2 (referred to as Sensor R2) to the Sensor. Sensor R2 includes the Sensor configuration information from frame N+1. However, due to insufficient performance of the electronic device, the IFE driver does not send IFE Request1 (referred to as IFE R1) to the IFE normally. Instead, it delays sending IFE R1 to the IFE until frame N+1. This IFE R1 includes the IFE configuration information from frame N+1. Since the IFE configuration information takes effect with a one-frame delay, that is, the IFE writes IFE R1 into the register in frame N+1, and the IFE configuration information from frame N+1 only takes effect in frame N+2.

[0279] The IFE still takes effect according to the IFE configuration information of frame N at the effective point of frame N+1, while the Sensor takes effect according to the Sensor configuration information of frame N+1. The Sensor exposes according to the Sensor configuration information of frame N+1 and outputs the image according to the original mode. The exposed image is then transmitted to the IFE. At this time, the IFE's image processing and the Sensor's output do not match, triggering the IFE driver and the Sensor driver to perform a rollback operation.

[0280] Reference Figure 9 The Sensor driver and IFE driver will fall back to the frame before the IFE driver delayed sending the request. The specific process is as follows:

[0281] 1. The IFE driver discards the current frame (e.g.) Figure 9 The (N+1)th frame in the image and the next frame (e.g., the next frame) Figure 9 The content of the (N+2)th frame in the text;

[0282] 2. In frame N+1, the Sensor driver rolls back to the frame before the IFE driver sends IFE R11. That is, the Sensor driver resends Sensor R12 and the IFE driver resends IFE R10 in frame N+1.

[0283] The sensor driver detects that the sensor activation configuration information for frame N has been sent, and sends the first data packet of frame N to the sensor.

[0284] 3. In frame N+2, the IFE driver reissues IFE's R11.

[0285] 4. In frame N+2, the Sensor driver reissues the Sensor's R12.

[0286] Since the Sensor driver does not resend the Sensor activation configuration information for frame N during step 2, the Sensor switching configuration information for frame N is paired with the Sensor activation configuration information for frame N, thus avoiding functional problems and preventing frame loss issues that could affect the user experience.

[0287] In one example, Figure 10 The schematic block diagram illustrating an embodiment of the present application shows an apparatus 1000. The apparatus 1000 may include a processor 1001 and a transceiver / transceiver pin 1002, and optionally, a memory 1003.

[0288] The various components of device 1000 are coupled together via bus 1004, which includes a data bus, a power bus, a control bus, and a status signal bus. However, for clarity, all buses are referred to as bus 1004 in the figure.

[0289] Optionally, the memory 1003 can be used for the instructions in the foregoing method embodiments. The processor 1001 can be used to execute the instructions in the memory 1003, control the receive pin to receive signals, and control the transmit pin to transmit signals.

[0290] The device 1000 may be an electronic device or a chip of an electronic device in the above method embodiments.

[0291] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0292] This embodiment also provides a computer storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the methods described in the above embodiments.

[0293] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the methods described in the above embodiments.

[0294] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the methods in the above-described method embodiments.

[0295] In this embodiment, the electronic device, computer storage medium, computer program product or chip are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding method provided above, and will not be repeated here.

[0296] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for setting photographing parameters, characterized in that, Applied to electronic devices, the method includes: The sensor driver obtains the image output mode configuration information of the Nth frame from the sensor node. The image output mode configuration information of the Nth frame includes switching configuration information for switching from the first image output mode to the second image output mode, effective configuration information corresponding to the switching configuration information, and a first data packet. The first data packet includes first exposure information and first gain information. At the first moment, the sensor driver sends the switching configuration information of the Nth frame to the camera sensor, and the first moment is within the (N-2)th frame; The sensor driver determines whether the effective configuration information of the Nth frame has been sent to the camera sensor; When the sensor driver detects that the effective configuration information of the Nth frame has been sent to the camera sensor, it sends the first data packet of the Nth frame to the camera sensor at a second time, the first time being before the second time. When the sensor driver determines that the effective configuration information of the Nth frame has not been sent to the camera sensor, the effective configuration information of the Nth frame is sent to the camera sensor at the second moment. The camera sensor outputs an image of the Nth frame based on the image output mode configuration information of the Nth frame, wherein the image output mode of the Nth frame is the second image output mode. Among them, the output mode of the (N-3)th frame, the (N-2)th frame and the (N-1)th frame is the first output mode, and N is an integer greater than or equal to 3.

2. The method according to claim 1, characterized in that, The sensor driver determines whether the effective configuration information for the Nth frame has been sent to the camera sensor, including: The first identifier of the Nth frame acquired by the sensor driver corresponds to the image output mode configuration information of the Nth frame. The first identifier of the Nth frame is used to record whether the sensor driver has sent the effective configuration information of the Nth frame to the camera sensor. When the sensor driver detects that the first identifier of the Nth frame is a first state value, it determines that the sensor driver has sent the effective configuration information of the Nth frame to the camera sensor. When the sensor driver detects that the first identifier of the Nth frame is the second state value, it determines that the sensor driver has not sent the effective configuration information of the Nth frame to the camera sensor, and the first state value is different from the second state value.

3. The method according to claim 1, characterized in that, The sensor driver obtains the image output mode configuration information of the Nth frame from the sensor node, including: Within the (N-3)th frame, the sensor node sends the switching configuration information for the Nth frame to the sensor driver; Within the (N-2)th frame, the sensor node sends the effective configuration information and the first data packet of the Nth frame to the sensor driver.

4. The method according to claim 3, characterized in that, Before the sensor driver sends the switching configuration information for the Nth frame to the camera sensor, the method further includes: When the sensor driver receives the effective configuration information and the first data packet of the Nth frame sent by the sensor node, the sensor driver sends the first notification of the Nth frame to the camera request manager (CRM). The first notification of the Nth frame is used to indicate that the sensor driver has received the effective configuration information and the first data packet of the Nth frame. When the IFE driver receives the Start of Frame (SOF) signal of the N-2th frame output by the camera sensor, it sends the first indication information to the CRM. In response to the first indication information, the CRM sends a second notification to the camera sensor. The second notification is used to instruct the sensor driver to send the switching configuration information of the Nth frame to the camera sensor. When the sensor driver receives the second notification, it triggers the sensor driver to perform the operation of sending the switching configuration information of the Nth frame to the camera sensor; Before the sensor driver determines whether the effective configuration information of the Nth frame has been sent to the camera sensor, the method further includes: Within the (N-1)th frame, when the sensor driver receives the camera parameters of the (N+1)th frame from the sensor node, it sends a third notification to the Camera Request Manager (CRM). The third notification is used to indicate that the sensor driver has received the camera parameters of the (N+1)th frame. When the IFE driver receives the frame start delimiter (SOF) signal of the (N-1)th frame output by the camera sensor, it sends a second indication message to the CRM. In response to the second indication information, the CRM sends a fourth notification to the camera sensor, the fourth notification being used to instruct the sensor driver to send the effective configuration information of the Nth frame to the camera sensor; When the sensor driver receives the fourth notification, it triggers the operation of determining whether the effective configuration information of the Nth frame has been sent to the camera sensor.

5. The method according to claim 4, characterized in that, When the second time point is within the (N-1)th frame, after sending the effective configuration information for the Nth frame to the camera sensor at the second time point, the method further includes: Within the Nth frame, the sensor driver sends the camera parameters of the N+1th frame to the camera sensor, and the IFE driver does not send the IFE configuration information of the N+1th frame to the IFE. The IFE configuration information of the N+1th frame is sent from the IFE node to the IFE driver within the N-1th frame. When the CRM determines that the sensor driver has sent the camera parameters of the N+1th frame to the camera sensor in the Nth frame and the IFE driver has not sent the IFE configuration information of the N+1th frame to the IFE in the Nth frame, it instructs the camera sensor to discard the information in the N+1th frame and instructs the IFE to discard the information in the N+1th frame. The camera sensor discards information within the (N+1)th frame, and the IFE discards information within the (N+1)th frame. The CRM sends a first rollback instruction to the sensor driver and a second rollback instruction to the IFE driver in the N+1th frame. The first rollback instruction is used to instruct the sensor driver to resend the effective configuration information of the Nth frame in the N+1th frame, and the second rollback instruction is used to instruct the IFE driver to resend the IFE configuration information of the Nth frame in the N+1th frame. The sensor driver responds to the first rollback command by determining whether the effective configuration information of the Nth frame has been sent to the camera sensor; When the sensor driver detects that the effective configuration information of the Nth frame has been sent, it sends the first data packet of the Nth frame to the camera sensor in the N+1th frame. In response to the second rollback instruction, the IFE driver sends the IFE configuration information of the Nth frame to the IFE within the N+1th frame; The camera sensor outputs an image for the (N+2)th frame based on the image output mode configuration information of the Nth frame. The IFE performs image processing on the (N+2)th frame image output by the camera sensor based on the IFE configuration information of the Nth frame.

6. The method according to claim 5, characterized in that, The method further includes: Within the N+2nd frame, the sensor driver sends the camera parameters of the N+1th frame to the camera sensor; The camera sensor outputs an image for the N+3rd frame based on the camera parameters of the N+1th frame.

7. The method according to claim 6, characterized in that, The method further includes: Within the N+2nd frame, the IFE driver sends the IFE configuration information of the N+1th frame to the IFE; The IFE performs image processing on the (N+3)th frame image output by the camera sensor based on the configuration information of the (N+1)th frame.

8. The method according to claim 1, characterized in that, include: The point at which the output mode of the Nth frame takes effect is before the Nth frame, and the duration between the point at which the output mode of the Nth frame takes effect and the SOF of the Nth frame is the first duration. The camera sensor outputs an image of the Nth frame based on the second image output mode configuration information of the Nth frame, including: The camera sensor acquires the second image output mode configuration information of the Nth frame at the point where the image output mode takes effect in the Nth frame; The camera sensor outputs an image of the Nth frame at SOF.

9. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the electronic device to perform the photographing parameter setting method according to any one of claims 1 to 8.

10. A computer-readable storage medium comprising a computer program, characterized in that, When the computer program is run on the electronic device, the electronic device performs the photographing parameter setting method according to any one of claims 1 to 8.

Images