Picture output control method, electronic equipment and chip system

Abstract

The embodiment of the invention provides a shooting method which is applied to electronic equipment comprising a camera and an image sensor. According to the method, after the new drawing output mode configuration parameters are generated, drawing output of at least one frame can be continuously carried out through the configured configuration parameters, and frame rate hopping in the drawing output configuration adjustment process is avoided.

Description

[0001] This application is a divisional application. The application number of the original application is 202311583534.7, and the original application date is November 23, 2023. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The embodiments of the present application relate to the technical field of electronic devices, and in particular to a method for controlling image output, an electronic device, and a chip system. Background Art

[0003] Electronic devices can control image sensors to output images at different frame rates. When the image output frame rate needs to be adjusted, the image sensor needs to wait for the new frame rate configuration to take effect before the adjustment can be completed. In this way, after the image sensor receives the new image output frame rate configuration, it will stop outputting images before the new frame rate configuration takes effect. As a result, the frame rate jumps multiple times before and after the image output frame rate is adjusted, affecting the display effect. Summary of the invention

[0004] The present application provides a method for controlling image output, an electronic device and a chip system, which can continue to output at least one frame of image through the configured configuration after generating new image output mode configuration parameters, thereby avoiding frame rate jumps during the image output configuration adjustment process.

[0005] In order to achieve the above technical objectives, this application adopts the following technical solutions:

[0006] In a first aspect, a method for controlling image output is provided, the method being applied to an electronic device, wherein a first image sensor is configured in the electronic device, and the first image sensor corresponds to at least one camera configured in the electronic device. The method comprises: generating first control information, wherein the first control information comprises a first frame rate. Controlling the first image sensor to output the first image. The time interval between outputting the first image and outputting the second image is a first duration. The first duration corresponds to a second frame rate, and the second image is a previous frame image of the first image.

[0007] In this way, after generating new configuration parameters (such as control information including the first frame rate), the electronic device can continue to control the first image sensor to output the next frame of image (such as the first image) according to the existing configuration (such as including the second frame rate). This avoids the frame rate jump caused by waiting for the new configuration parameters to take effect after receiving the new configuration parameters.

[0008] In some possible designs, before generating the first control information, the method further includes: controlling the first image sensor to output the second image at a second frame rate. This clarifies that before receiving the first control information, the electronic device has configured parameters including the second frame rate, that is, outputting the image according to the second frame rate.

[0009] In some possible designs, the first control information also includes a first camera identifier. The first camera identifier is an identifier of a first camera currently in use, the first camera is included in the at least one camera, and the first camera corresponds to the first image sensor. Before controlling the first image sensor to output the first image, the method also includes: determining, based on the first camera identifier, that the first image sensor corresponding to the first camera supports a first function, and the first function corresponds to continuing to output at least one frame according to the configured frame rate after receiving a new frame rate configuration. Thus, when the first image sensor can support continuing to output images according to the existing configuration after receiving a new frame rate configuration, the solution provided in the present application is enabled so that the solution can be effectively executed.

[0010] In some possible designs, the new frame rate configuration includes the first frame rate, and the configured frame rate includes the second frame rate.

[0011] In some possible designs, the electronic device is configured with a first function list, the first function list includes at least one camera identifier, and each image sensor corresponding to the camera identifier in the first function list supports the first function. According to the first camera identifier, determining that the first image sensor corresponding to the first camera supports the first function includes: according to the first camera identifier included in the first function list, determining that the first image sensor corresponding to the first camera supports the first function. Thus, a judgment mechanism for image sensors that support the first function is provided. In some embodiments, the first function list may be actively generated by the electronic device, for example, the electronic device generates the first function list when the electronic device is powered on for hardware verification. In other embodiments, the first function list may be pre-configured.

[0012] In some possible designs, the electronic device is configured with a first register, a second register, and a third register for the first image sensor. After determining that the first image sensor supports the first function, the method further includes: configuring the first register to a first value, indicating that the second register and the third register are enabled. Configuring the second register to a first value, indicating that the first function is enabled. The first frame rate is stored in the third register.

[0013] In some possible designs, the third register includes a first storage area and a second storage area. Storing the first frame rate in the third register includes: storing the first frame rate in the second storage area. Before storing the first frame rate in the second storage area, the second frame rate is stored in the first storage area.

[0014] In some possible designs, the method further includes: configuring the first register to a second value, indicating disabling configuration of the second register and the third register.

[0015] In some possible designs, controlling the first image sensor to output the first image includes: when the second register is the first value, controlling the first image sensor to output the first image at the second frame rate stored in the first storage area.

[0016] In some possible designs, after controlling the first image sensor to output the first image, the method further includes: storing the first frame rate overwriting the second frame rate in the first storage area.

[0017] Thus, through the configuration of the three registers, the first function of the first image sensor is enabled, so that after receiving the new configuration, the image is continuously output according to the existing configuration.

[0018] In some possible designs, after controlling the first image sensor to output the first image, the method further includes: controlling the first image sensor to output a third image. The time interval between outputting the third image and outputting the first image is a second duration. The second duration corresponds to the first frame rate, and the third image is the next frame image of the first image. In this example, taking the example of continuing to output a frame of image according to the existing configuration after receiving a new configuration. In other implementations, before the new configuration takes effect, the electronic device can output multiple frames of images according to the existing configuration.

[0019] In some possible designs, the first frame length is the same as or similar to the second frame length; the first frame length is the frame length of the third image, and the second frame length is the frame length of the first image or the second image; the first frame length and the second frame length are similar including: the difference between the first frame length and the second frame length does not exceed 10%.

[0020] In some possible designs, controlling the first image sensor to output the third image includes: in the process of controlling the first image sensor to output the third image, the exposure time of the first image sensor is less than or equal to an exposure limit parameter, so that the first frame length is the same as or similar to the second frame length.

[0021] Therefore, by controlling the exposure time of the first frame of image after the new configuration takes effect, the image frame lengths before and after the new configuration takes effect are made consistent, thereby avoiding image jumps.

[0022] In some possible designs, before controlling the first image sensor to output a third image, the method further includes: determining the exposure limit parameter according to a first configuration parameter and a second configuration parameter. The first configuration parameter is the configuration parameter corresponding to the first frame rate, and the second configuration parameter is the configuration parameter corresponding to the second frame rate.

[0023] In some possible designs, before generating the first control information, the method further includes: obtaining the image parameters of a fourth image. The image parameters include the auto-exposure (AE) exposure duration and / or AE exposure gain of the fourth image. Generating the first control information is determined according to the image parameters of the fourth image.

[0024] In some possible designs, the determining to generate the first control information according to the image parameters of the fourth image includes: when receiving first operation information or second operation information, determining to generate the first control information according to the image parameters of the fourth image. The first operation information is generated by the electronic device according to a received first operation, and the first operation is used to instruct the electronic device to perform focal length adjustment. The second operation information is generated by the electronic device according to a received second operation, and the second operation is used to instruct the electronic device to perform shooting mode adjustment.

[0025] In some possible designs, the first operation is specifically used to instruct the electronic device to adjust the focal length magnification from 1× to 2×.

[0026] Thus, a mechanism for the electronic device to determine to perform an image output mode (such as including frame rate) adjustment is provided.

[0027] In some possible designs, the first frame rate corresponds to the frame rate in the InsensorZoom mode, and the second frame rate corresponds to the frame rate in the binning mode.

[0028] In a second aspect, an electronic device is provided. The electronic device includes: a memory, one or more processors, and at least one image sensor. The memory, the processor, and the image sensor are coupled to each other in pairs. Among them, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device controls the image sensor to output images according to the method provided in the first aspect and any one of its possible designs.

[0029] In a third aspect, the present application also provides a chip system, which is applied to an electronic device; the chip system may include one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected by a line, and the interface circuit is used to receive a signal from a memory of the electronic device and send the signal to the processor, and the signal includes a computer instruction stored in the memory. When the processor executes the above-mentioned computer instruction, the electronic device executes the technical solution provided in the above-mentioned first aspect and any possible implementation thereof.

[0030] In a fourth aspect, the present application also provides a computer-readable storage medium, including computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the technical solution provided in the above-mentioned first aspect and any possible implementation thereof.

[0031] In a fifth aspect, the present application also provides a computer program product, which, when executed on a computer, enables the computer to execute the technical solution provided in the above-mentioned first aspect and any possible implementation thereof.

[0032] It can be understood that the solutions provided in the second aspect to the fifth aspect of the present application can respectively correspond to the first aspect and any possible design thereof, so the beneficial effects that can be achieved are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of an interface interaction;

[0034] Figure 2 A schematic diagram of the interaction between modules inside an electronic device;

[0035] Figure 3 This is a schematic diagram of register configuration inside a camera driver;

[0036] Figure 4 A schematic diagram of an interface interaction for adjusting focus and switching shooting modes;

[0037] Figure 5 A schematic diagram of register parameter adjustment;

[0038] Figure 6 A schematic diagram of the output logic of a frame image;

[0039] Figure 7 A schematic diagram of the output logic of a frame image after the solution provided in an embodiment of the present application takes effect;

[0040] Figure 8 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application;

[0041] Fig. 9A schematic diagram of the composition of an electronic device provided in an embodiment of the present application;

[0042] Fig.10 A schematic diagram of a register configuration provided in an embodiment of the present application;

[0043] Fig.11 A schematic diagram of a flow chart of interaction between modules provided in an embodiment of the present application;

[0044] Fig.12 A schematic diagram of a flow chart of interaction between modules provided in an embodiment of the present application;

[0045] Fig.13 A schematic diagram of a register configuration provided in an embodiment of the present application;

[0046] Fig.14 A schematic diagram of a flow chart of interaction between modules provided in an embodiment of the present application;

[0047] Fig.15 A schematic diagram of a flow chart of interaction between modules provided in an embodiment of the present application;

[0048] Fig.16 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application;

[0049] Fig.17 A schematic diagram of the composition of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0051] Currently, electronic devices can provide users with a shooting function by configuring a camera module.

[0052] Exemplarily, the camera module may include a camera and an image sensor. In the case where multiple camera modules are configured in the electronic device, each camera module may correspond to a camera. Different cameras may be respectively configured with corresponding image sensors. Alternatively, two or more of the different cameras may share one image sensor.

[0053] For an image sensor, after acquiring the light signal collected by a camera, it can output raw images of different modes in different scenes. The raw images can be RAW images.

[0054] Exemplarily, different modes may include modeA and modeB. The frame rate of the output image in modeA is different from the frame rate of the input image in modeB. In some cases, the image size of the original image output in modeA and the original image output in modeB may also be different. For example, modeA may be a binning mode, and modeB may be an InsensorZoom mode.

[0055] The frame rate is in frames. The frame rate is used to indicate the frequency at which the image sensor exposes and outputs images in a unit of time. In the following description, the process of the image sensor exposing and outputting images is referred to as image sensor output. The frame rate is usually expressed in fps (frame per second). For example, if the frame rate is 30fps, it means that 30 frames of images are output in 1 second.

[0056] Image size is used to indicate the size of the original image output by the image sensor.

[0057] As a possible implementation, the image size may include the length, width, resolution, etc. of the image data.

[0058] As another possible implementation, the image size may include parameters such as line length and frame length corresponding to the image data.

[0059] The line length is the length of the image data output by the image sensor on a line. The line length may include HBlank. The line length may also be called HTS.

[0060] Frame length is the number of lines in a frame. Frame length can include V Blank. Frame length can also be called VTS.

[0061] It is understood that the line length may correspond to the length of the original image output by the image sensor. The frame length may determine the width of the original image output by the image sensor. The line length and the frame length may jointly determine the resolution of the output image.

[0062] Thus, the electronic device can obtain the original images required in different scenes for display by controlling the image sensor to output the original images in modeA or modeB.

[0063] For example, a camera application is installed in an electronic device, and the electronic device controls an image sensor to output an image through the camera application, thereby providing a shooting function.

[0064] refer to Figure 1 , which is a schematic diagram of an interface interaction.

[0065] like Figure 1As shown, the electronic device may display an icon 102 of the camera application on the main interface 101. After receiving a click operation 103 of the user on the icon 102, the electronic device may correspondingly display an interface 104 of the camera application.

[0066] For example, the camera application provides the photo taking function by default after opening. Figure 1 As shown in the interface 104 in FIG. 1 , the electronic device may display the current shooting mode as “photographing” in the center of the function bar 105. The interface 104 may also include a preview image 106 of the photographed object. Thus, when the user determines to take a photo, he may click a button 107. Correspondingly, the camera application may respond to the button 107 to perform processing such as photographing and saving the current photographed object.

[0067] In some implementations, such as Figure 1 As shown, other function options may also be displayed in the function bar 105. For example, the other function options may include the current focal length magnification (such as the current magnification is 1×, and the optional magnification is 2×). For another example, the other function options may include other shooting modes (such as video mode, HDR mode), etc.

[0068] refer to Figure 2 , is a schematic diagram of the internal interaction of an electronic device. Figure 2 As shown in the interaction example, the electronic device can provide the user with a corresponding shooting function when the user inputs operation 103.

[0069] like Figure 2 As shown, after receiving the user's operation 103, the camera application sends an instruction 21 to the camera abstraction (Camera HAL) configured in the electronic device.

[0070] Exemplarily, the instruction 21 may include operation information corresponding to the operation 103 .

[0071] In some implementations, the electronic device may be configured with multiple camera modules. Different camera modules may be configured with different camera identifiers, so that by carrying the camera identifier in the command, the electronic device can correctly call the corresponding camera module to work.

[0072] Exemplarily, in some embodiments, the camera abstraction in the electronic device can determine the camera identifier of the camera module that needs to be pulled up according to the operation information carried in the instruction 21. In other implementations, the operation information can also include the camera identifier of the camera module that needs to be pulled up.

[0073] The camera abstraction may also determine that the image sensor needs to be configured as modeA or modeB for image output. In the present application, the frame rate of modeA may also be referred to as the second frame rate, and the frame rate of modeB may also be referred to as the first frame rate. When modeA is used for image output, the interval between two adjacent frames of images is the first duration, and when modeB is used for image output, the interval between two adjacent frames of images is the second duration.

[0074] Then, the camera abstraction may send an instruction 22 to the camera driver corresponding to the camera module to be pulled up. The instruction 22 may carry a configuration parameter of modeA or modeB, which is used to instruct the image sensor to output the original image according to the mode corresponding to the identifier.

[0075] When multiple camera modules are configured in an electronic device, each camera module may be respectively configured with a corresponding camera driver. The camera driver may be a function mapping of the corresponding camera module in the electronic device. When the electronic device needs to use a certain camera module, it may send relevant instructions to the corresponding camera driver.

[0076] In this way, the camera abstraction can send the instruction 22 to the camera driver of the camera module that needs to be pulled up. The camera driver can control the corresponding camera module to work according to the instruction 22. For example, the camera driver can send the instruction 22 to the image sensor of the camera module to control the image sensor to output the image.

[0077] It should be noted that one or more registers may be configured in the camera driver for storing various configuration parameters required for controlling the image sensor to output images.

[0078] Exemplary, reference Figure 3 , the camera driver may be configured with a register 31. After receiving the instruction 22, the camera driver may write the configuration parameters carried by the instruction 22 into the register 31. For example, the configuration parameters carried by the instruction 22 may include the configuration parameter A of modeA. For example, the configuration parameter A of modeA may include a frame rate of f0 and an image size S1 corresponding to modeA.

[0079] Thus, the camera driver can be Figure 3 The configuration parameter A in the register 31 shown controls the image sensor to output images.

[0080] For example, under the control of the camera driver, the image sensor can obtain the light signal from the camera and output the raw image data 23 according to modeA.

[0081] In such Figure 2In the example of , the image sensor can directly transmit the raw image data 23 to an image signal processor (ISP) for subsequent processing.

[0082] In other embodiments, the image sensor may send the raw image data 23 to a camera driver, and the camera driver may transmit the raw image data 23 to the ISP for processing.

[0083] Correspondingly, the ISP can process the received original image data 23 to obtain processed image data 24. The electronic device can display the image data 24 on the display screen. For example, the image data 24 can be displayed as follows: Figure 1 A preview image 106 is shown.

[0084] In this way, the image sensor can output one frame of image based on mode A. After that, the image sensor can continuously output images at the frame rate f0. Correspondingly, the electronic device can continuously display multiple frame images on the interface to form a preview stream.

[0085] In some cases, the electronic device may control the image sensor output mode to switch from modeA to modeB.

[0086] Exemplarily, the electronic device may switch from mode A to mode B through decision making when any of the following situations occurs:

[0087] Switching to a different focal length ratio, such as switching from 1× to 2×; that is, the electronic device receives an instruction input by the user to perform a focal length adjustment operation;

[0088] Switching to a different shooting mode, such as switching from a photo mode to a video mode; that is, the electronic device receives an instruction input by a user to perform an operation of switching the shooting mode.

[0089] As an example, Figure 4 Provides an example of interface interaction for focus adjustment and shooting mode switching.

[0090] like Figure 4 As shown, when the user inputs operation 401 to indicate adjusting the focal length magnification to 2×, the electronic device can determine to switch to a different focal length magnification according to operation 401 .

[0091] like Figure 4 As shown, in the case where the user inputs operation 402 to instruct to switch the shooting mode to the video mode, the electronic device can determine to switch to a different shooting mode according to operation 401.

[0092] In some other implementations, the electronic device may determine to switch from mode A to mode B through decision making according to current environmental parameters.

[0093] The current environmental parameters may include 3A data of the acquired image. The 3A data refers to the automatic exposure (AE) data, automatic white balance (AWB), and automatic focus (AF) data acquired by the ISP during the processing of the original image data. For example, the electronic device may determine whether to switch from mode A to mode B based on the AE gain and AE exposure duration.

[0094] As an example, when the electronic device receives operation 401 or operation 402, it determines that the AE gain of the acquired image is less than a preset gain threshold, and / or determines that the AE exposure time of the acquired image is less than a preset time threshold, and then determines that it is necessary to switch from mode A to mode B.

[0095] In the above implementation, the electronic device decides to switch from mode A to mode B according to the user input operation and the 3A data of the acquired image. In other implementations, the electronic device can also switch from mode A to mode B by itself.

[0096] Exemplarily, the electronic device may determine to switch from mode A to mode B when triggering to enter an HDR scene.

[0097] Among them, the electronic device can enter the HDR scene under the instruction of the user; or the electronic device can trigger itself to enter the HDR scene according to the high-light environment and low-light environment included in the current captured image.

[0098] There are many different existing solutions for switching from mode A to mode B.

[0099] In some implementations, the camera abstraction may send a stop flow instruction to the camera driver. The stop flow instruction may be used to instruct to stop outputting images based on the current mode (such as modeA). Correspondingly, after receiving the stop flow instruction, the camera driver controls the image sensor to stop outputting images.

[0100] Afterwards, the camera abstraction may send a stream start instruction to the camera driver. The stream start instruction may carry the configuration parameter B of modeB. Thus, the camera driver may control the image sensor to start outputting images based on modeB according to the configuration parameter B.

[0101] During the implementation of this solution, after the camera driver receives the stop streaming instruction and before receiving the start streaming instruction, the image sensor will stop outputting images, which will cause the preview image display to be interrupted during the switching process.

[0102] In some other implementation scenarios, the camera abstraction can directly send the configuration parameters of mode B to the camera driver. This scenario can also be referred to as the seamless switching scenario. In this seamless switching scenario, the camera driver can be configured to control the image sensor to output images according to the latest received configuration parameters (such as configuration parameter B).

[0103] Thus, after the image sensor receives configuration parameter B, it can output images according to configuration parameter B. For example, output images according to the frame rate f1 indicated by configuration parameter B.

[0104] It can be understood that when the frame rate of the image sensor's output images is greater than or equal to the display frame rate of the electronic device's display screen, the preview stream displayed on the display screen will not freeze or interrupt.

[0105] As the display frame rate of the electronic device increases, users are more sensitive to changes in the frame rate of the image sensor's output images.

[0106] Based on this seamless switching scenario, even if the image sensor directly switches the output images from configuration parameter A after receiving configuration parameter B, problems such as freezing of the preview stream may occur due to the frame rate difference before and after the switch.

[0107] This is caused by the mechanism for controlling the output images of the image sensor.

[0108] Exemplarily, in combination with Figure 3 , refer to Figure 5 . Before receiving configuration parameter B, register 31 can store the effective configuration parameter A. Correspondingly, the image sensor can output images according to the frame rate f0 indicated by this configuration parameter A.

[0109] When receiving configuration parameter B, the image sensor stops outputting images according to frame rate f0. For example, the camera driver corresponding to the image sensor can configure the configuration parameter in register 31 as unavailable or delete it.

[0110] Then, the camera driver can write configuration parameter B into register 31. After configuration parameter B is written into the register, the image sensor can output images according to the frame rate f1 indicated by this configuration parameter B.

[0111] Since there is a certain time consumption in the processes such as data modification, deletion, and writing in register 31 before the new configuration parameter takes effect, the image sensor cannot output images normally during this configuration adjustment process. In addition, for the image sensor, the output images in different modes involve changes in internal configurations. When switching from mode A to mode B, it also takes a certain amount of time for the configuration corresponding to mode B to take effect before it can output images according to the configuration parameters corresponding to mode B.

[0112] As a result, the preview stream may become stuck or interrupted.

[0113] Combination Figure 6 The frame image output timing provided specifically explains the problems in the above process.

[0114] like Figure 6 As shown, when the image sensor outputs the N-1th frame image and the previous N-2nd and N-3th frames, the configuration parameter A takes effect. Then, the frame rate corresponding to the output interval of the N-3th frame image and the N-2th frame image is frame rate f0. The frame rate corresponding to the output interval of the N-2th frame image and the N-1th frame image is frame rate f0.

[0115] After the N-1th frame is output, the camera driver of the image sensor receives the configuration parameter B. In this way, the camera driver can Figure 5 In the implementation shown, the configuration parameter B is updated to the register 31. After the configuration parameter B is updated, the image sensor can output the Nth frame image and subsequent frame images according to the new configuration parameter B. For example, the output interval of the Nth frame image and the N+1th frame image can correspond to the frame rate f1. The output interval of the N+1th frame image and the N+2th frame image can correspond to the frame rate f1.

[0116] After the N-1th frame image is output and before the Nth frame image is output, due to the configuration adjustment process, the frame rate corresponding to the output interval between the Nth frame image and the N-1th frame image is significantly lower than f0 or f1. For example, the frame rate corresponding to the output interval between the Nth frame image and the N-1th frame image is f2.

[0117] Therefore, if Figure 6 As shown in the figure, even if the seamless switching solution is effective, an additional frame rate f2 will appear during the configuration adjustment process. The image displayed on the display will also jump from frame rate f0 to frame rate f2 to frame rate f1, which will be manifested as a freeze in the preview stream.

[0118] It should be noted that in some implementations, f0 and f1 are different. In other implementations, f0 and f1 may be the same. In this way, due to the appearance of frame rate f2, the frame rate jump will also occur during the switching of modeA to modeB, which will be manifested as a freeze in the preview stream.

[0119] Based on this, the technical solution provided in the embodiment of the present application enables the electronic device to control the image sensor to continue to output at least one frame of image according to the effective configuration parameters before the new configuration parameters take effect when configuration parameter adjustment is required.

[0120] Exemplary, reference Figure 7, when the solution provided in the embodiment of the present application takes effect, after the N-1th frame is output, when the configuration parameter B is received, the image sensor can continue to output the Nth frame image at the frame rate f0. As a result, the output interval between the N-1th frame image and the Nth frame image is maintained at the frame rate f0. As a result, after the configuration parameter B takes effect, the N-1th frame image is directly output at the new frame rate f1.

[0121] It can be seen that when the solution provided in the present application is effective, the output interval between the N-1th frame image and the Nth frame image is kept at f0, and the following will not occur: Figure 6 As shown, the frame rate changes to f2. As a result, the frame rate of the continuous images only changes from f0 to f1, thus avoiding the following Figure 6 Display stuttering issue that occurs in the situation shown.

[0122] It should be noted that, in the present application, the size relationship between f0 and f1 is not limited. By implementing this solution, the appearance of frame rate f2 is avoided, so whether f0 is the same as f1 or not, the problem of frame rate jump caused by the appearance of frame rate f2 can be solved.

[0123] The solution provided in the embodiments of the present application is described in detail below.

[0124] It should be noted that the electronic device in the embodiment of the present application may include at least one of a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device, or a smart city device. The embodiment of the present application does not impose any special restrictions on the specific type of the electronic device.

[0125] refer to Figure 8 , which is a schematic diagram of the software composition of an electronic device provided in an embodiment of the present application.

[0126] In this example, the software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device is illustrated.

[0127] like Figure 8 As shown in FIG. 1 , the layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, The system is divided into five layers, from top to bottom: application layer, application framework layer, Android runtime (ART) and native C / C++ library, hardware abstraction layer (HAL) and kernel layer.

[0128] The application layer can include a series of application packages.

[0129] like Figure 8 As shown, the application package may include applications such as camera, calendar, map, WLAN, music, SMS, call, navigation, Bluetooth, video, etc.

[0130] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0131] like Figure 8 As shown, the application framework layer may include a window manager, a content provider, a view system, a resource manager, a notification manager, an activity manager, an input manager, and the like.

[0132] The window manager provides window management services (WMS). WMS can be used for window management, window animation management, surface management, and as a transfer station for the input system.

[0133] Content providers are used to store and retrieve data and make it accessible to applications. This data can include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0134] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0135] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0136] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0137] The activity manager can provide activity management services (AMS), which can be used to start, switch, and schedule system components (such as activities, services, content providers, and broadcast receivers) as well as manage and schedule application processes.

[0138] The input manager can provide input management services (IMS), which can be used to manage system input, such as touch screen input, key input, sensor input, etc. IMS takes events from input device nodes and distributes them to appropriate windows through interaction with WMS.

[0139] The Android runtime includes the core library and the Android runtime. The Android runtime is responsible for converting source code into machine code. The Android runtime mainly uses the ahead of time (AOT) compilation technology and the just in time (JIT) compilation technology.

[0140] The core library is mainly used to provide basic Java class library functions, such as basic data structures, mathematics, IO, tools, databases, networks, etc. The core library provides an API for users to develop Android applications.

[0141] The native C / C++ library can include multiple functional modules, such as surface manager, media framework, libc, OpenGL ES, SQLite, Webkit, etc.

[0142] Among them, the surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media framework supports playback and recording of multiple common audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. OpenGL ES provides drawing and operation of 2D and 3D graphics in applications. SQLite provides a lightweight relational database for electronic device applications.

[0143] The hardware abstraction layer runs in user space, encapsulates the kernel layer driver, and provides a calling interface to the upper layer. Exemplarily, the hardware abstraction layer may include a display module, an audio module, a camera module, a Bluetooth module, etc. In some embodiments, the camera module may also be referred to as a camera abstraction.

[0144] The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a Bluetooth driver. In combination with the above description, in this application, when multiple camera modules are configured in an electronic device, each camera module can be configured with a corresponding camera driver.

[0145] It should be noted that the above Figure 8 The electronic device components provided are only examples and do not constitute a limitation on the electronic device involved in the embodiments of the present application. In other embodiments, the electronic device may also have other software components.

[0146] Exemplary, reference Fig. 9 , is a schematic diagram of the composition of another electronic device provided in an embodiment of the present application.

[0147] In this Fig. 9 In the example, the software system running in the processor can be divided into user space and kernel space. Figure 8 As shown in the software architecture, the user space can include Figure 8 In some embodiments, the hardware abstraction layer Figure 8 The application layer, application framework layer, and native C / C++ library shown in FIG. 1 can also be considered to be configured in the user space. Fig. 9 The kernel space shown can correspond to Figure 8 The kernel layer is shown.

[0148] like Fig. 9 As shown, the user space and kernel space of the electronic device can be configured in the processor in the form of software modules.

[0149] Among them, the user space can be configured with camera applications and camera abstraction.

[0150] The camera abstraction can include decision modules, as well as sensor nodes.

[0151] Among them, the decision module can also be called an AEC decision module.

[0152] The decision module can be used to obtain the operation information of the user's current input operation from the camera application. For example, the operation information may include: Figure 1 The operation information of operation 103 shown, Figure 4 The operation information of operation 401 shown, Figure 4 Operation information of operation 402 shown, etc.

[0153] The decision module can also be used to obtain 3A data of the last frame of image acquired from the ISP. For example, the decision module can obtain data such as AE exposure duration, AE gain, etc. of the last frame of image acquired from the IFE of the ISP.

[0154] The decision module can also be used to decide whether to switch between different image output modes based on the information already obtained. For example, the switching between different image output modes can include switching from modeA to modeB, or from modeB to modeA.

[0155] A sensor node is also called a sensor node. The sensor node may specifically include an enabling control module and an exposure control module.

[0156] Among them, the enabling control module can also be called a reshutter control module. The enabling control module can be used to determine the target image sensor corresponding to the camera module that currently needs to be pulled up. The enabling control module can also be used to determine whether the target image sensor supports the reshutter function. In the embodiment of the present application, the reshutter function can also be called the first function.

[0157] In an embodiment of the present application, supporting the reshutter function may correspond to: the image sensor may be configured to, after receiving a new image output configuration, continue to perform image output processing of at least one frame of image corresponding to the existing image output configuration.

[0158] The enabling control module is also used to transmit relevant control information to the camera driver of the target image sensor. For example, taking the target sensor supporting the reshutter function as an example, the control information may include: a reshutter enabling flag, and a configuration parameter B of modeB.

[0159] like Fig. 9 As shown, the sensor node may also include an exposure control module.

[0160] In some embodiments of the present application, the exposure control module can be used to control the frame length of the first frame image after switching between image output modes when switching between image output modes is executed.

[0161] Exemplarily, the exposure control module can control the exposure time of the first frame image after switching to modeB according to the configuration parameter B of modeB and the configuration parameter A of modeA. As a result, after the image output mode is switched, the frame length of the first frame image output by the image sensor is the same or similar to the frame length of the last frame image before the image output mode is switched. In some implementations, two frame lengths can be considered similar if the frame length difference is within 10%. In this way, it can be ensured that the frame rate of the output image will not change significantly before and after the image output mode is switched.

[0162] It should be noted that in some embodiments of the present application, the exposure control module may also be omitted. Thus, the control module may be enabled to achieve no frame rate jump after the image output mode is switched.

[0163] like Fig. 9 As shown, the camera driver and ISP can be configured in the kernel space.

[0164] The camera driver can be used to transmit commands between the relevant modules in the user space and the image sensor. If multiple camera modules are configured in the electronic device, multiple camera drivers can be configured in the kernel space accordingly.

[0165] It should be noted that, in the embodiment of the present application, the image sensor and the camera may together constitute a camera module. The camera module may include one or more camera modules.

[0166] The number of image sensors (eg, N) and the number of cameras (eg, M) configured in the electronic device may be the same or different.

[0167] Correspondingly, the configuration of the camera driver can be configured in a one-to-one correspondence with the image sensor. Alternatively, the configuration of the camera driver can be configured in a one-to-one correspondence with the camera.

[0168] In the following description, a one-to-one configuration of a camera driver and an image sensor is taken as an example.

[0169] In this application, if Fig.10 As shown, for an image sensor supporting the reshutter function, at least three registers may be configured in the corresponding camera driver, such as a GPH register (or first register), a reshutter register (or second register), and a configuration register (or third register).

[0170] The GPH register is used to enable the simultaneous writing of multiple sets of data corresponding to the same frame. In other words, the GPH register is used to enable the configuration of the reshutter register and the configuration register.

[0171] The reshutter register is used to enable the reshutter function. The configuration register is used to store the configuration parameters of the output mode. For example, the storage area A of the configuration register is used to store the configuration parameters that have taken effect (such as the configuration parameter A of modeA). The storage area B of the configuration register is used to store the configuration parameters to be taken effect (such as the configuration parameter B of modeB).

[0172] The camera driver is also used to control the image sensor to output images according to the configuration of the register. For example, when the reshutter register is configured to 1 (or the first value), the camera driver can control the image sensor to output the next frame of image according to the configuration parameters of storage area A in the configuration register. The camera driver can also control the image sensor to output subsequent images according to the configuration parameters of storage area B in the configuration register.

[0173] In some implementations, the ISP may include an image front end processing unit (Image Front End, IFE) and an image processing engine (Image processing Engine, IPE).

[0174] Among them, IFE is mainly responsible for the pre-processing of the image data collected by the camera. For example, color processing, denoising, enhancement, sharpening, etc. The output of IFE is the pre-processed RAW image data. In this application, 3A data such as AE exposure time, AE gain, etc. can be obtained after IFE processing. IFE can also be used to send 3A data to the decision module.

[0175] IPE is the main processing unit in ISP, responsible for performing various algorithm processing on the image data output by IFE, such as white balance, color correction, noise reduction, etc. The data output after IPE processing can be used as the basis for electronic devices to send and display data.

[0176] The technical solutions provided in the embodiments of this application can be applied to Figure 8 or Fig. 9 In the electronic device shown.

[0177] In the following example, combined with Fig. 9 The software composition shown is used to provide a detailed description of the implementation scheme provided in the embodiment of the present application.

[0178] refer to Fig.11, which is a schematic diagram of an interaction between modules provided in an embodiment of the present application. In the implementation of this solution, it is taken as an example that the currently used camera module supports the reshutter function. Before the N-1 frame image is output, the electronic device determines to use modeA for output as an example. In this way, the GPH register in the camera driver can be configured to 0, the reshutter register can be configured to 0, and the configuration parameter A can be stored in the configuration register.

[0179] Through this Fig.11 The process shown can realize outputting the N-1th frame image with the configuration parameter A (such as frame rate f0, etc.) corresponding to modeA.

[0180] like Fig.11 As shown, the program may include:

[0181] S1101, the camera driver sends an image output instruction 111 to the image sensor.

[0182] The image output instruction 111 may correspond to the configuration parameter A. For example, the image output instruction 111 may include the frame rate f0 and the image size S1 corresponding to the configuration parameter A.

[0183] S1102 , the image sensor sends raw image data 112 to the IFE according to the image output instruction 111 .

[0184] The original image data 112 may correspond to the original image of the N-1th frame image.

[0185] In this example, the image sensor can process the light signal from the camera according to the image size S1 carried by the image output instruction 111, and output the original image data 112 corresponding to the image size S1. In addition, the frequency at which the image sensor sends the original image data 112 to the IFE can correspond to the frame rate f0.

[0186] S1103. IFE sends reference data 113 to the decision module.

[0187] The reference data 113 may include 3A data corresponding to the N-1th frame image. For example, the reference data 113 may include AE ​​exposure duration and / or AE gain corresponding to the N-1th frame image.

[0188] Exemplarily, the IFE may perform preliminary processing on the received raw image data 112 , thereby obtaining 3A data of the N-1th frame image.

[0189] In this example, the IFE can send the 3A data of the corresponding frame image obtained after each frame processing to the decision module, so that the decision module can determine whether to trigger the adjustment of the image output mode in the subsequent frame image output process in combination with the 3A data of the frame image that has been obtained.

[0190] S1104: The IFE sends the image data 114 to the IPE. The image data 114 may be the data of the N-1th frame of image after preliminary processing by the IFE.

[0191] S1105 . The IPE generates image data 115 based on the image data 114 and outputs the image data 115 .

[0192] In this way, through S1104-S1105, the ISP processing of the N-1th frame image is implemented, and then the image data 114 corresponding to the N-1th frame image that can be used for display is obtained.

[0193] In this example, the electronic device determines to switch from mode A to mode B for image output according to user operations and environmental parameters after the electronic device outputs the N-1th frame image and before the electronic device outputs the Nth frame image.

[0194] refer to Fig.12 , which is a schematic diagram of another type of module interaction provided in an embodiment of the present application. Fig.12 The process shown in the figure can be implemented so that after receiving the mode switching instruction, the camera driver of the image sensor can continue to control the image sensor to output at least one frame according to the effective modeA. That is, the Nth frame image is output according to the frame rate f0 indicated by modeA, without waiting for modeB to take effect before outputting the image according to the frame rate f1 of modeB. This avoids waiting before the Nth frame image is output.

[0195] like Fig.12 As shown, the program may include:

[0196] S1201. The camera application sends operation information 121 to the decision module.

[0197] Exemplarily, after receiving the user's operation 401 or operation 402, the camera application may generate corresponding operation information and send it to the decision module.

[0198] Take the case where the camera application receives the user's operation 401 as an example. Figure 4 According to the description in FIG. 4 , operation 401 is used to instruct to adjust the focal length magnification to 2×.

[0199] Correspondingly, the camera application may generate operation information 121 according to operation 401 and send it to the decision module.

[0200] The operation information 121 may include information indicating that the focal length magnification is adjusted from 1× to 2×.

[0201] S1202: The decision module sends configuration parameter B and camera identifier C1 to the enabling control module. In some embodiments, the configuration parameter B and camera identifier C1 may be carried in the first control information and sent down. In other embodiments, the first control information may also only include the frame rate f1 in the configuration parameter B, or the first control information may include the frame rate f1 and camera identifier C1.

[0202] The camera identifier C1 may be the identifier of the currently called camera.

[0203] In some embodiments, the decision module may obtain the camera identifier C1 from a camera application. In other embodiments, the decision module may obtain the camera identifier C1 from other modules (such as a camera management module in a framework layer of an electronic device).

[0204] In this example, the decision module can be based on the operation information 121, as well as Fig.11 The reference data 113 obtained in S1103 determines that the output mode needs to be switched from modeA to modeB.

[0205] Exemplarily, the decision module may determine to switch from modeA to modeB according to the current operation information indicated by the operation information 121, including: the focal length magnification is adjusted from 1× to 2×, and the 3A data corresponding to the previous frame image meets the preset conditions.

[0206] The preset conditions may include: the AE gain of the previous frame image is less than a preset gain threshold, and / or the AE exposure time of the previous frame image is less than a preset time threshold.

[0207] Take the case where the reference data 113 meets the preset condition as an example.

[0208] In this way, the decision module can determine that it is necessary to switch from modeA to modeB, and control the image sensor to output images according to modeB.

[0209] It should be noted that the judgment condition provided in this example for the decision module to determine that it is necessary to switch from modeA to modeB is only an example. In other implementations, the decision module may also determine that it is necessary to switch from modeA to modeB based on other mechanisms. The embodiments of the present application are not limited to this.

[0210] In this way, when the decision module determines that modeA needs to be switched to modeB for image output, the decision module can send the configuration parameter B of modeB and the camera identifier C1 currently in use to the enabling control module.

[0211] S1203 . The enabling control module sends the configuration parameter B and the reshutter enabling flag to the camera driver.

[0212] Exemplarily, when the enabling control module receives the configuration parameter B and the camera identifier C1, it can know that modeA needs to be switched to modeB for image output.

[0213] In the present application, the enabling control module can determine whether the image sensor corresponding to the currently used camera supports the reshutter function according to the camera identifier C1.

[0214] In some implementations, a reshutter list may be configured in the enabling control module, and the reshutter list may also be referred to as a first function list.

[0215] The reshutter list may include at least one camera identifier.

[0216] Take the correspondence between at least one camera identifier and an image sensor including the camera identifier C1 as an example. This indicates that the image sensor used by the camera corresponding to the camera identifier C1 supports the reshutter function. That is, the image sensor applicable to the camera corresponding to the camera identifier C1 can continue to output at least one frame according to the configuration parameter A under the control of the camera driver.

[0217] Thus, the enabling control module can query the camera identifier C1 in the reshutter list. If the camera identifier C1 is included in the reshutter list, it is determined that the currently used image sensor supports the reshutter function.

[0218] It is understandable that when the electronic device is turned on, all hardware functions will be traversed. Thus, in other embodiments of the present application, the reshutter list can be generated and stored after the electronic device completes the function traversal of all cameras and corresponding image sensors.

[0219] In some other embodiments of the present application, the reshutter list may also be configured and stored when the electronic device leaves the factory.

[0220] In other embodiments of the present application, the reshutter list may also be flexibly adjusted according to actual conditions.

[0221] Therefore, when the enabling control module determines that the currently used image sensor supports the reshutter function, the reshutter enabling flag can be sent to the camera driver to enable the camera driver to control the image sensor to continue to output at least one frame of image according to the configuration parameter A. In the following example, enabling the camera driver to control the image sensor to continue to output one frame of image according to the configuration parameter A is taken as an example.

[0222] In addition, the enabling control module may also send new configuration parameters B to the camera driver.

[0223] like Fig.12 As shown, the camera driver can configure the corresponding register according to the received reshutter enable flag.

[0224] Exemplarily, after receiving the reshutter enable flag, the camera driver may set the GPH register to 1, set the reshutter register to 1, and store the configuration parameter B in the configuration register.

[0225] The GPH register is set to 1, which corresponds to the need to configure the parameters of the reshutter register and the configuration register. The reshutter register is configured to 1, which corresponds to the Nth frame of the image to be output continuing to be output according to the existing configuration. The existing configuration can be stored in the storage area A in the configuration register, such as configuration parameter A.

[0226] Configuration parameter B may be stored in storage area B of the configuration register.

[0227] In some implementations, after completing the data storage of the reshutter register and the configuration register, the camera driver can reset the GPH register to 0.

[0228] It should be noted that in some other embodiments of the present application, if the enabling control module determines that the corresponding image sensor does not support the reshutter function according to the camera identifier C1, the reshutter enabling identifier may not be generated, and only the configuration parameter B is sent to the camera driver. Figure 3 or Figure 5 The solution shown is implemented to control the image sensor to output images.

[0229] S1204: The camera driver sends an image output instruction 122 to the image sensor.

[0230] Exemplarily, the camera driver may be configured as 1 according to the reshutter register, and does not wait for the new configuration parameter B to take effect, but outputs the next frame of image (such as the Nth frame of image) according to the existing configuration parameter A.

[0231] As a possible implementation, after the GPH register is configured to 0 (ie, the register is configured), the camera driver can read the existing configuration parameter A from the storage area A of the configuration register according to the reshutter register being configured to 1.

[0232] The camera driver can generate an image output instruction 122 according to the configuration parameter A, instructing the image sensor to output the image at a frame rate f0 and an image size S1.

[0233] Correspondingly, the image sensor can generate an Nth frame image for output. The Nth frame image can be output at a frame rate f0, and the size of the Nth frame image can correspond to the image size S1.

[0234] S1205 . The image sensor sends raw image data 123 to the IFE.

[0235] S1206. IFE sends reference data 124 to the decision module.

[0236] Similar to Fig.11 In S1103, the IFE may send the 3A data of the Nth frame image (such as the AE exposure time and / or AE gain of the Nth frame image) to the decision module so that the decision module can determine the output mode of subsequent frame images accordingly.

[0237] S1207: The IFE sends the image data 125 to the IPE. The image data 125 may be image data obtained after the IFE processes the original image data 123.

[0238] S1208, the IPE generates and outputs image data 126 according to the image data 125. The image data 126 can be used for displaying the Nth frame image.

[0239] In this way, before the Nth frame is output, if the camera driver of the image sensor receives the output mode adjustment instruction, the Nth frame can still be output according to the existing modeA. Figure 7 The effect shown.

[0240] It is understandable that in some embodiments of the present application, after receiving the configuration parameter B (i.e., the new image output mode) from the decision module, the enabling control module can control the subsequent Nth frame image to the N+xth frame image to continue to output images according to the existing modeA through similar logic from S1203 to S1208. In this way, it can be ensured that after the modeB configuration takes effect, the image can be output according to modeB immediately. The time of the x-frame image that continues to output images according to modeA is filled as follows: Figure 6The time period in which the frame rate f2 appears is avoided, thereby avoiding the frame rate jumping to the frame rate f2 between switching from f0 to f1.

[0241] In different implementations, x can be flexibly configured to be 1 or a positive integer greater than 1. In this application, x is configured to be 1 as an example. Therefore, after the Nth frame image is output, the configuration parameter B has taken effect. That is, after the Nth frame image is output, the image sensor can immediately output the image at the frame rate f1 corresponding to the configuration parameter B under the control of the camera driver.

[0242] Therefore, after the Nth frame image is output, the electronic device can drive and control the image sensor through the camera to output subsequent frame images (such as the N+1th frame image) according to the new configuration parameter B.

[0243] Exemplarily, in some embodiments, this process can be implemented by configuring registers in the camera driver.

[0244] For example, refer to Fig.13 After the Nth frame image is output (e.g., after the camera driver sends the output instruction 122 to the image sensor), the camera driver can configure the GPH register to 0, configure the reshutter register to 0, and delete the configuration parameter A in the configuration register. Optionally, the camera driver can store the configuration parameter B in the storage space A.

[0245] In this way, according to the modified register status, the camera driver can follow similar Fig.11 The solution shown is implemented to control the image sensor to output subsequent images (such as the N+1th frame image) according to the configuration parameter B.

[0246] Exemplary, reference Fig.14 , which is a schematic diagram of another type of module interaction provided in an embodiment of the present application. Fig.14 The process shown is implemented so that the camera driver can control the image sensor to output images based on modeB according to the new effective configuration parameter B.

[0247] like Fig.14 As shown, the program may include:

[0248] S1401, the camera driver sends an image output instruction 141 to the image sensor.

[0249] The image output instruction 141 may correspond to the configuration parameter B. For example, the image output instruction 141 may include the frame rate f1 and the image size S2 corresponding to the configuration parameter B.

[0250] S1402: The image sensor sends original image data 142 to the IFE according to the image output instruction 141. The original image data 142 may correspond to the original image of the N+1th frame image.

[0251] S1403, IFE sends reference data 143 to the decision module. The reference data 143 may include 3A data corresponding to the N+1th frame image. For example, the reference data 143 may include AE ​​exposure duration and / or AE gain corresponding to the N+1th frame image.

[0252] S1404: The IFE sends the image data 144 to the IPE. The image data 144 may be the data of the N+1th frame image after the IFE performs preliminary processing.

[0253] S1405 . IPE generates image data 145 based on image data 144 and outputs the image data.

[0254] In this way, the output of the N+1th frame image is achieved according to the configuration parameters of the new modeB.

[0255] It should be noted that in some other embodiments of the present application, the electronic device can also control the exposure time of the first frame image after the new configuration parameters take effect by configuring the exposure control module in the sensor node. In this way, the frame length of the first frame image after the new configuration parameters take effect is limited, thereby ensuring that the frame rate displayed on the interface before and after the image output mode is switched will not change too much.

[0256] Exemplary, combined Fig.12 ,refer to Fig.15 , which is a schematic diagram of another type of interaction between modules provided in an embodiment of the present application.

[0257] The Fig.15 In the example, the electronic device outputs the Nth frame image according to the frame rate f0 of modeA according to S1201-S1208, and can also configure the exposure time of the N+1th frame image in the camera drive through S1501-S1502.

[0258] For example, Fig.15 As shown, after the enabling control module receives the configuration parameter B and the camera identifier C1 according to S1202, it can be as follows Fig.12 The logic shown determines that the image sensor supports the reshutter function based on the camera identifier C1.

[0259] In addition to executing S1203, the enabling control module may also execute the following S1501.

[0260] S1501. The enabling control module sends a reshutter enabling flag to the exposure control module.

[0261] Exemplarily, the enabling control module may trigger the exposure control module to perform exposure control on the first frame image (eg, the N+1th frame image) after switching to the image output mode by sending a reshutter enabling flag to the exposure control module.

[0262] In this example, the exposure control module may determine the exposure limit parameter of the N+1th frame image when receiving the reshutter enable flag.

[0263] As an example, the exposure control module can determine the exposure limit parameters of the first frame image (such as the N+1th frame image) after modeB takes effect based on the configuration parameters of the effective modeA and the configuration parameters of the to-be-effective modeB.

[0264] For example, the exposure control module may determine the exposure limit parameter according to the following formula (1).

[0265] Formula (1): Ts=(FLL(A)-(M+y_add_end(A)-y_add_sta(A)+65) / N+96)*LLP(A) / LLP(B)-96.

[0266] Where, Ts is the exposure limit parameter, FLL(A) is the frame length of mode A, y_add_end(A) is the end position of the image height coordinate of mode A, y_add_sta(A) is the start position of the image height coordinate of mode A, LLP(A) is the line length of mode A, and LLP(B) is the line length of mode B. M and N are both fixed coefficients. For example, when mode A is full-size, M=4, N=1. For another example, when mode A is not full-size, M=8, N=2.

[0267] Therefore, by controlling the exposure time of the N+1th frame image to be less than the exposure limit parameter, it can be ensured that the frame length of the first frame after switching to mode B is the same as or similar to the frame length of mode A.

[0268] Exemplarily, the enabling control module may implement exposure control of the N+1th frame image according to the exposure limit parameters according to the following S1502 and related logics.

[0269] S1502: The exposure control module sends exposure limit parameters to the camera driver.

[0270] The camera driver may be the camera driver corresponding to the currently used camera identifier C1. In some implementations, the exposure control module may obtain the camera identifier C1 from the enabling control module. Fig.15 not shown).

[0271] In this way, the camera driver can end the configuration parameter B and the reshutter enabling flag according to S1203, and can also receive the exposure limit parameter according to S1502.

[0272] Combination Fig.12 Instructions on how to update the camera driver registers are as follows: Fig.15 In the example of Fig.12 The solution shown is implemented to update the GPH register, reshutter register and configuration register.

[0273] In addition, in this Fig.15 In the example of FIG. 4 , the camera driver may also store the acquired exposure limit parameters in the configuration register.

[0274] Exemplarily, the camera driver may store the exposure limit parameter and the configuration parameter B together in the storage area B. So that when the N+1th frame image is output, the exposure limit parameter and the configuration parameter B take effect together.

[0275] Therefore, the camera driver follows the Fig.14 The scheme shown is implemented, when sending the image output instruction 141 to the image sensor, in addition to carrying the frame rate f1 and image size S2 corresponding to the configuration parameter B, the exposure limit parameter Ts can also be sent to the image sensor. Therefore, when the image sensor outputs the N+1th frame image, the image output is processed according to the configuration parameter B, and the exposure time does not exceed the exposure limit parameter Ts.

[0276] In this way, by enabling the cooperation of the control module and the exposure control module, the Nth frame image can be framed at a frame rate of f0, and the exposure time of the N+1th frame image can be controlled to be less than or equal to the exposure limit parameter Ts, thereby achieving the purpose of controlling the frame length of the N+1th frame image to be equal to or close to the frame length of the Nth frame image.

[0277] It should be noted that, in each embodiment of the present application, the switching of the image output mode from modeA to modeB is used as an example for explanation. In other embodiments of the present application, when the current image output mode is modeB, if the electronic device (such as a decision module configured in the electronic device) determines that modeB needs to be switched to modeA, the above-mentioned Figure 11-Figure 15Any of the technical solutions provided in the foregoing enables the camera driver to continue to control the image sensor to output at least one frame in mode B after receiving the image output mode switching instruction. Fig.15 In the implementation of the provided solution, the exposure control module can determine the corresponding exposure limit parameters in a similar manner to the above formula (1) (such as exchanging the parameters related to modeA with the parameters related to modeB in formula (1)). Based on the exposure limit parameters, the exposure time of the first frame image after switching to modeA is controlled so that the frame length of the first frame image is the same as or similar to the frame length of the image output in modeB. The specific implementation in this scenario can refer to the specific descriptions in the above embodiments, which will not be repeated here.

[0278] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of each functional module. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0279] The above integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0280] For example, Fig.16 FIG. 1 is a schematic diagram showing the composition of an electronic device 1600. Fig.16 As shown, the electronic device 1600 may include: a processor 1601 and a memory 1602. The memory 1602 is used to store computer-executable instructions. Exemplarily, in some embodiments, when the processor 1601 executes the instructions stored in the memory 1602, the electronic device 1600 may execute any of the methods shown in the above embodiments.

[0281] For example, Fig.16 As shown, the electronic device 1600 may further include an image sensor 1603. When the electronic device 1600 executes the method provided in the above embodiment, the image sensor 1603 may be controlled to output corresponding images according to the solution provided in the embodiment of the present application.

[0282] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0283] Fig.17 A schematic diagram of the composition of a chip system 1700 is shown. The chip system 1700 may include: a processor 1701 and a communication interface 1702, which are used to support related devices to implement the functions involved in the above embodiments. In a possible design, the chip system also includes a memory for storing necessary program instructions and data for electronic devices. The chip system may be composed of chips, or may include chips and other discrete devices. It should be noted that in some implementations of the present application, the communication interface 1702 may also be referred to as an interface circuit.

[0284] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0285] The functions or actions or operations or steps in the above embodiments can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or may include one or more servers, data centers and other data storage devices that can be integrated with the medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0286] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A shooting method, It is characterized in that Applied to an electronic device, the electronic device includes a camera and an image sensor, and the method includes: receiving a first user operation; In response to the first user operation, the electronic device opens a camera application; After opening the camera application, the image sensor outputs a first image at a first frame rate; receiving a second user operation; In response to the second user operation, the electronic device sends control information to the image sensor; In response to the control information, the image sensor outputs a second image at a second frame rate, wherein the second image is the next frame image of the first image, and the time interval between starting to output the second image and starting to output the first image is a first duration, and the first duration corresponds to the first frame rate.

2. The method according to claim 1, It is characterized in that The first user operation includes: an operation of a user clicking an icon of the camera application.

3. The method according to claim 1 or 2, It is characterized in that The second user operation includes: an operation of a user switching a focal length magnification of the camera application or an operation of a user switching a shooting mode of the camera application.

4. The method according to claim 1 or 2, It is characterized in that The second user operation includes: a user clicks on a function option of the camera application to switch the focal length magnification from 1X to 2X.

5. The method according to claim 4, It is characterized in that When the focal length magnification of the camera application is 1X, the image sensor outputs images in a first mode, and when the focal length magnification of the camera application is 2X, the image sensor outputs images in a second mode.

6. The method according to claim 5, It is characterized in that The first mode is a binning mode, and the second mode is an InsensorZoom mode.

7. The method according to any one of claims 1 to 3, It is characterized in that The image sensor outputs the first image in a binning mode and outputs the second image in an InsensorZoom mode.

8. The method according to any one of claims 1 to 3, It is characterized in that The image sensor outputs the first image in an InsensorZoom mode and outputs the second image in a binning mode.

9. An electronic device, It is characterized in that The electronic device comprises: a memory and one or more processors; wherein the memory is used to store computer program codes, and the computer program codes comprise computer instructions, and when the processor executes the computer instructions, the electronic device executes the method as claimed in any one of claims 1 to 8.

10. A chip system, It is characterized in that The chip system is applied to an electronic device; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive a signal from a memory of the electronic device and send the signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the method as described in any one of claims 1-8.

Images