Photographing method, apparatus, device, and storage medium

Abstract

This application provides a shooting method, apparatus, device, and storage medium, relating to the field of photography. The method includes: determining a first camera module and a second camera module from a configured set of camera modules; adjusting the exposure frame rate of the first camera module to a first exposure frame rate and the exposure frame rate of the second camera module to a second exposure frame rate; and controlling the second camera module to coordinate with the first camera module for shooting. Specifically, the first camera module is controlled to start shooting a first frame, and the second camera module is controlled to start shooting a second frame. The start time of the second frame is delayed by a first duration compared to the start time of the first frame, and the first duration is related to both the first and second exposure frame rates. In this method, the first and second camera modules can shoot at different exposure frame rates, and their respective exposure outputs can remain synchronized at the end of the frame, ensuring the image output quality while reducing power consumption caused by unnecessarily high frame rates in the camera modules.

Description

[0001] This application claims priority to Chinese Patent Application No. 202111491363.6, filed with the State Intellectual Property Office of China on December 8, 2021, entitled “A Display Method and Electronic Device”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of photography, and more particularly to a photography method, apparatus, device, and storage medium. Background Technology

[0003] In multi-camera collaborative shooting scenarios, terminal devices (such as mobile phones) can be equipped with multiple camera modules. During shooting, the terminal device can simultaneously capture multiple images corresponding to each camera module, and then fuse these images to obtain the final captured image.

[0004] Currently, when a terminal device takes pictures using multiple camera modules, hard synchronization control technology can be used to ensure that the timestamps of the start of exposure of each frame of the multiple camera modules are kept consistent during the shooting stage. This ensures the uniformity of the images captured by the multiple camera modules, thereby improving the output effect of the fused image (i.e., the final captured image).

[0005] However, in current hard synchronization control technology, the slave camera module needs to follow the master camera module and maintain the same exposure frame rate as the master camera module. Maintaining the same exposure frame rate between the slave camera module and the master camera module will result in higher power consumption in the terminal device. Summary of the Invention

[0006] This application provides a shooting method, apparatus, device, and storage medium that can be applied to scenarios where terminal devices configured with multiple (e.g., at least two) camera modules perform multi-camera collaborative shooting. It enables the main camera module and the slave camera modules to shoot at different exposure frame rates, minimizing the power consumption of the slave camera modules due to unnecessary high frame rates, thereby effectively reducing the power consumption of the terminal device in multi-camera collaborative shooting scenarios.

[0007] In a first aspect, this application provides a shooting method, the method comprising: a terminal device determining a first camera module and a second camera module from a configured set of camera modules; the terminal device adjusting the exposure frame rate of the first camera module to a first exposure frame rate and adjusting the exposure frame rate of the second camera module to a second exposure frame rate; the first exposure frame rate and the second exposure frame rate are different; the terminal device controlling the first camera module to shoot at the first exposure frame rate, and controlling the second camera module to perform cooperative shooting with the first camera module at the second exposure frame rate.

[0008] The terminal device controls the first camera module to shoot at a first exposure frame rate and controls the second camera module to shoot in coordination with the first camera module at a second exposure frame rate, including: the terminal device controls the first camera module to start shooting a first frame and controls the second camera module to start shooting a second frame, wherein the start time of the second frame is delayed by a first duration compared to the start time of the first frame, and the first duration is related to the first exposure frame rate and the second exposure frame rate.

[0009] In this shooting method, the first camera module can be considered the main camera module, and the second camera module can be considered the slave camera module. The terminal device configures different exposure frame rates (such as a first exposure frame rate and a second exposure frame rate) for the main and slave camera modules, allowing them to shoot at different exposure frame rates. This minimizes the power consumption of the slave camera module due to unnecessarily high frame rates, thereby effectively reducing the power consumption of the terminal device in multi-camera collaborative shooting scenarios.

[0010] The terminal device controls the first camera module to start capturing the first frame and the second camera module to start capturing the second frame. The start time of the second frame is delayed by a certain duration compared to the start time of the first frame. This first duration is related to both the first and second exposure frame rates. This allows control over the start time of exposure and image capture by the secondary camera module based on the exposure frame rates of the main and secondary camera modules. By controlling the start time of exposure and image capture by the secondary camera module during the shooting process according to the exposure frame rates of the main and secondary camera modules, synchronization between the exposure and image capture of the secondary and main camera modules can be maintained at the end of the frame. This ensures the uniformity of the images captured by the main and secondary camera modules, thereby improving the final image quality.

[0011] In some implementations, the first duration is related to both the first and second exposure frame rates, including: the first duration is equal to the difference between the frame length corresponding to the third exposure frame rate and the frame length corresponding to the second exposure frame rate. The third exposure frame rate is the maximum value among all values ​​that are less than or equal to the second exposure frame rate and divisible by the first exposure frame rate.

[0012] Taking the first exposure frame rate as Sm and the second exposure frame rate as Ss as an example, the third exposure frame rate can be determined by reducing the frame rate downwards from Ss based on Sm to find the first frame rate Ss1 that is divisible by Sm. That is, Ss1 is the maximum value among all values ​​that are less than or equal to Ss and divisible by Sm. The first duration can be the difference between the frame length corresponding to Ss1 and the frame length corresponding to Ss (such as the absolute value of the difference).

[0013] Optionally, the method further includes: the terminal device controlling the first camera module to emit a first signal after a first delay in the start time of the first frame. The terminal device controlling the second camera module to start capturing the second frame includes: the terminal device controlling the second camera module to sense the first signal according to a configured interval number of times the first signal is sensed; and in response to the sensed first signal, the terminal device controlling the second camera module to start capturing the second frame.

[0014] The first signal can be a pulse signal such as an XVS pulse signal, a VSYNC pulse signal, or a FSIN pulse signal. The terminal device controls the first camera module to emit the first signal after a first duration delay from the start time of the first frame. In other words, the terminal device can control the first camera module to emit the first signal at a time delay after the start time of the first frame. For example, assuming the first duration is Δt, and the time corresponding to the frame header of the first frame captured by the first camera module is t, then the actual time when the first camera module emits the first signal is (t + Δt).

[0015] The terminal device controls the second camera module to sense the first signal according to the configured interval number of times the first signal is sensed; in response to the sensed first signal, the terminal device controls the second camera module to start shooting the second frame, which can delay the start time of the second frame shooting by a first duration compared to the first frame, thereby realizing the control of the time when the second camera module starts exposing and outputting the image according to the exposure frame rate of the first camera module and the exposure frame rate of the second camera module.

[0016] Optionally, the method further includes: the terminal device configuring the second camera module to sense the first signal at intervals according to the first exposure frame rate and the second exposure frame rate, and configuring the first camera module to emit the first signal at a time delayed by a first duration from the start time of the first frame.

[0017] As described above, taking a first exposure frame rate of Sm and a second exposure frame rate of Ss as an example, the first duration can be the difference between the frame length corresponding to Ss1 and the frame length corresponding to Ss. The terminal device can configure the first camera module to emit the first signal after a first duration delay from the start time of the first frame by configuring the first camera module to emit the first signal after a first duration delay from the start time of the first frame. The terminal device can also configure the second camera module to sense the first signal at a certain interval by configuring the interval number for sensing the first signal, thereby controlling the second camera module to sense the first signal according to the configured interval number for sensing the first signal.

[0018] In some implementations, the number of intervals at which the second camera module senses the first signal is equal to the quotient of the first exposure frame rate divided by the third exposure frame rate minus 1.

[0019] Again, taking the first exposure frame rate as Sm and the second exposure frame rate as Ss as an example, the number of intervals at which the second camera module senses the first signal can be determined by lowering the frame rate from Sm to Ss1 to find the first third exposure frame rate that is divisible by Sm, and then subtracting 1 from the quotient of Sm divided by Ss1. Alternatively, it can be the value obtained by rounding down the value obtained by dividing Sm by Ss.

[0020] In some implementations, the method further includes: the terminal device determining a third camera module from the configured camera modules; the terminal device adjusting the exposure frame rate of the third camera module such that the exposure frame rate of the third camera module satisfies: the second exposure frame rate is an integer multiple of the exposure frame rate of the third camera module; and the terminal device controlling the third camera module to perform coordinated shooting with the first and second camera modules at the adjusted exposure frame rate.

[0021] The terminal device controls the third camera module to perform coordinated shooting with the first and second camera modules at an adjusted exposure frame rate, including: the terminal device controls the third camera module to sense a first signal according to a configured interval number of times the first signal is sensed. In response to the sensed first signal, the terminal device controls the third camera module to start shooting the current frame.

[0022] The third camera module can also serve as a slave camera module. In multi-camera collaborative shooting scenarios, the number of slave cameras can be one or more. When there is only one slave camera module, there is no third camera module. When there are multiple slave camera modules (e.g., two or more), the minimum exposure frame rates of these multiple slave camera modules may be the same or different. In this case, the second camera module can be the target slave camera module with the highest minimum exposure frame rate among the multiple slave camera modules, and the third camera module can be any of the remaining slave camera modules other than the target slave camera module.

[0023] In this application, the terminal device adjusts the exposure frame rate of the third camera module so that the second exposure frame rate is an integer multiple of the third camera module's exposure frame rate. This allows the third camera module to coordinate with the first and second camera modules at the adjusted exposure frame rate, essentially achieving synchronized image output. For example, the terminal device can also control the third camera module to sense the first signal according to a configured interval. In response to the sensed first signal, the terminal device can control the third camera module to start capturing the current frame.

[0024] Optionally, the method further includes: the terminal device configuring the third camera module to sense the first signal at a certain interval based on the second exposure frame rate, the exposure frame rate of the third camera module, and the number of intervals configured for sensing the first signal by the second camera module.

[0025] By configuring the third camera module to sense the first signal at the configured interval, the terminal device can control the third camera module to sense the first signal according to the configured interval.

[0026] In some implementations, the number of intervals in which the third camera module senses the first signal is equal to the sum of the number of intervals in which the second camera module senses the first signal plus a first value, where the first value is the quotient of the second exposure frame rate divided by the exposure frame rate of the third camera module minus 1.

[0027] Taking the second exposure frame rate as Ss, the third camera module's exposure frame rate as Sr (Ss is an integer multiple of Sr), and the second camera module's interval number of pulse signal sensing as (Y-1) as an example, the terminal device can determine the value of Ss divided by Sr minus 1 as the first value, and the interval number of the third camera module's sensing of the first signal can be equal to the sum of the value of Ss divided by Sr minus 1 and (Y-1).

[0028] In some implementations, the method further includes: detecting a change in scene information of the shooting scene, the terminal device adjusting the exposure frame rate of the first camera module to a fourth exposure frame rate and adjusting the exposure frame rate of the second camera module to a fifth exposure frame rate. The terminal device controls the first camera module to shoot at the fourth exposure frame rate and controls the second camera module to cooperate with the first camera module to shoot at the fifth exposure frame rate.

[0029] The terminal device controls the first camera module to shoot at the fourth exposure frame rate and controls the second camera module to shoot in coordination with the first camera module at the fifth exposure frame rate. This includes: the terminal device controls the first camera module to start shooting the third frame and controls the second camera module to start shooting the fourth frame. The start time of the fourth frame is delayed by a second duration compared to the start time of the third frame. The second duration is related to the fourth exposure frame rate and the fifth exposure frame rate.

[0030] For example, in some possible scenarios, the scene information of the shooting environment may change. The terminal device needs to adjust some or all of the camera modules according to the changes in scene information to ensure better shooting results. For instance, when the brightness of the shooting environment changes, the terminal device needs to adjust the exposure time of some or all of the camera modules to maintain brightness consistency in the shooting effect. When the ambient brightness is very low, a long exposure time is usually required to achieve consistent image brightness. If the exposure time is longer than the current frame, the frame length needs to be increased accordingly, i.e., the frame rate needs to be reduced, or simply downsampling. When the ambient brightness returns from a dark scene to normal brightness, the excessively long frame length needs to be restored, i.e., the frame rate needs to be increased, or simply upsampling. When the ambient brightness changes significantly or the sensitivity of the devices varies greatly, flexible upsampling and downsampling of the camera modules can bring better shooting results, achieving consistent exposure effects among multiple cameras and better exposure consistency under different ambient brightness levels, thus improving image fusion effects.

[0031] In this application, after detecting a change in the scene information of the shooting scene, the terminal device can adjust the exposure frame rate of the first camera module and the second camera module. For example, the terminal device adjusts the exposure frame rate of the first camera module to the fourth exposure frame rate and the exposure frame rate of the second camera module to the fifth exposure frame rate.

[0032] Optionally, the fifth exposure frame rate is different from the second exposure frame rate, and the fourth exposure frame rate and the first exposure frame rate can be the same or different.

[0033] After adjusting the exposure frame rates of the first and second camera modules, the terminal device can control the first camera module to shoot at a fourth exposure frame rate and control the second camera module to shoot in coordination with the first camera module at a fifth exposure frame rate. Specifically, the terminal device can control the first camera module to start shooting the third frame and control the second camera module to start shooting the fourth frame. The start time of the fourth frame is delayed by a second duration compared to the start time of the third frame, and the second duration is related to both the fourth and fifth exposure frame rates.

[0034] For scenarios where frame rate changes occur due to changes in scene information, this method still ensures that the exposure output of the second camera module and the exposure output of the first camera module remain synchronized at the end of the frame. This guarantees the consistency of the images captured by the first and second camera modules in frame rate changes, thereby improving the final image output quality.

[0035] Optionally, after detecting a change in the scene information of the shooting scene, the terminal device can also adjust the exposure frame rate of the third camera module. The exposure frame rate of the third camera module is related to the exposure frame rate of the second camera module. The adjustment method can be found in the aforementioned implementation method and will not be repeated here.

[0036] In some implementations, the second duration is related to both the fourth and fifth exposure frame rates, including: the second duration is equal to the difference between the frame length corresponding to the sixth exposure frame rate and the frame length corresponding to the fifth exposure frame rate. The sixth exposure frame rate is the maximum value among all values ​​less than or equal to the fifth exposure frame rate and divisible by the fourth exposure frame rate.

[0037] The second duration is similar to the first duration. For example, taking the fourth exposure frame rate (which can be called the expected exposure frame rate of the first camera module) as Sm_new and the fifth exposure frame rate (which can be called the expected exposure frame rate of the second camera module) as Ss_new, the sixth exposure frame rate can be the first frame rate Ss1_new that is divisible by Sm_new, determined by reducing the frame rate downwards from Ss_new. That is, Ss1_new is the maximum value among all values ​​that are less than or equal to Ss_new and divisible by Sm_new. The second duration can be equal to the difference (which can be the absolute value of the difference) between the frame length corresponding to Ss1_new and the frame length corresponding to Ss_new.

[0038] Optionally, the method further includes: the terminal device controlling the first camera module to emit a first signal after a second delay from the start time of the third frame. The terminal device controlling the second camera module to start capturing the fourth frame includes: the terminal device controlling the second camera module to sense the first signal according to a configured interval number for sensing the first signal. In response to the sensed first signal, the terminal device controls the second camera module to start capturing the fourth frame.

[0039] The terminal device controls the first camera module to emit a first signal after a second delay from the start time of the third frame. In other words, the terminal device can control the first camera module to emit the first signal at a time delay after the start time of the third frame. The terminal device controls the second camera module to sense the first signal according to a configured interval. In response to the sensed first signal, the terminal device controls the second camera module to start capturing the fourth frame. The start time of the fourth frame can be delayed by the second delay compared to the third frame, thereby controlling the start time of the second camera module's exposure and image capture based on the exposure frame rates of the first and second camera modules.

[0040] Optionally, the method further includes: the terminal device configuring the first camera module to emit a first signal at a time delayed by a second duration from the start time of the third frame, based on the fourth exposure frame rate and the fifth exposure frame rate.

[0041] As described above, taking the fourth exposure frame rate as Sm_new and the fifth exposure frame rate as Ss_new as an example, the second duration can be equal to the difference between the frame length corresponding to Ss1_new and the frame length corresponding to Ss_new. The terminal device can control the first camera module to emit the first signal after a second delay from the start time of the third frame by configuring the first camera module to emit the first signal after a second delay from the start time of the third frame.

[0042] Optionally, the fourth exposure frame rate is less than or equal to Z times the fifth exposure frame rate, or Z times the frame length corresponding to the fourth exposure frame rate is greater than or equal to the frame length corresponding to the fifth exposure frame rate. Here, Z equals the value of the number of intervals for sensing the first signal configured in the second camera module plus 1.

[0043] Since the interval number at which the second camera module senses the first signal is determined during the initial flow phase, based on the constraints of hard synchronization itself, the fourth and fifth exposure frame rates satisfy the following condition: the fourth exposure frame rate is less than or equal to Z times the fifth exposure frame rate, or Z times the frame length corresponding to the fourth exposure frame rate is greater than or equal to the frame length corresponding to the fifth exposure frame rate. Given this relationship, the desired frame rate combination can always be achieved between the fourth and fifth exposure frame rates.

[0044] In some implementations, the method further includes: the terminal device performing a fusion process on the first frame and the second frame to obtain a first image.

[0045] The first image is the final image obtained in a multi-camera collaborative shooting scenario, which can have a better output image quality.

[0046] In some implementations, the terminal device determines the first camera module and the second camera module from the configured camera modules, which may include: the terminal device acquiring scene information of the current shooting scene and capability information of the configured camera modules; the terminal device determining the first camera module and the second camera module from the configured camera modules based on the scene information and capability information.

[0047] For example, when a terminal device initiates a shooting function (such as launching and running a shooting application), it can obtain scene information of the current shooting scene. Exemplarily, the scene information includes one or more of the following: focal length information, lighting information, and the user's shooting requirements.

[0048] The terminal device acquires scene information of the current shooting scene, which may include: the terminal device acquiring scene information of the current shooting scene based on one or more of the following: a preview image corresponding to the current shooting scene, sensor information (such as light information collected by a light sensor), and a first operation. The first operation is the operation of selecting a shooting mode and / or setting shooting parameters on a first interface displayed on the terminal device.

[0049] Alternatively, the terminal device can also obtain scene information corresponding to the current shooting scene by recognizing the preview image corresponding to the current shooting scene. For example, the terminal recognition can determine whether the user's shooting requirement is a portrait, a landscape, or the lighting information of the current shooting scene by recognizing the preview image corresponding to the current shooting scene. Alternatively, the terminal device can also combine the recognition results of the preview image corresponding to the current shooting scene, the detection parameters of hardware such as the light sensor, and the user's operation to comprehensively determine the scene information of the current shooting scene.

[0050] Optionally, the camera module's capability information may include one or more of the following: maximum exposure frame rate, reasonable focal length for shooting, and output color mode.

[0051] In some implementations, the first exposure frame rate and the second exposure frame rate can be determined by the terminal device based on scene information of the current shooting scene and the capability information of the camera module configured on the terminal device, after determining the minimum exposure frame rate of the first and second camera modules respectively. For example, the first exposure frame rate is greater than or equal to the minimum exposure frame rate of the first camera module, and the second exposure frame rate is greater than or equal to the minimum exposure frame rate of the second camera module.

[0052] Secondly, embodiments of this application provide a shooting device that can be applied to a terminal device to implement the shooting method described in the first aspect. The function of this device can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, the device may include a configuration module, a shooting module, etc.

[0053] The configuration module is used to determine the first camera module and the second camera module from the configured camera modules; adjust the exposure frame rate of the first camera module to the first exposure frame rate, and adjust the exposure frame rate of the second camera module to the second exposure frame rate; the first exposure frame rate and the second exposure frame rate are different.

[0054] The shooting module is used to control the first camera module to shoot at a first exposure frame rate, and to control the second camera module to shoot in coordination with the first camera module at a second exposure frame rate.

[0055] The shooting module is specifically used to control the first camera module to start shooting the first frame and control the second camera module to start shooting the second frame. The start time of the second frame is delayed by a first duration compared to the start time of the first frame. The first duration is related to the first exposure frame rate and the second exposure frame rate.

[0056] In some implementations, the first duration is related to both the first and second exposure frame rates, including: the first duration is equal to the difference between the frame length corresponding to the third exposure frame rate and the frame length corresponding to the second exposure frame rate. The third exposure frame rate is the maximum value among all values ​​that are less than or equal to the second exposure frame rate and divisible by the first exposure frame rate.

[0057] Optionally, the shooting module is further configured to control the first camera module to emit a first signal after a first delay in the start time of the first frame; and to control the second camera module to sense the first signal according to a configured interval number of times the first signal is sensed, and in response to the sensed first signal, to control the second camera module to start shooting the second frame.

[0058] Optionally, the configuration module is further configured to configure the second camera module to sense the first signal at intervals according to the first exposure frame rate and the second exposure frame rate, and to configure the first camera module to emit the first signal at a time delayed by a first duration from the start time of the first frame.

[0059] In some implementations, the number of intervals at which the second camera module senses the first signal is equal to the quotient of the first exposure frame rate divided by the third exposure frame rate minus 1.

[0060] In some implementations, the configuration module is further configured to determine a third camera module from the configured camera modules, and adjust the exposure frame rate of the third camera module so that the exposure frame rate of the third camera module satisfies the condition that the second exposure frame rate is an integer multiple of the exposure frame rate of the third camera module. The shooting module is further configured to control the third camera module to perform coordinated shooting with the first and second camera modules at the adjusted exposure frame rate.

[0061] The shooting module is specifically used to control the third camera module to sense the first signal according to the configured interval number of times the first signal is sensed, and in response to the sensed first signal, control the third camera module to start shooting the current frame.

[0062] Optionally, the configuration module is further configured to configure the number of intervals for the third camera module to sense the first signal based on the second exposure frame rate, the exposure frame rate of the third camera module, and the number of intervals for sensing the first signal configured by the second camera module.

[0063] In some implementations, the number of intervals in which the third camera module senses the first signal is equal to the sum of the number of intervals in which the second camera module senses the first signal plus a first value, where the first value is the quotient of the second exposure frame rate divided by the exposure frame rate of the third camera module minus 1.

[0064] In some implementations, the configuration module is also used to detect changes in scene information during shooting, adjust the exposure frame rate of the first camera module to the fourth exposure frame rate, and adjust the exposure frame rate of the second camera module to the fifth exposure frame rate. The shooting module is also used to control the first camera module to shoot at the fourth exposure frame rate, and control the second camera module to shoot in coordination with the first camera module at the fifth exposure frame rate.

[0065] The shooting module is specifically used to control the first camera module to start shooting the third frame and control the second camera module to start shooting the fourth frame. The start time of the fourth frame is delayed by a second duration compared to the start time of the third frame. The second duration is related to the fourth exposure frame rate and the fifth exposure frame rate.

[0066] In some implementations, the second duration is related to both the fourth and fifth exposure frame rates, including: the second duration is equal to the difference between the frame length corresponding to the sixth exposure frame rate and the frame length corresponding to the fifth exposure frame rate. The sixth exposure frame rate is the maximum value among all values ​​less than or equal to the fifth exposure frame rate and divisible by the fourth exposure frame rate.

[0067] Optionally, the shooting module is further configured to control the first camera module to emit a first signal after a second delay from the start time of the third frame, and to control the second camera module to sense the first signal according to a configured interval number of times the first signal is sensed, and in response to the sensed first signal, to control the second camera module to start shooting the fourth frame.

[0068] Optionally, the configuration module is also configured to configure the first camera module to emit a first signal at a time delayed by a second duration from the start time of the third frame, based on the fourth and fifth exposure frame rates.

[0069] Optionally, the fourth exposure frame rate is less than or equal to Z times the fifth exposure frame rate, or Z times the frame length corresponding to the fourth exposure frame rate is greater than or equal to the frame length corresponding to the fifth exposure frame rate. Here, Z equals the value of the number of intervals for sensing the first signal configured in the second camera module plus 1.

[0070] In some implementations, the shooting module is also used to fuse the first frame and the second frame to obtain the first image.

[0071] It should be understood that the shooting device can be used to achieve all the functions of the shooting method described in the first aspect above, and will not be repeated here.

[0072] Thirdly, embodiments of this application provide an electronic device, including: a processor, and a memory for storing processor-executable instructions; the processor is configured to, when executing the instructions, cause the electronic device to implement the method as described in the first aspect and any possible implementation thereof.

[0073] Fourthly, embodiments of this application provide a computer-readable storage medium having computer program instructions stored thereon; when the computer program instructions are executed by an electronic device, the electronic device causes the electronic device to implement the method described in the first aspect and any possible implementation thereof.

[0074] Fifthly, embodiments of this application also provide a computer program product, including computer-readable code, which, when executed in an electronic device, causes the electronic device to implement the method described in the first aspect and any possible implementation thereof.

[0075] In the third to fifth aspects mentioned above, the electronic device may be the terminal device described in the first aspect.

[0076] The beneficial effects of the second to fifth aspects mentioned above can be referred to in the first aspect, and will not be repeated here.

[0077] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0078] Figure 1 This is a schematic diagram illustrating the hard synchronization control principle of two camera modules.

[0079] Figure 2 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application;

[0080] Figure 3 A schematic flowchart illustrating the shooting method provided in an embodiment of this application;

[0081] Figure 4 A schematic diagram illustrating the hard synchronization control principle in a scenario where the camera module only includes the target camera module, as provided in this application embodiment;

[0082] Figure 5 A schematic diagram illustrating the hard synchronization control principle in a scenario where the camera module includes a target camera module and other camera modules, provided for embodiments of this application;

[0083] Figure 6 This is another schematic flowchart of the shooting method provided in the embodiments of this application;

[0084] Figure 7 A schematic diagram illustrating the principle of frame drop in a scene where the camera module only includes the target, provided for embodiments of this application;

[0085] Figure 8 This is a schematic diagram illustrating the software architecture implementation principle of the terminal device provided in the embodiments of this application;

[0086] Figure 9 This is a schematic diagram of the imaging device provided in an embodiment of this application. Detailed Implementation

[0087] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two). The character “ / ” generally indicates that the preceding and following objects are in an “or” relationship.

[0088] References to "one embodiment" or "some embodiments" as used in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes both direct and indirect connections, unless otherwise stated.

[0089] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0090] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0091] In multi-camera collaborative shooting scenarios, a terminal device (such as a mobile phone) can be equipped with multiple (two or more) camera modules, also known as camera module sets. When shooting, the terminal device can simultaneously capture multiple images corresponding to each camera module. The terminal device can then fuse these images to obtain the final captured image. The final captured image can have better shooting quality (or output image quality).

[0092] For example, in one possible scenario, the terminal device may include a color camera and a depth camera. When taking a picture, the terminal device can capture a color image using the color camera and a black-and-white image using the depth camera. The terminal device can then fuse the color image from the color camera and the black-and-white image from the depth camera to obtain the final captured image. The final captured image can contain richer depth information, resulting in better image quality.

[0093] Currently, when a terminal device takes pictures using multiple camera modules, hard synchronization control technology can be used to ensure that the timestamps of the start of exposure of each frame of the multiple camera modules are kept consistent during the shooting stage. This ensures the uniformity of the images captured by the multiple camera modules, thereby improving the output effect of the fused image (i.e., the final captured image).

[0094] For example, the main principle of commonly used hard synchronization control technology is as follows: When a terminal device shoots using multiple camera modules, these modules are divided into a master camera module and other slave camera modules. The master and slave camera modules are controlled to maintain the same exposure frame rate (that is, the exposure frame rate of the slave camera modules follows that of the master camera module, and the number of exposure frames is the same for both modules within the same shooting preview time). During the shooting phase, the master camera module emits an XVS pulse signal at the moment corresponding to the frame header of each frame. The other slave camera modules sense the XVS pulse signal emitted by the master camera module. Each time a slave camera module senses the XVS pulse signal emitted by the master camera module, it begins the exposure and image capture of the current frame. After completing the exposure and image capture of the current frame, the slave camera modules wait for the next XVS pulse signal emitted by the master camera module until they sense the next XVS pulse signal emitted by the master camera module, and then begin the exposure and image capture of the next frame. Therefore, the timestamps at the start of exposure for each frame of an image from multiple camera modules can be kept consistent during the shooting phase.

[0095] For example, taking a scenario where a terminal device uses two camera modules to achieve multi-camera collaborative shooting as an example, Figure 1 This is a schematic diagram illustrating the hard synchronization control principle of two camera modules. One camera module acts as the master camera module, and the other acts as the slave camera module.

[0096] like Figure 1As shown, the master camera module and slave camera module can maintain the same exposure frame rate. During the shooting phase, the master camera module emits an XVS pulse signal at the moment corresponding to the frame header of each frame. The frame header can be represented by the start-of-frame delimiter (SOF), and the frame tail by the end-of-frame delimiter (EOF). When the slave camera module senses the XVS pulse signal, it can begin exposure and image capture for the current frame. The effective frame length of the master camera module is slightly longer than that of the slave camera module. For example, the frame length of a camera module generally refers to the length between SOF and the next SOF, while the effective frame length is the sum of the length between SOF and EOF and the length of the non-exposure blank lines (vertical blank, vblank, also known as blank invalid line time, vertical blanking, or field blanking) in each frame. The terminal device can adjust the length of the vblank of the master and slave camera modules to make the effective frame length of the master camera module slightly longer than that of the slave camera module. vblank represents the time from the end of the last line readout of the previous frame to the start of the first line readout of the next frame. After the slave camera module finishes exposing and rendering the image in the current frame, it will wait for the XVS pulse signal sent by the master camera. It will only begin exposing and rendering the next frame when it senses the next XVS pulse signal from the master camera module. This method ensures that the timestamps of the frame headers of the master and slave camera modules are consistent, achieving millisecond-level image rendering time.

[0097] However, in the current hard synchronization control technology, the main camera module and the slave camera module maintain the same exposure frame rate, which leads to high power consumption in the terminal device.

[0098] For example, in multi-camera collaborative shooting scenarios, when a terminal device performs multi-image fusion, it generally needs to use the image output from the main camera module as the benchmark. The main camera module will be set to a higher exposure frame rate (i.e., more exposure frames per unit time) to ensure high dynamic range in the shooting effect. The slave camera modules will follow the exposure frame rate of the main camera module, maintaining the same high exposure frame rate. When the main camera module and the slave camera modules maintain the same high exposure frame rate, it will cause the terminal device to generate high camera power consumption, such as the terminal device overheating or even becoming hot. When the terminal device reaches a certain temperature limit, the camera module of the terminal device may not be able to maintain high-performance operation mode, thus affecting the final shooting effect.

[0099] Against this background, embodiments of this application provide a shooting method applicable to scenarios where a terminal device configured with multiple (e.g., at least two) camera modules performs multi-camera collaborative shooting. In this method, the terminal device first determines the main camera module and slave camera modules needed for shooting from the configured camera modules, and configures different exposure frame rates for the main and slave camera modules. Then, the terminal device can control the main and slave camera modules to shoot at different exposure frame rates. During shooting, the terminal device can control the start time of exposure and image output of the slave camera module based on the exposure frame rates of the main and slave camera modules, ensuring that the exposure and image output of the slave camera module remains synchronized with the exposure and image output of the main camera module at the end of the frame.

[0100] In this method, the main camera module and the slave camera modules shoot at different exposure frame rates, which minimizes the power consumption of the slave camera module due to unnecessarily high frame rates, thereby effectively reducing the power consumption of the terminal device in multi-camera collaborative shooting scenarios. The exposure output of the slave camera module is synchronized with that of the main camera module at the end of the frame, ensuring the consistency of the images captured by the main and slave camera modules, thus improving the final image quality. In other words, this method can minimize the power consumption of the terminal device while ensuring the shooting quality of multiple camera modules.

[0101] Optionally, in this application embodiment, the terminal device may be a mobile terminal with shooting function such as a mobile phone, tablet computer, wearable device, vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. Alternatively, it may be a professional shooting device such as a digital camera, SLR camera / mirrorless camera, action camera, gimbal camera, drone, etc. This application embodiment does not limit the specific type of terminal device.

[0102] It should be understood that the terminal device has at least two camera modules for multi-camera collaborative shooting.

[0103] For example, taking a mobile phone as the terminal device, Figure 2 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 2As shown, a mobile phone may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, a sensor module 280, buttons 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc.

[0104] Processor 210 may include one or more processing units, such as application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0105] The controller can serve as the nerve center and command center of the mobile phone. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0106] The processor 210 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory can store instructions or data that the processor 210 has just used or that are used repeatedly. If the processor 210 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0107] In some embodiments, the processor 210 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and / or a USB interface, etc.

[0108] The external storage interface 220 can be used to connect an external storage card, such as a Micro SD card, to expand the phone's storage capacity. The external storage card communicates with the processor 210 through the external storage interface 220 to perform data storage. For example, images captured in multi-camera collaborative shooting scenarios, or other images, music, videos, and other files, can be saved on the external storage card.

[0109] The internal memory 221 can be used to store computer executable program code, which includes instructions. The processor 210 executes various functions and data processing of the mobile phone by running the instructions stored in the internal memory 221. For example, the internal memory 221 can store computer executable program code for implementing the shooting method provided in the embodiments of this application. The processor 210 can implement the shooting method provided in the embodiments of this application by running the instructions of this computer executable program code.

[0110] The internal memory 221 may further include a program storage area and a data storage area. The program storage area may store the operating system, at least one application required for a function (such as a camera application), etc. The data storage area may store data created during phone use (such as image data, phonebook data, etc.). Furthermore, the internal memory 221 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0111] The charging management module 240 receives charging input from the charger. While charging the battery 242, the charging management module 240 can also power the mobile phone via the power management module 241. The power management module 241 connects to the battery 242, the charging management module 240, and the processor 210. The power management module 241 can also receive input from the battery 242 to power the mobile phone.

[0112] The wireless communication function of a mobile phone can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile phone can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0113] Mobile phones can perform audio functions, such as music playback and recording, through an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, and an application processor.

[0114] The sensor module 280 may include a pressure sensor 280A, a gyroscope sensor 280B, a barometric pressure sensor 280C, a magnetic sensor 280D, an accelerometer sensor 280E, a proximity sensor 280F, a proximity light sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, and a bone conduction sensor 280M, etc. When the phone is taking a picture, the sensor module 280 can collect scene information of the current shooting scene, such as collecting the temperature of the current shooting scene through the temperature sensor 280J, and collecting the light intensity of the current shooting scene through the ambient light sensor 280L.

[0115] The mobile phone can take pictures using camera 293. Camera 293 can also be called a camera module. In this embodiment, the number of cameras 293 is at least two, and the types of different cameras can be different. For example, camera 293 can be a color camera module, a monochrome camera module, a depth-of-field camera module, a telephoto (or telephoto) camera module, a wide-angle camera module, a macro camera module, etc. This application does not limit the number of cameras 293 or the specific type of each camera 293.

[0116] For example, the camera 293 may include a lens and a sensor. When taking a photo or recording video, the shutter is opened, and light is transmitted through the lens of the camera 293 to the sensor. The sensor can convert the light signal passing through the lens into an electrical signal, and then perform an A / D conversion on the electrical signal to output the corresponding digital signal. This digital signal undergoes subsequent RAW domain processing, ISP processing, and YUV domain processing to obtain the captured photo or video image.

[0117] In one possible design, the sensor's photosensitive element can be a charge-coupled device (CCD), and the sensor also includes an A / D converter. In another possible design, the sensor's photosensitive element can be a complementary metal-oxide-semiconductor (CMOS).

[0118] Display screen 294 is used to display images, videos, etc. Display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the mobile phone may include one or N displays 294, where N is a positive integer greater than 1. For example, display screen 294 can be used to display an application interface, which may be a camera interface.

[0119] The mobile phone implements its display function through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0120] Understandable Figure 2 The structure shown does not constitute a specific limitation on the mobile phone. In some embodiments, the mobile phone may also include... Figure 2This could mean having more or fewer components, combining some components, separating some components, or having different component arrangements. Or, Figure 2 Some of the components shown can be implemented in hardware, software, or a combination of both.

[0121] Additionally, when the terminal device is another mobile terminal with shooting capabilities, such as a tablet, wearable device, in-vehicle device, AR / VR device, laptop, UMPC, netbook, or PDA, or a professional shooting device such as a digital camera, SLR / mirrorless camera, action camera, gimbal camera, or drone, the specific structure of these other terminal devices can also be referenced. Figure 2 As shown. For example, other terminal devices may be... Figure 2 The components were added or removed based on the given structure, which will not be elaborated here.

[0122] It should also be understood that a terminal device (such as a mobile phone) may run one or more camera applications to enable the camera function. For example, the camera application may include the system-level application "Camera." Alternatively, the camera application may include other applications installed on the terminal device that can be used for taking pictures. There is no limitation on the specific type of camera application.

[0123] The following is combined Figure 3 The process shown is an exemplary description of the shooting method provided in the embodiments of this application.

[0124] For example, Figure 3 This is a schematic flowchart illustrating the shooting method provided in an embodiment of this application. This shooting method can be applied to terminal devices. Figure 3 As shown, the method may include:

[0125] 301. Determine the master camera module and slave camera module from the configured camera modules, and determine the minimum exposure frame rate for the master camera module and slave camera module respectively.

[0126] For example, when the terminal device starts the shooting function, it can obtain the scene information corresponding to the current shooting scene. Then, based on the scene information and the capability information of the configured camera modules, it can determine the main camera module and the slave camera module from the configured camera modules (that is, determine the main camera module and the slave camera module suitable for shooting the current shooting scene), and determine the minimum exposure frame rate of the main camera module and the slave camera module respectively.

[0127] Optionally, scene information may include: focal length information of the current shooting scene, lighting information of the current shooting scene, user shooting requirements, etc.

[0128] The focal length information of the current shooting scene can refer to the focal length (i.e., shooting distance) between the terminal device and the object being photographed in the current shooting scene. The focal length determines the distance of the image in focus; the larger the focal length, the smaller the depth of field and the smaller the shooting angle.

[0129] The lighting information of the current shooting scene can refer to the brightness of the light in the current shooting scene, whether there are dynamic changes in brightness (multi-light source scenes), etc. Optionally, the lighting information of the current shooting scene can be obtained from sensor information collected by sensors configured in the terminal device. For example, the lighting information can be collected by an ambient light sensor (light sensor).

[0130] User shooting needs refer to the shooting modes, shooting parameters, etc. that users select or set on their terminal devices.

[0131] For example, a terminal device may include a camera application. The terminal device can launch and run the camera application in response to a user's action. After launching the camera application, the terminal device initiates its shooting function. The terminal device can use the camera application to call the configured camera module to perform the shooting function. After launching the camera application, the terminal device can display a shooting interface. This interface can include function controls corresponding to various shooting modes available to the user, such as: large aperture mode, portrait mode, bokeh mode, slow motion mode, high dynamic range mode, and night scene mode. Users can select one or more shooting modes by clicking the corresponding function controls on the shooting interface. The terminal device can respond to the user's selection of a shooting mode and determine the user's shooting needs. For example, if a user clicks the function control corresponding to portrait mode, the terminal device can determine that the user's shooting needs include: shooting a portrait or using portrait mode.

[0132] In this context, the shooting interface displayed by the shooting application after the terminal device starts running is also called the first interface. The user's actions of selecting a shooting mode and / or setting shooting parameters on the shooting interface can also be called the first operation.

[0133] In some embodiments, the terminal device can also obtain scene information corresponding to the current shooting scene by recognizing the preview image corresponding to the current shooting scene. For example, the terminal recognition can determine whether the user's shooting needs are for portraits, scenery, or the lighting information of the current shooting scene by recognizing the preview image corresponding to the current shooting scene.

[0134] Alternatively, the terminal device can combine the recognition results of the preview image corresponding to the current shooting scene, the detection parameters of hardware such as the light sensor, and the user's operation to comprehensively determine the scene information of the current shooting scene.

[0135] It should be noted that the scene information corresponding to the current shooting scene mentioned above is only an illustrative example. This application embodiment does not limit the specific type of scene information corresponding to the current shooting scene obtained by the terminal device, nor the specific method by which the terminal device obtains the scene information of the current shooting scene.

[0136] After acquiring scene information corresponding to the current shooting scene, the terminal device can determine the main camera module and slave camera modules from the configured camera modules based on the scene information and the capability information of the configured camera modules. The number of slave camera modules can be one or more. The minimum exposure frame rate of the main camera module is greater than the minimum exposure frame rate of any slave camera module.

[0137] It should be understood that the main camera module and the slave camera module determined by the terminal device constitute a camera module combination (or multi-camera collaborative combination) suitable for shooting the current shooting scene, which can be used to collaboratively shoot the current shooting scene.

[0138] In some embodiments, the capability information of the camera modules configured in the terminal device can be preset in the terminal device. For example, each camera module can store its own capability information, and the terminal device can obtain the capability information of each camera module.

[0139] For example, the camera module's capability information may include: maximum exposure frame rate, reasonable focal length for shooting, output color mode, etc.

[0140] For example, assuming the terminal device is equipped with camera modules including: a color camera module, a monochrome camera module, a depth-of-field camera module, a telephoto camera module, and a wide-angle camera module, the capability information of the aforementioned camera modules can be as follows:

[0141] 1) The maximum exposure frame rate of the color camera module is 30 frames per second (FPS) or 60 FPS. The reasonable focal length for shooting is between 0.8X and 5X, and the output color mode is color.

[0142] 2) The maximum exposure frame rate of the black and white camera module is 30 FPS, the reasonable focal length for shooting is between 0.8X and 5X, and the output color mode is no color or black and white.

[0143] 3) The maximum exposure frame rate of the depth-of-field camera module is 30FPS, the reasonable focal length for shooting is between 0.8X and 5X, the output color mode is color, and it can quickly and at a distance (1-10 meters) obtain more accurate depth-of-field information.

[0144] 4) The maximum exposure frame rate of the telephoto camera module is 30FPS, the reasonable focal length for shooting is 3X or higher, and the output color mode is color.

[0145] 5) The maximum exposure frame rate of the wide-angle camera module is 30 FPS, the reasonable focal length for shooting is between 0.1X and 1X, and the output color mode is color.

[0146] Assuming the scene information collected by the terminal device in step 301 corresponds to the current shooting scene as follows: large aperture portrait mode, focal length between 1X and 3X, the terminal device can combine the capability information of the color camera module, monochrome camera module, depth-of-field camera module, telephoto camera module, and wide-angle camera module. Based on the 1X-3X focal length scene, it determines that a telephoto camera module and a wide-angle camera module are not needed. Based on the large aperture mode, the color camera module is selected as the main camera module to provide the primary image information, and the monochrome camera module is selected as the secondary camera module to provide richer light and shadow information. Based on the portrait mode, the depth-of-field camera module is selected as the secondary camera module to provide better bokeh effect. That is, in this scene, the terminal device can select the color camera module as the main camera module, and the monochrome camera module and the depth-of-field camera module as secondary camera modules.

[0147] Assuming the scene information collected by the terminal device in step 301 corresponds to the current shooting scene as follows: large aperture non-portrait mode, focal length between 1X and 3X, the terminal device can combine the capability information of the color camera module, monochrome camera module, depth-of-field camera module, telephoto camera module, and wide-angle camera module. Based on the 1X-3X focal length scene, it determines that a telephoto camera module and a wide-angle camera module are not needed; based on the large aperture mode, the color camera module is selected as the main camera module to provide the primary image information, and the monochrome camera module is selected as the secondary camera module to provide richer light and shadow information; based on the non-portrait mode, it is determined that a depth-of-field camera module is not needed. In other words, in this scene, the terminal device can select the color camera module as the main camera module and the monochrome camera module as the secondary camera module.

[0148] The above example illustrates the scenario where a telephoto and wide-angle camera module are not needed when the focal length is between 1X and 3X. The following explains the scenarios where a telephoto or wide-angle camera module is required.

[0149] Optionally, when the focal length is between 0.5X and 1X, the terminal device can determine whether a wide-angle camera module is needed based on the specific changes in the focal length, and if a wide-angle camera module is needed, it can make a master-slave camera module decision between the wide-angle camera module and the color camera module.

[0150] For example, during the transition from 1X to 0.5X focal length, the terminal device can make decisions regarding the master and slave camera modules based on the specific changes in focal length as follows:

[0151] 1) When the focal length is less than 1X and greater than 0.9X, the terminal device can select the color camera module as the main camera module and determine that a wide-angle camera module is not needed.

[0152] 2) When the focal length is less than or equal to 0.9X and greater than 0.7X, the terminal device can select the color camera module as the main camera module and the wide-angle camera module as the slave camera module.

[0153] 3) When the focal length is less than or equal to 0.7X and greater than 0.5X, the terminal device can select the color camera module as the slave camera module and the wide-angle camera module as the master camera module.

[0154] 4) When the focal length is less than or equal to 0.5X, the terminal device can select the wide-angle camera module as the main camera module and determine that a color camera module is not needed.

[0155] Similarly, when the focal length is between 3X and 5X, the terminal device can determine whether a telephoto camera module is needed based on the specific changes in focal length, and when a telephoto camera module is needed, it can make a master-slave decision between the telephoto camera module and the color camera module.

[0156] For example, during the transition from 3X to 5X focal length, the terminal device can make decisions regarding the master and slave camera modules based on the specific changes in focal length as follows:

[0157] 1) When the focal length is greater than 3X and less than 3.5X, the terminal device can select the color camera module as the main camera module and the telephoto camera module as the slave camera module.

[0158] 2) When the focal length is greater than or equal to 3.5X and less than 5X, the terminal device can select the color camera module as the slave camera module and the telephoto camera module as the master camera module.

[0159] 3) When the focal length is greater than or equal to 5X, the terminal device can select the telephoto camera module as the main camera module and determine that a color camera module is not needed.

[0160] It should be noted that the process described above, in which the terminal device determines the suitable primary and secondary camera modules for shooting the current scene based on scene information corresponding to the current shooting scene and the capability information of the camera modules configured on the terminal device, is merely an illustrative example. It should be understood that when the scene information corresponding to the current shooting scene is of other types, the method by which the terminal device determines the suitable primary and secondary camera modules for shooting the current shooting scene may also vary, and is not limited here.

[0161] In some embodiments, the rules by which the terminal device determines the suitable main camera module and slave camera module for shooting the current shooting scene can be manually configured. The terminal device can determine the suitable main camera module and slave camera module for shooting the current shooting scene according to the manually configured rules.

[0162] The embodiments of this application do not limit the specific method by which the terminal device determines the main camera module and the slave camera module suitable for shooting the current shooting scene.

[0163] After the terminal device determines the main camera module and the slave camera module that are suitable for shooting the current shooting scene, it can determine the minimum exposure frame rate for the main camera module and the slave camera module respectively.

[0164] For example, as mentioned above, when the scene information corresponding to the current shooting scene includes: large aperture portrait mode, focal length between 1X and 3X, the terminal device can determine the color camera module as the main camera module, and the monochrome camera module and depth-of-field camera module as slave camera modules. Since the general large aperture portrait mode does not involve the dynamic movement of the subject, there is no high requirement for the frame rate. Therefore, the terminal device can determine the minimum exposure frame rate of the color camera module to be 30FPS to ensure good image output, determine the minimum exposure frame rate of the monochrome camera module to be 10-12FPS to meet the needs of providing light and shadow variations, and determine the minimum exposure frame rate of the depth-of-field camera module to be 7.5FPS or 10FPS to meet the needs of providing distance information in the scene.

[0165] Optionally, the rules by which the terminal device determines the minimum exposure frame rate of the main camera module and the slave camera module can also be manually configured. For example, the manufacturer can configure the rules for determining the minimum exposure frame rate of the main camera module and the slave camera module for the terminal device before it leaves the factory.

[0166] This application does not impose any restrictions on the specific method by which the terminal device determines the minimum exposure frame rate of the master camera module and the slave camera module.

[0167] 302. Adjust the exposure frame rate of the main camera module to the first exposure frame rate, which is greater than or equal to the minimum exposure frame rate of the main camera module.

[0168] For example, the terminal device can adjust the exposure frame rate of the main camera module to its minimum exposure frame rate, or adjust it to a value greater than the minimum exposure frame rate of the main camera module. For example, if the minimum exposure frame rate of the main camera module is 30 FPS, the terminal device can adjust the exposure frame rate of the main camera module to 30 FPS, or 40 FPS, or 60 FPS, etc.

[0169] Understandably, the maximum exposure frame rate of the main camera module can be adjusted to the maximum exposure frame rate supported by the main camera module. That is, the first exposure frame rate is less than or equal to the maximum exposure frame rate supported by the main camera module.

[0170] 303. Adjust the exposure frame rate of the target camera module with the highest minimum exposure frame rate in the camera module to the second exposure frame rate, where the second exposure frame rate is greater than or equal to the minimum exposure frame rate of the target camera module.

[0171] As described above, the number of slave camera modules determined in step 301 can be one or more. When the number of slave camera modules is one, the target slave camera module mentioned in step 303 refers to this single slave camera module. When the number of slave camera modules is multiple (e.g., two or more), the minimum exposure frame rates of the multiple slave camera modules determined in step 301 may be the same or different. In this case, the target slave camera module mentioned in step 303 refers to the slave camera module with the highest minimum exposure frame rate among the multiple slave camera modules.

[0172] Understandably, the number of target camera modules can be one or more. For example, if multiple camera modules have the same maximum minimum exposure frame rate, then all of these multiple camera modules with the maximum minimum exposure frame rate are considered target camera modules.

[0173] Optionally, when there are multiple target camera modules, in step 303, the exposure frame rate of all target camera modules can be adjusted according to the lowest exposure frame rate of any one target camera module.

[0174] For example, taking the second exposure frame rate as equal to the minimum exposure frame rate of the target slave camera module as an example, assuming there are 3 slave camera modules, the minimum exposure frame rate of the first slave camera module is 18 FPS, the minimum exposure frame rate of the second slave camera module is also 18 FPS, and the minimum exposure frame rate of the third slave camera module is 10 FPS, then the first and second slave camera modules are the target slave camera modules. In step 303, the exposure frame rates of the first and second slave camera modules can be adjusted to 18 FPS respectively. The third slave camera module can be referred to as the remaining slave camera modules.

[0175] In this application, the target camera module can be considered as the relatively important camera module among all camera modules. Among all camera modules, the target camera module has a relatively greater impact on the output image quality of the final captured image obtained by multi-camera collaborative shooting.

[0176] 304. Adjust the exposure frame rate of the remaining camera modules other than the target camera module in the camera module so that the exposure frame rate of the remaining camera modules satisfies the following conditions: the second exposure frame rate is an integer multiple of the exposure frame rate of the remaining camera modules, and the exposure frame rate of the remaining camera modules is greater than or equal to the lowest exposure frame rate of the remaining camera modules.

[0177] It is understandable that when the second exposure frame rate is an integer multiple of the exposure frame rate of the remaining camera modules, the exposure frame rate of the remaining camera modules is less than or equal to the second exposure frame rate.

[0178] For example, assuming the number of remaining camera modules other than the target camera module is i, where i is an integer greater than 0, the second exposure frame rate is S0, the minimum exposure frame rate of the first remaining camera module is S1, the minimum exposure frame rate of the second remaining camera module is S2, ..., and the minimum exposure frame rate of the i-th remaining camera module is Si, then the specific method by which the terminal device adjusts the exposure frame rate of each remaining camera module in step 304 may include: using S0 as a reference... Adjust the exposure frame rate S'1 of the first remaining camera module so that S'1 satisfies: S0 = X1 * S'1, where X1 is a positive integer, and S1 ≤ S'1 ≤ S0; adjust the exposure frame rate S'2 of the second remaining camera module so that S'2 satisfies: S0 = X2 * S'2, where X2 is a positive integer, and S2 ≤ S'2 ≤ S0; ...; and so on, adjust the exposure frame rate S'i of the i-th remaining camera module so that S'i satisfies: S0 = Xi * S'i, where Xi is a positive integer, and Si ≤ S'i ≤ S0. "≤" indicates less than or equal to. "*" indicates a product.

[0179] Optionally, during the process of adjusting the exposure frame rate of each remaining slave camera module in the manner described above, if there are multiple combinations of Xi and S'i corresponding to the i-th remaining slave camera module, the terminal device can adjust the exposure frame rate S'i of the i-th remaining slave camera module to the smallest of the multiple combinations.

[0180] For example, assuming S0 is 30 FPS and the minimum exposure frame rate Si of the i-th remaining camera module is 10 FPS, the terminal device adjusts the exposure frame rate S'i of the i-th remaining camera module so that S'i satisfies: S0 = Xi * S'i, where Xi is an integer greater than 0. When Si ≤ S'i ≤ S0, the combination of Xi and S'i can include the three types shown in (1) to (3) below:

[0181] (1) Xi is 1, S'i is 30 FPS;

[0182] (2) Xi is 2, S'i is 15 FPS;

[0183] (3) Xi is 3, S'i is 10 FPS;

[0184] At this point, the terminal device can adjust the exposure frame rate S'i of the i-th remaining camera module to 10 FPS.

[0185] That is, for the i-th remaining slave camera module: when the range from the lowest exposure frame rate to the second exposure frame rate of the remaining slave camera module includes multiple S'i, and the second exposure frame rate is an integer multiple of S'i, the terminal device can configure the minimum value among the multiple S'i as the exposure frame rate of the i-th remaining slave camera module.

[0186] Of course, during the process of adjusting the exposure frame rate of each remaining slave camera module in the manner described above, for the i-th remaining slave camera module, when there are multiple combinations of Xi and S'i corresponding to the i-th remaining slave camera module, the terminal device can also adjust the exposure frame rate S'i of the i-th remaining slave camera module to any one of the multiple combinations, without any restrictions.

[0187] Taking the terminal device as an example where the main camera module is a color camera module, and the secondary camera modules are a monochrome camera module and a depth-of-field camera module, with a minimum exposure frame rate of 30 FPS for the color camera module, 12 FPS for the monochrome camera module, and 10 FPS for the depth-of-field camera module: In this embodiment, the monochrome camera module can be the target secondary camera module, and the depth-of-field camera module can be any of the other secondary camera modules. The terminal device can adjust the exposure frame rate of the color camera module to 30 FPS (i.e., the first exposure frame rate), adjust the exposure frame rate of the monochrome camera module to 12 FPS (i.e., the second exposure frame rate), and adjust the exposure frame rate of the depth-of-field camera module to 12 FPS (i.e., the second exposure frame rate is 1 times the exposure frame rate of the depth-of-field camera module) in the manner described above.

[0188] The process described in steps 301-304 above is that the terminal device determines the main camera module and the slave camera module to be used for shooting from the configured camera modules, and configures different exposure frame rates for the main camera module and the slave camera module.

[0189] In this application, the main camera module may be referred to as the first camera module, the target secondary camera module may be referred to as the second camera module, and the remaining secondary camera modules may be referred to as the third camera module.

[0190] It should be noted that this application does not restrict the execution order of steps 302-304. For example, steps 302-304 can be executed simultaneously, or steps 303 and 304 can be executed before step 302, or step 303 can be executed after step 304, etc.

[0191] Taking steps 302-304 as an example, the terminal device can predetermine the target values ​​of the first exposure frame rate, the second exposure frame rate, and the exposure frame rate of each of the remaining slave camera modules (see step 304). Then, the exposure frame rates of the main camera module and the slave camera modules are configured respectively.

[0192] Additionally, it should be noted that in some scenarios, the number of other camera modules may be zero. For example, when there is only one camera module, only the target camera module exists, and the other camera modules are not present. Similarly, when there are multiple camera modules, and all of them have the same minimum exposure frame rate, only the target camera module exists, and the other camera modules are not present. In the case where the number of other camera modules is zero, step 304 is not actually present in the shooting method provided in this embodiment.

[0193] The following section continues to explain the process by which the terminal device controls the time for the camera module to start exposure and image output based on the exposure frame rate of the main camera module and the exposure frame rate of the secondary camera module.

[0194] 305. Based on the first exposure frame rate and the second exposure frame rate, configure the interval number of the sensing pulse signal from the camera module for the target, and configure the delay time for the pulse signal to be emitted by the main camera module.

[0195] The interval number of sensing pulse signals configured for the target from the camera module refers to the interval at which the target from the camera module senses the pulse signals emitted by the main camera module. For example, if the terminal device configures the interval number of sensing pulse signals for a target from the camera module to be 2, then the target from the camera module will sense the pulse signals emitted by the main camera module every 2 intervals. The target from the camera module sensing a specific pulse signal emitted by the main camera module means that the target from the camera module can respond to the pulse signal emitted by the main camera module in this instance and begin capturing the current frame.

[0196] Taking a first exposure frame rate of Sm and a second exposure frame rate of Ss as an example, the terminal device configures the interval number of the sensing pulse signal from the camera module for the target based on the first and second exposure frame rates. This can include: using Sm as a reference, determining the first frame rate Ss1 that is divisible by Sm by decreasing the frame rate downwards based on Ss, and configuring the value of the quotient of Sm divided by Ss1 minus 1 as the interval number of the sensing pulse signal configured from the camera module for the target. Alternatively, it can be understood that the interval number of the sensing pulse signal configured from the camera module for the target is actually the value obtained by dividing Sm by Ss and rounding it down.

[0197] In other words, the terminal device can use Sm as a reference, and determine the first frame rate Ss1 that satisfies Sm = Y·Ss1, where Y is an integer greater than 0, by reducing the frame rate downwards based on Ss. The value of Y minus 1 is then configured as the interval number at which the target configures the sensing pulse signal from the camera module. Alternatively, the terminal device can use Sm as a reference, and configure the value obtained by dividing Sm by Ss and rounding it down as the interval number at which the target configures the sensing pulse signal from the camera module.

[0198] For example, assuming the first exposure frame rate is 30 FPS and the second exposure frame rate is 18 FPS, the terminal device can use 30 FPS as a baseline, and then reduce the frame rate downwards from 18 FPS to determine the first frame rate divisible by 30 FPS, which is 15 FPS. The result is 30 FPS divided by 15 FPS minus 1, which equals 1. The device then configures the interval for the target to receive the sensing pulse signal from the camera module to be once. Alternatively, the terminal device can first use 30 FPS as a baseline, divide 30 FPS by 18 FPS, round down to the nearest integer (1), and then configure the interval for the target to receive the sensing pulse signal from the camera module to be once. As can be seen, the results determined by both methods are the same.

[0199] Taking the first exposure frame rate as Sm and the second exposure frame rate as Ss as an example, the terminal device configures the delay time of the pulse signal for the main camera module based on the first exposure frame rate and the second exposure frame rate. This can include: using Sm as a reference, determining the first frame rate Ss1 that can be divided by Sm by reducing the frame rate downward based on Ss (see above, not repeated here), and configuring the difference between the frame length (frame interval duration) corresponding to Ss1 and the frame length corresponding to Ss (which can be the absolute value of the difference) as the delay time for the main camera module to emit the pulse signal.

[0200] For example, as described above, when the first exposure frame rate is 30 FPS and the second exposure frame rate is 18 FPS, the terminal device can use 30 FPS as a baseline and reduce the frame rate downwards from 18 FPS to determine the first frame rate divisible by 30 FPS, which is 15 FPS. At this point, the terminal device can determine that the frame length corresponding to 15 FPS is approximately 66.667 milliseconds (ms), and the frame length corresponding to 18 FPS is approximately 55.556 ms, with a difference of 11.11 ms between the two frame lengths. The terminal device can then configure the delay time for the pulse signal emitted by the main camera module to be 11.11 ms.

[0201] Alternatively, in other implementations, the terminal device can also configure the interval number of the sensing pulse signal from the camera module for the target, and configure the delay time for the main camera module to emit the pulse signal, based on the frame length corresponding to the first exposure frame rate and the frame length corresponding to the second exposure frame rate. The principle is similar to configuring the interval number and delay time based on the first and second exposure frame rates.

[0202] For example, the terminal device can use the frame length corresponding to the first exposure frame rate (such as L). Sm Based on the second exposure frame rate, the frame length (e.g., L) is used as a benchmark. Ss Increasing the frame length upwards determines the first element that can be L. Sm Frame rate L divisible Ss1 , will L Ss1 Divide by L Sm The value after subtracting 1 from the quotient is configured as the target number of intervals for sensing pulse signals from the camera module.

[0203] For example, the terminal device can use the frame length corresponding to the first exposure frame rate (such as L). Sm Based on the second exposure frame rate, the frame length (e.g., L) is used as a benchmark. Ss Increasing the frame length upwards determines the first element that can be L. Sm Frame rate L divisible Ss1 , will L Ss1 With L SsThe difference between them (which can be the absolute value of the difference) is configured as the delay time for the pulse signal emitted by the main camera module.

[0204] In this application, Ss1 can be referred to as the third exposure frame rate. It is understood that the third exposure frame rate is an intermediate value used for calculation and is not actually configured for any particular camera module. The third exposure frame rate (Ss1) is the maximum value among all values ​​less than or equal to Ss and divisible by Sm. That is, the delay time of the pulse signal is related to both Sm and Ss.

[0205] 306. Based on the second exposure frame rate, the exposure frame rates of the remaining camera modules, and the number of intervals in which the target senses the pulse signal from the camera module, configure the number of intervals in which the target senses the pulse signal from the remaining camera modules.

[0206] Similar to the interval number for sensing pulse signals from the main camera module, the interval number for sensing pulse signals configured for other slave camera modules refers to how many times each slave camera module senses the pulse signal emitted by the main camera module. For example, if the terminal device configures the interval number for sensing pulse signals for a certain slave camera module to be 2, then that slave camera module will sense the pulse signal emitted by the main camera module every 2 intervals. Sensing a specific pulse signal emitted by the main camera module from another slave camera module means that the slave camera module can respond to the pulse signal emitted by the main camera module in this instance and begin capturing the current frame.

[0207] Taking a second exposure frame rate of Ss and the exposure frame rates of the remaining camera modules as Sr (Ss being an integer multiple of Sr), and the interval number of pulse signals sensed by the target camera module as (Y-1) as an example, the terminal device configures the interval number of pulse signals sensed by the remaining camera modules according to the second exposure frame rate, the exposure frame rates of the remaining camera modules, and the interval number of pulse signals sensed by the target camera module. This may include: determining the value of Ss divided by Sr minus 1, and configuring the sum of the value of Ss divided by Sr minus 1 and (Y-1) as the interval number of pulse signals sensed by the remaining camera modules.

[0208] That is, the remaining intervals in which the target senses the first signal configured from the camera module are equal to the sum of the intervals in which the target senses the first signal configured from the camera module plus the first value, where the first value is the quotient of Ss divided by Sr minus 1.

[0209] For example, assuming the second exposure frame rate is 24 FPS, the exposure frame rate of a certain other camera module is 24 FPS, and the interval between the target camera module sensing the pulse signal is 1, then the terminal device can configure the interval between the other camera module sensing the pulse signal to be (24 / 24-1+1=1) times.

[0210] For example, assuming the second exposure frame rate is 24 FPS, the exposure frame rate of a certain other camera module is 12 FPS, and the interval between the target camera module sensing the pulse signal is 1, then the terminal device can configure the interval between the other camera module sensing the pulse signal to be (24 / 12-1+1=2) times.

[0211] In some embodiments, the terminal device may have a pre-defined correspondence between the second exposure frame rate and the interval number of pulse signals sensed from the other camera modules. This correspondence can be obtained in the manner described above for determining the interval number of pulse signals sensed from the other camera modules, and can be pre-configured in the terminal device. The terminal device can query this correspondence based on the second exposure frame rate and the exposure frame rates of the other camera modules to determine the interval number of pulse signals sensed from the other camera modules.

[0212] It should also be noted that, similar to step 304, for the other scenarios where the number of camera modules is 0, step 306 is not actually present in the shooting method provided in this application embodiment.

[0213] 307. Shooting is performed by a main camera module and a slave camera module. During the shooting process, the main camera module emits a pulse signal at the moment corresponding to the frame header of each frame after a delay of the pulse signal emission time. The slave camera module senses the pulse signal emitted by the main camera module according to the configured interval number of pulse signal sensing, and starts shooting the current frame after sensing the pulse signal.

[0214] It should be understood that the slave camera modules mentioned herein may include the target slave camera module and the remaining slave camera modules. That is, for any slave camera module, the slave camera module can start capturing the current frame in response to sensing the pulse signal emitted by the master camera module.

[0215] Optionally, the terminal device can configure the frequency of the pulse signal emitted by the main camera module so that the main camera module emits the pulse signal after a delay of the pulse signal emission time at the time corresponding to the frame header of each frame. For example, assuming the delay time of the pulse signal emission is Δt, and the time corresponding to the frame header of a certain frame captured by the main camera module is t, then the actual time when the main camera module emits the pulse signal is (t+Δt).

[0216] The process described in steps 305-307 above refers to the process by which the terminal device controls the start time of exposure and image capture for the slave camera module based on the exposure frame rate of the main camera module and the exposure frame rate of the slave camera module. The main camera module emits a pulse signal at the beginning of each frame after a delay of the pulse signal emission time. The slave camera module senses the pulse signal emitted by the main camera module according to the configured interval number of pulse signal sensing, and begins capturing the current frame upon sensing the pulse signal. This ensures that the exposure and image capture of the slave camera module can be basically synchronized with that of the main camera module at the end of the frame.

[0217] In this application, the current frame captured by the main camera module can be referred to as the first frame, and the current frame captured by the target from the camera module can be referred to as the second frame. The terminal device can control the main camera module to start capturing the first frame and control the target camera module to start capturing the second frame. The start time of the second frame is delayed by the aforementioned pulse signal delay time compared to the start time of the first frame. The pulse signal delay time is related to both the first exposure frame rate and the second exposure frame rate.

[0218] The following examples, using the XVS pulse signal as an example, illustrate two scenarios: one where the camera module only includes the target camera module, and the other where the camera module includes both the target camera module and other camera modules. It should be noted that this application does not limit the specific type of pulse signal. For example, the pulse signal described in the embodiments of this application can also be a VSYNC pulse signal, a FSIN pulse signal, etc. In this application, the aforementioned XVS pulse signal, VSYNC pulse signal, FSIN pulse signal, etc., can all be referred to as the first signal.

[0219] For example, Figure 4 This is a schematic diagram illustrating the hard synchronization control principle in a scenario where the camera module only includes the target camera module, as provided in an embodiment of this application. Figure 4As shown, assuming a certain shooting scenario (such as a low-light scene at dusk), the terminal device determines that the main camera module is a color camera module and the slave camera module is a monochrome camera module. The minimum exposure frame rate of the color camera module is 30 FPS, and the minimum exposure frame rate of the monochrome camera module is 18 FPS. Among them, the color camera module has higher shooting performance; the monochrome camera module can capture grayscale information, providing richer brightness and darkness variations. At this time, the slave camera module used for multi-camera collaborative shooting only includes one target slave camera module (i.e., the monochrome camera module), and there are no other slave camera modules.

[0220] As described above, the terminal device can adjust the exposure frame rate of the color camera module to 30 FPS and the exposure frame rate of the monochrome camera module to 18 FPS. Based on 30 FPS and 18 FPS, the device configures the interval for sensing pulse signals for the monochrome camera module to be once, and configures the delay time for emitting pulse signals for the color camera module to be 11.11 ms (t shown in the figure). delay (This refers to the delay time of the emitted pulse signal). During shooting using both a color camera module and a monochrome camera module, the color camera module emits an XVS pulse signal at 30 FPS, delayed by 11.11 ms after the frame header (SOF) of each frame. The monochrome camera module senses the XVS pulse signal emitted by the color camera module once every interval, and begins shooting the current frame after sensing the XVS pulse signal emitted by the color camera module each time. In other words, after sensing the XVS pulse signal emitted by the color camera module once, the monochrome camera module does not sense the first XVS pulse signal received from the color camera module, and waits for the XVS pulse signal after completing the shooting of the current frame before sensing the second XVS pulse signal received from the color camera module.

[0221] Combination Figure 4 As can be seen, the exposure and image output of the color camera module and the monochrome camera module will remain synchronized at the end of the frame, and their exposure and image output timestamps will be consistent. For example, the duration of each two frames for the color camera module is 66.66ms, and the actual interval between each frame for the monochrome camera module is also 66.66ms. The actual effective duration of each frame for both the color and monochrome camera modules is the same, maintaining consistent exposure and image output timestamps.

[0222] For example, Figure 5This is a schematic diagram illustrating the hard synchronization control principle in a scenario where the slave camera module includes a target slave camera module and other slave camera modules, as provided in the embodiments of this application. Figure 5 As shown, assume the terminal device determines the main camera module to be a color camera module, and the slave camera modules to be a monochrome camera module and a depth-of-field camera module; the minimum exposure frame rate of the color camera module is 30 FPS, the minimum exposure frame rate of the monochrome camera module is 18 FPS, and the minimum exposure frame rate of the depth-of-field camera module is 12 FPS. The slave camera modules used for multi-camera collaborative shooting include one target slave camera module (i.e., the monochrome camera module) and one other slave camera module (i.e., the depth-of-field camera module).

[0223] As described above, the terminal device can adjust the exposure frame rate of the color camera module to 30 FPS, the exposure frame rate of the monochrome camera module to 18 FPS, and the exposure frame rate of the depth-of-field camera module to 18 FPS (the exposure frame rate of the monochrome camera module is an integer multiple of the exposure frame rate of the depth-of-field camera module). Based on 30 FPS and 18 FPS, the interval for the sensing pulse signal is configured to be 1 for the monochrome camera module, and the delay time for emitting the pulse signal is configured to be 11.11 ms for the color camera module (i.e., t as shown in the figure). delay The depth-of-field camera module is configured to sense XVS pulse signals at an interval of once. During shooting using the color camera module, monochrome camera module, and depth-of-field camera module, the color camera module emits an XVS pulse signal at 30 FPS, delayed by 11.11 ms after the frame header (SOF) of each frame. The monochrome camera module and the depth-of-field camera module sense the XVS pulse signal emitted by the color camera module once at intervals, and begin shooting the current frame after each sensed XVS pulse signal emitted by the color camera module. That is, after sensing the XVS pulse signal emitted by the color camera module once, the monochrome camera module and the depth-of-field camera module do not sense the first XVS pulse signal received from the color camera module, and wait for the XVS pulse signal after completing the shooting of the current frame, and then sense the second XVS pulse signal received from the color camera module.

[0224] Combination Figure 5As can be seen, the exposure and image output of the color camera module, monochrome camera module, and depth-of-field camera module will remain synchronized at the end of the frame, and their exposure and image output timestamps will be consistent. For example, the duration of each two frames for the color camera module is 66.66ms, the actual interval between each frame for the monochrome camera module is 66.66ms, and the actual interval between each frame for the depth-of-field camera module is also 66.66ms. The actual effective duration of each frame for the color camera module, monochrome camera module, and depth-of-field camera module is consistent, maintaining a consistent exposure and image output timestamp.

[0225] Optionally, in this embodiment, a counter may be provided inside the slave camera module. The slave camera module can use the counter to count the number of XVS pulse signals received from the master camera module. The slave camera module can determine which XVS pulse signal it has received this time based on the counter's count result, determine whether the configured interval for sensing XVS pulse signals has been reached, and further determine whether to sense the XVS pulse signal received from the master camera module this time.

[0226] As described in step 307, the terminal device can capture images corresponding to each camera module through the main camera module and the secondary camera modules. Then, the terminal device can perform fusion processing on the images captured by the main camera module and the secondary camera modules to obtain the final captured image. For example, the method also includes step 308.

[0227] 308. The images captured by the main camera module and the secondary camera module are fused to obtain the final captured image.

[0228] In this application, the terminal device performs fusion processing on the images captured by the main camera module and the images captured by the secondary camera module, and the resulting final captured image can be referred to as the first image. For example, the terminal device can perform fusion processing on the first frame captured by the main camera module and the second frame captured by the secondary camera module to obtain the first image.

[0229] Optionally, when the terminal device performs fusion processing on images captured by the main camera module and the secondary camera module, it can select the most recently captured frame from each module for fusion processing. For example, after the color camera module captures a frame, when the terminal device performs fusion processing on images captured by the color camera module, the monochrome camera module, and the depth-of-field camera module, if the monochrome camera module and the depth-of-field camera module also sensed the XVS pulse signal and captured a frame when the color camera module captured that frame, then the terminal device will perform fusion processing on the three frames currently captured by the color camera module, the monochrome camera module, and the depth-of-field camera module. If the color camera module captures this frame while the monochrome camera module senses the XVS pulse signal and captures a frame, but the depth camera module does not sense the XVS pulse signal, the terminal device can fuse the current frame image captured by the color camera module and the monochrome camera module, as well as the image captured by the depth camera module last time it sensed the XVS pulse signal.

[0230] In the shooting method provided in this application embodiment, the terminal device configures different exposure frame rates for the main camera module and the slave camera module, enabling them to shoot at different exposure frame rates. This minimizes the power consumption of the slave camera module due to unnecessarily high frame rates, thereby effectively reducing the power consumption of the terminal device in multi-camera collaborative shooting scenarios. Simultaneously, by controlling the start time of exposure and image output of the slave camera module based on the exposure frame rates of the main and slave camera modules during the shooting process, the exposure and image output of the slave camera module can be synchronized with that of the main camera module at the end of the frame, ensuring the uniformity of the images captured by the main and slave camera modules, thus improving the final image output quality.

[0231] In other words, the target camera module in this method can achieve hard synchronization with the main camera module at any frame rate. This method can minimize the power consumption of the terminal device while ensuring the shooting effect of multiple camera modules.

[0232] For example, the aforementioned Figure 5In the example shown, the color camera module can shoot at an exposure frame rate of 30 FPS, while the monochrome camera module and depth-of-field camera module can each shoot at an exposure frame rate of 18 FPS. Simultaneously, the exposures of the color, monochrome, and depth-of-field camera modules will remain synchronized at the end of each frame to ensure image consistency. Since the exposure frame rates of the monochrome and depth-of-field camera modules are lower than those of the color camera module, they do not need to maintain the same exposure frame rate as the color camera module. Therefore, the monochrome and depth-of-field camera modules can consume less power.

[0233] Optionally, when the terminal device takes pictures using the main camera module and the slave camera modules, during the first frame pre-exposure stage when the main camera module and each slave camera module start working, the first frame length of the main camera module and each slave camera module can be controlled to be the same (i.e., the first frame of the exposed image has the same length). This ensures that the first frame output time of the main camera module and each slave camera module is consistent, thus ensuring that the main camera module and each slave camera module maintain the same frame output time when the first frame is output. Afterwards, starting from the second frame, the terminal device controls the main camera module and each slave camera module to take pictures in the manner described above.

[0234] For example, taking the example of a color camera module as the main camera module in the aforementioned embodiments, and using a monochrome camera module and a depth-of-field camera module as examples, assuming that the frame intervals corresponding to the color camera module, monochrome camera module, and depth-of-field camera module during shooting are approximately 33 milliseconds (ms), 55 ms, and 55 ms respectively, the terminal device can control the frame length of the first frame corresponding to the color camera module, monochrome camera module, and depth-of-field camera module to be the same during the first frame pre-exposure stage when the color camera module, monochrome camera module, and depth-of-field camera module start working. For example, the terminal device can compress the duration of the vblank of the first frame corresponding to the monochrome camera module and depth-of-field camera module, controlling the frame length of the first frame corresponding to the color camera module, monochrome camera module, and depth-of-field camera module to be 33 ms. Alternatively, the terminal device can extend the duration of the vblank for the first frame corresponding to the color camera module, controlling the frame length of the first frame for the color camera module, the monochrome camera module, and the depth-of-field camera module to be 55ms, etc.

[0235] This application does not restrict the specific implementation method in which the first frame length of the terminal device controls the main camera module and each slave camera module to be the same.

[0236] Optionally, the terminal device can send the first frame pre-exposure frame length to the main camera module and each slave camera module, and the main camera module and each slave camera module can expose the first frame according to the first frame pre-exposure frame length sent by the terminal device.

[0237] The above embodiments mainly describe the scenario where the master camera module and slave camera module achieve hard synchronization during startup in the shooting method provided in this application. In some other possible scenarios, the scene information of the shooting environment may change, and the terminal device needs to adjust some or all of the camera modules according to the changes in scene information to ensure better shooting results.

[0238] For example, when the ambient brightness changes, the terminal device needs to adjust the exposure time of some or all camera modules to maintain consistent brightness in the shooting effect. When the ambient brightness is very low, a long exposure time is usually required to achieve consistent brightness. If the exposure time is longer than the current frame, the frame length needs to be increased accordingly, i.e., the frame rate needs to be reduced, or simply downsampling. When the ambient brightness returns from a dark scene to normal brightness, the excessively long frame length needs to be restored, i.e., the frame rate needs to be increased, or simply upsampling. When the ambient brightness changes significantly or the sensitivity of the devices varies greatly, flexible upsampling and downsampling of the camera modules can bring better shooting results, achieving consistent exposure effects across multiple cameras and better exposure consistency under different ambient brightness levels, thus improving image fusion.

[0239] The following is an exemplary description of the frame rate adjustment in the shooting method provided in the embodiments of this application.

[0240] For example, Figure 6 This is another schematic flowchart illustrating the imaging method provided in an embodiment of this application. For example... Figure 6 As shown, the method may include:

[0241] 601. Based on changes in scene information and the interval number of pulse signals sensed by the target from the camera module, determine the expected exposure frame rate of the main camera module and the expected exposure frame rate of the target from the camera module.

[0242] Among them, the number of intervals between the target sensing the pulse signal from the camera module during the current initiation phase ( Figure 3The process shown has been determined. Therefore, based on the constraints of hard synchronization itself, the expected exposure frame rate of the main camera module and the expected exposure frame rate of the target slave camera module satisfy the following relationship: the expected exposure frame rate of the main camera module is less than or equal to Z times the expected exposure frame rate of the target slave camera module, or, Z times the frame length corresponding to the expected exposure frame rate of the main camera module is greater than or equal to the frame length corresponding to the expected exposure frame rate of the target slave camera module. Z is an integer greater than 0, and Z equals the value of the interval number of the configured sensing pulse signal of the target slave camera module plus 1. That is, Z equals Y as described in the above embodiment.

[0243] Taking the expected exposure frame rate of the main camera module as Sm_new and the expected exposure frame rate of the target secondary camera module as Ss_new as an example, the relationship between the expected exposure frame rates of the main camera module and the target secondary camera module can be expressed as: Sm_new ≤ Z * Ss_new. Alternatively, taking the frame length corresponding to the expected exposure frame rate of the main camera module as L... Sm_new The target is L, the expected exposure frame rate of the camera module. Ss_new For example, the relationship between the expected exposure frame rate of the main camera module and the expected exposure frame rate of the target secondary camera module can be expressed as: L Sm_new *Z≥L Ss_new .

[0244] When the above relationship is satisfied between the expected exposure frame rate of the main camera module and the expected exposure frame rate of the target secondary camera module, the desired frame rate combination can always be achieved between the expected exposure frame rate of the main camera module and the expected exposure frame rate of the target secondary camera module.

[0245] Optionally, compared to the first and second exposure frame rates during the initial flow phase, when determining the expected exposure frame rates of the main camera module and the target secondary camera module based on changes in scene information and the intervals at which the target senses pulse signals from the camera module, the following situations may occur: 1) The expected exposure frame rate of the main camera module changes compared to the first exposure frame rate, and the expected exposure frame rate of the target secondary camera module changes compared to the second exposure frame rate; 2) The expected exposure frame rate of the main camera module is equal to the first exposure frame rate (i.e., it has not changed), and the expected exposure frame rate of the target secondary camera module changes compared to the second exposure frame rate; 3) The expected exposure frame rate of the main camera module changes compared to the first exposure frame rate, and the expected exposure frame rate of the target secondary camera module is equal to the second exposure frame rate (i.e., it has not changed).

[0246] It should be noted that the choice between the expected exposure frame rate of the main camera module and the expected exposure frame rate of the target secondary camera module is determined by the terminal device based on changes in specific scene information. This application does not restrict the decision-making mechanism of the terminal device.

[0247] For example, assuming the main camera module's first exposure frame rate is 30 FPS during the initial exposure phase, and the target camera module's second exposure frame rate is 24 FPS, then the delay time (represented by "XVS delay") for the main camera module to emit the pulse signal during the initial exposure phase is 25 ms. At this time, the duration of each two frames for the main camera module is 66.66 ms; the target camera module's own frame length is 41.66 ms, and the actual interval between each frame is 66.66 ms. The actual effective duration of each frame for both the main camera module and the target camera module is consistent, maintaining consistent exposure timestamps. However, when the shooting environment suddenly darkens and the brightness is very low, assuming the target camera module has poor light sensitivity and requires an exposure of more than 40 ms, the frame rate of the main camera module can be kept constant (i.e., the expected exposure frame rate of the main camera module equals the first exposure frame rate), while the frame rate of the target camera module is reduced to meet the need for a longer exposure (i.e., the expected exposure frame rate of the target camera module is less than the second exposure frame rate). For example, if it is necessary to reduce the exposure frame rate of the target from the camera module from 24 FPS to 15 FPS, step 601 can determine that the expected exposure frame rate of the target from the camera module is 15 FPS.

[0248] Similarly, the scenarios for increasing frame rate are similar to those for decreasing frame rate, so I won't go into details.

[0249] 602. Adjust the exposure frame rate of the main camera module to the desired exposure frame rate of the main camera module, and adjust the exposure frame rate of the target secondary camera module to the desired exposure frame rate of the target secondary camera module.

[0250] 603. Adjust the exposure frame rate of the remaining camera modules so that the exposure frame rate of the remaining camera modules meets the following conditions: the expected exposure frame rate of the target camera module is an integer multiple of the exposure frame rate of the remaining camera modules, and the exposure frame rate of the remaining camera modules is greater than or equal to the minimum exposure frame rate of the remaining camera modules.

[0251] In step 603, the exposure frame rate of the remaining camera modules is adjusted so that the exposure frame rate of the remaining camera modules satisfies the condition that the expected exposure frame rate of the target camera module is an integer multiple of the exposure frame rate of the remaining camera modules. This can be achieved by adjusting the exposure frame rate of the remaining camera modules in step 304 above, so that the exposure frame rate of the remaining camera modules satisfies the condition that the second exposure frame rate is an integer multiple of the exposure frame rate of the remaining camera modules. This will not be elaborated further.

[0252] It should also be understood that, similar to step 304 above, for other scenarios where the number of camera modules is 0, step 603 does not actually exist in the shooting method provided in this application embodiment.

[0253] 604. Based on the expected exposure frame rate of the main camera module and the expected exposure frame rate of the target secondary camera module, configure the delay time for emitting pulse signals to the main camera module.

[0254] The process of configuring the delay time for emitting pulse signals for the main camera module based on the expected exposure frame rate of the main camera module and the expected exposure frame rate of the target secondary camera module can refer to the process of configuring the delay time for emitting pulse signals for the main camera module based on the first exposure frame rate and the second exposure frame rate in step 305 above.

[0255] For example, taking the expected exposure frame rate of the main camera module as Sm_new and the expected exposure frame rate of the target secondary camera module as Ss_new, the terminal device configures the delay time of the pulse signal for the main camera module based on Sm_new and Ss_new. This can include: using Sm_new as a reference, determining the first frame rate Ss1_new that can be divided by Sm_new by decreasing the frame rate downwards based on Ss_new, and configuring the difference between the frame length corresponding to Ss1_new and the frame length corresponding to Ss_new (which can be the absolute value of the difference) as the delay time for the main camera module to emit the pulse signal.

[0256] For example, as described above, when the expected exposure frame rate of the main camera module is 30 FPS and the expected exposure frame rate of the target secondary camera module is 15 FPS, the terminal device can use 30 FPS as a baseline and determine the first frame rate divisible by 30 FPS (itself) by decreasing the frame rate downwards from 15 FPS. At this point, the terminal device can determine that the difference between the frame length corresponding to Ss1_new and the frame length corresponding to Ss_new is 0 ms. The terminal device can then configure the delay time for the pulse signal emitted by the main camera module to be 0 ms.

[0257] Alternatively, in some other implementations, the terminal device can also use the frame length corresponding to the expected exposure frame rate of the main camera module (such as L). Sm_new Based on the baseline, the frame length (e.g., L) corresponding to the expected exposure frame rate of the target from the camera module is used. Ss_new Increasing the frame length upwards determines the first element that can be L. Sm_new Frame rate L divisible Ss1_new , will L Ss1_new With L Ss_new The difference between the values ​​(which can be the absolute values ​​of the difference) is configured to determine the delay time for the pulse signal emitted by the main camera module. There are no restrictions on this.

[0258] 605. Shooting is performed by a main camera module and a slave camera module. During the shooting process, the main camera module emits a pulse signal at the moment corresponding to the frame header of each frame after a delay of the pulse signal emission time. The slave camera module senses the pulse signal emitted by the main camera module according to the configured interval number of pulse signal sensing, and starts shooting the current frame after sensing the pulse signal.

[0259] Step 605 can be referred to in step 307 above, and will not be repeated here.

[0260] It should be understood that the delay time for emitting the pulse signal in step 605 refers to the delay time for emitting the pulse signal configured for the main camera module in step 604 based on the expected exposure frame rate of the main camera module and the expected exposure frame rate of the target secondary camera module. The delay time for emitting the pulse signal in step 307 refers to the delay time for emitting the pulse signal configured for the main camera module in step 305 based on the first exposure frame rate and the second exposure frame rate.

[0261] In this application, the delay time for emitting the pulse signal configured for the main camera module in step 305 can be referred to as the first duration, and the delay time for emitting the pulse signal configured for the main camera module in step 605 can be referred to as the second duration. The expected exposure frame rate of the main camera module can be referred to as the fourth exposure frame rate, and the expected exposure frame rate of the target secondary camera module can be referred to as the fifth exposure frame rate. That is, in this application, after detecting a change in the scene information of the shooting scene, the terminal device can adjust the exposure frame rate of the first camera module to the fourth exposure frame rate, adjust the exposure frame rate of the second camera module to the fifth exposure frame rate, control the first camera module to shoot at the fourth exposure frame rate, and control the second camera module to cooperate with the first camera module to shoot at the fifth exposure frame rate.

[0262] In this scenario, when the terminal device controls the first camera module to shoot at a fourth exposure frame rate and controls the second camera module to shoot in coordination with the first camera module at a fifth exposure frame rate, the current frame captured by the first camera module can be referred to as the third frame, and the current frame captured by the second camera module can be referred to as the fourth frame. That is, the terminal device controlling the first camera module to shoot at a fourth exposure frame rate and controlling the second camera module to shoot in coordination with the first camera module at a fifth exposure frame rate can include: the terminal device controlling the first camera module to start shooting the third frame and controlling the second camera module to start shooting the fourth frame, wherein the start time of the fourth frame is delayed by a second duration compared to the start time of the third frame, and the second duration is related to both the fourth and fifth exposure frame rates.

[0263] The second duration is related to the fourth and fifth exposure frame rates, including: the second duration is equal to the difference between the frame length corresponding to the sixth exposure frame rate and the frame length corresponding to the fifth exposure frame rate; the sixth exposure frame rate is the maximum value among all values ​​that are less than or equal to the fifth exposure frame rate and can be divided evenly by the fourth exposure frame rate. For example, the sixth exposure frame rate is the aforementioned Ss1_new.

[0264] The process described in steps 601-605 above refers to the process in the frame rate adjustment scenario where the terminal device adjusts the exposure frame rate of the camera module and then takes a picture using the adjusted exposure frame rate through the camera module. After completing the relevant configuration for the frame rate adjustment scenario through the process described in steps 601-604, during the shooting in step 605, the exposure output image from the camera module can still remain synchronized with the exposure output image from the main camera module at the end of the frame.

[0265] In other words, in the shooting method provided in this application embodiment, even when the scene information of the shooting scene changes and the frame is raised or lowered, it is still possible to ensure that the exposure output of the camera module and the exposure output of the main camera module remain synchronized at the end of the frame. This ensures the uniformity of the images captured by the main camera module and the camera module in the frame-raising and lowering scenario, and improves the output effect of the final captured image.

[0266] For example, Figure 7 This is a schematic diagram illustrating the principle of frame drop in a scene where the camera module only includes the target camera module, as provided in the embodiments of this application. Figure 7As shown, assuming a certain shooting scenario, the terminal device identifies a master camera module and a slave camera module (as the target slave camera module). During the initial flow phase, the exposure frame rate of the master camera module is adjusted to 30 FPS, and the exposure frame rate of the slave camera module is adjusted to 24 FPS. Based on 30 FPS and 24 FPS, the interval for sensing pulse signals is configured to be 1 for the slave camera module, and the delay time for emitting pulse signals is configured to be t for the master camera module. delay (t delay (25ms). During the shooting process through the main camera module and the slave camera module, the main camera module can delay by t at the time corresponding to the frame header (SOF) of each frame at 30FPS. delay Then, an XVS pulse signal is emitted. The camera module can sense the XVS pulse signal emitted by the main camera module once every interval, and start shooting the current frame after sensing the XVS pulse signal emitted by the main camera module each time.

[0267] When the shooting environment suddenly darkens and the brightness is very low, assuming the secondary camera module has poor light sensitivity and requires an exposure time of more than 40ms, if the frame rate of the primary camera module remains unchanged while the frame rate of the secondary camera module is reduced to 15FPS, the delay time t for the pulse signal to be emitted from the primary camera module needs to be configured. delay =0ms. During the shooting process through the main camera module and the slave camera module, the main camera module can emit an XVS pulse signal at 30FPS, 0ms after the time corresponding to the frame header (SOF) of each frame (i.e., without delaying the time of emitting the XVS pulse signal). The slave camera module can sense the XVS pulse signal emitted by the main camera module once every interval, and start shooting the current frame after sensing the XVS pulse signal emitted by the main camera module each time.

[0268] Combination Figure 7 As you can see, the frame rate was reduced from the camera module, but the main camera module and the secondary camera module can still maintain the same exposure timestamp after the frame rate reduction.

[0269] In the shooting method provided in this application embodiment, when the main camera module and the slave camera module are shooting, for scenes with frame rate changes, only 1 to 2 frames of images are needed in between to achieve the desired exposure frame rate of the main camera module and the desired exposure frame rate of the slave camera module, so that the main camera module and the slave camera module can shoot according to the desired exposure frame rate.

[0270] In addition, it is understandable that for Figure 6 In the embodiments shown, the method may further include a step of fusing the images captured by the main camera module and the images captured by the secondary camera module to obtain the final captured image, which will not be described in detail here.

[0271] Optionally, the adjustment of the exposure frame rate of the remaining camera modules other than the target camera module in the above embodiments, so that the exposure frame rate of the remaining camera modules satisfies that the second exposure frame rate is an integer multiple of the exposure frame rate of the remaining camera modules, can also refer to: adjusting the exposure frame rate of the remaining camera modules other than the target camera module so that the second exposure frame rate is proportional to the exposure frame rate of any remaining camera module, and the exposure frame rates of any two remaining camera modules are also proportional to each other, without limitation.

[0272] It should be noted that although the foregoing embodiments of this application use two camera modules and three camera modules as examples, it should be understood that the shooting method provided in this application can be applied to scenarios where four camera modules or more other camera modules are used for collaborative shooting. This application does not limit the number of camera modules.

[0273] It should be understood that the above embodiments are merely illustrative examples of the imaging methods provided in this application. In other possible implementations, some execution steps may be omitted or added to the above embodiments, or the order of some steps described in the above embodiments may be adjusted, and this application does not impose any limitations on these aspects.

[0274] For example, Figure 8 This is a schematic diagram illustrating the software architecture implementation principle of the terminal device provided in the embodiments of this application. Figure 8 As shown in the embodiments of this application, the software architecture of the terminal device may include: application layer, application framework layer, hardware abstract layer (HAL), kernel layer, and hardware driver layer.

[0275] The application layer can contain multiple applications, such as system applications like email, SMS, calendar, maps, browser, and contact management, as well as photography applications (like camera apps).

[0276] In this embodiment, the terminal device, based on scene information corresponding to the current shooting scene and the capability information of the camera modules configured on the terminal device, determines (decides) the functions of the main camera module and slave camera modules suitable for shooting the current shooting scene, determines the minimum exposure frame rate of the main camera module and each slave camera module, and determines the actual exposure frame rate of the main camera module and each slave camera module. This can be implemented in the application framework layer. The terminal device, based on the decision results of the application framework layer, configures the functions of the main and slave camera modules, as well as the exposure frame rate of the main and slave camera modules and the interval number of pulse signals sensed by the slave camera modules. This can be implemented in the hardware abstraction layer (HAL). The kernel layer can read and write registers according to the configuration results of the hardware abstraction layer (HAL) to control the exposure frame rate of the main and slave camera modules, the frequency of pulse signals emitted by the main camera module, and the interval number of pulse signals sensed by the slave camera modules. The hardware driver layer can call the corresponding camera module to implement the corresponding functions according to the register read and write logic. For example, the camera module can include camera module 1, camera module 2, camera module 3, etc.

[0277] Corresponding to the shooting method described in the foregoing embodiments, this application also provides a shooting device, which can be applied to a terminal device to implement the shooting method described in the foregoing embodiments. The function of this device can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.

[0278] For example, Figure 9 This is a schematic diagram of the imaging device provided in an embodiment of this application. Figure 9 As shown, the device may include: a configuration module 901 and a shooting module 902.

[0279] The configuration module 901 is used to determine the first camera module and the second camera module from the configured camera modules; adjust the exposure frame rate of the first camera module to the first exposure frame rate, and adjust the exposure frame rate of the second camera module to the second exposure frame rate; the first exposure frame rate and the second exposure frame rate are different.

[0280] The shooting module 902 is used to control the first camera module to shoot at a first exposure frame rate, and to control the second camera module to shoot in coordination with the first camera module at a second exposure frame rate.

[0281] The shooting module 902 is specifically used to control the first camera module to start shooting the first frame and control the second camera module to start shooting the second frame. The start time of the second frame is delayed by a first duration compared to the start time of the first frame. The first duration is related to the first exposure frame rate and the second exposure frame rate.

[0282] In some implementations, the first duration is related to both the first and second exposure frame rates, including: the first duration is equal to the difference between the frame length corresponding to the third exposure frame rate and the frame length corresponding to the second exposure frame rate. The third exposure frame rate is the maximum value among all values ​​that are less than or equal to the second exposure frame rate and divisible by the first exposure frame rate.

[0283] Optionally, the shooting module 902 is further configured to control the first camera module to emit a first signal after a first delay in the start time of the first frame; and to control the second camera module to sense the first signal according to a configured interval number of times the first signal is sensed, and in response to the sensed first signal, to control the second camera module to start shooting the second frame.

[0284] Optionally, the configuration module 901 is further configured to configure the second camera module to sense the first signal at intervals according to the first exposure frame rate and the second exposure frame rate, and to configure the first camera module to emit the first signal at a time delayed by a first duration from the start time of the first frame.

[0285] In some implementations, the number of intervals at which the second camera module senses the first signal is equal to the quotient of the first exposure frame rate divided by the third exposure frame rate minus 1.

[0286] In some implementations, the configuration module 901 is further configured to determine a third camera module from the configured camera modules, and adjust the exposure frame rate of the third camera module so that the exposure frame rate of the third camera module satisfies the condition that the second exposure frame rate is an integer multiple of the exposure frame rate of the third camera module. The shooting module 902 is further configured to control the third camera module to perform coordinated shooting with the first and second camera modules at the adjusted exposure frame rate.

[0287] The shooting module 902 is specifically used to control the third camera module to sense the first signal according to the configured interval number of times the first signal is sensed, and in response to the sensed first signal, control the third camera module to start shooting the current frame.

[0288] Optionally, the configuration module 901 is further configured to configure the number of intervals for sensing the first signal for the third camera module based on the second exposure frame rate, the exposure frame rate of the third camera module, and the number of intervals for sensing the first signal configured for the second camera module.

[0289] In some implementations, the number of intervals in which the third camera module senses the first signal is equal to the sum of the number of intervals in which the second camera module senses the first signal plus a first value, where the first value is the quotient of the second exposure frame rate divided by the exposure frame rate of the third camera module minus 1.

[0290] In some implementations, the configuration module 901 is further configured to detect changes in scene information of the shooting scene, adjust the exposure frame rate of the first camera module to the fourth exposure frame rate, and adjust the exposure frame rate of the second camera module to the fifth exposure frame rate. The shooting module 902 is further configured to control the first camera module to shoot at the fourth exposure frame rate, and control the second camera module to perform collaborative shooting with the first camera module at the fifth exposure frame rate.

[0291] The shooting module 902 is specifically used to control the first camera module to start shooting the third frame and control the second camera module to start shooting the fourth frame. The start time of the fourth frame is delayed by a second duration compared to the start time of the third frame. The second duration is related to the fourth exposure frame rate and the fifth exposure frame rate.

[0292] In some implementations, the second duration is related to both the fourth and fifth exposure frame rates, including: the second duration is equal to the difference between the frame length corresponding to the sixth exposure frame rate and the frame length corresponding to the fifth exposure frame rate. The sixth exposure frame rate is the maximum value among all values ​​less than or equal to the fifth exposure frame rate and divisible by the fourth exposure frame rate.

[0293] Optionally, the shooting module 902 is further configured to control the first camera module to emit a first signal after a second delay from the start time of the third frame, and to control the second camera module to sense the first signal according to a configured interval number of times the first signal is sensed, and in response to the sensed first signal, to control the second camera module to start shooting the fourth frame.

[0294] Optionally, the configuration module 901 is further configured to configure the first camera module to emit a first signal at a time delayed by a second duration from the start time of the third frame, based on the fourth exposure frame rate and the fifth exposure frame rate.

[0295] Optionally, the fourth exposure frame rate is less than or equal to Z times the fifth exposure frame rate, or Z times the frame length corresponding to the fourth exposure frame rate is greater than or equal to the frame length corresponding to the fifth exposure frame rate. Here, Z equals the value of the number of intervals for sensing the first signal configured in the second camera module plus 1.

[0296] In some implementations, the capturing module 902 is also used to fuse the first frame and the second frame to obtain the first image.

[0297] Similarly, the shooting device can be used to achieve all the functions of the shooting method described in the foregoing method embodiments, and will not be described in detail hereafter.

[0298] It should be understood that the device may also include other modules or units for implementing the shooting method described in the foregoing embodiments, such as a display module, an auto exposure (AE) module, a control module, etc., which are not shown here one by one.

[0299] It should be understood that the division of units (or modules) in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, all units in the device can be implemented entirely in software through processing element calls; all units can be implemented entirely in hardware; or some units can be implemented in software through processing element calls, while others can be implemented in hardware.

[0300] For example, each unit can be a separate processing element, or it can be integrated into a chip within the device. Alternatively, it can be stored as a program in memory, invoked and executed by a processing element within the device. Furthermore, these units can be integrated in whole or in part, or implemented independently. The processing element described here can also be called a processor, which can be an integrated circuit with signal processing capabilities. In implementation, each step of the above method or each of the above units can be implemented through integrated logic circuits in the processor element or through software invoked by the processing element.

[0301] In one example, the unit in the above device may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0302] For example, when the units in the device can be implemented through a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. Alternatively, these units can be integrated together to form a system-on-a-chip (SOC).

[0303] In one implementation, the units that implement the corresponding steps in the above methods can be implemented in the form of a processing element scheduler. For example, the device may include a processing element and a storage element, wherein the processing element calls a program stored in the storage element to execute the methods described in the above embodiments. The storage element may be a storage element located on the same chip as the processing element, i.e., an on-chip storage element.

[0304] In another implementation, the program used to perform the above methods can be located on a storage element on a different chip than the processing element, i.e., an off-chip storage element. In this case, the processing element calls or loads the program from the off-chip storage element onto the on-chip storage element to call and execute the methods described in the above method embodiments.

[0305] For example, embodiments of this application may also provide an apparatus, such as an electronic device. The electronic device may include a processor and a memory for storing processor-executable instructions; the processor is configured to execute the instructions such that the electronic device performs the method described in the foregoing embodiments. The memory may be located within or outside the electronic device. Furthermore, the processor may include one or more processors.

[0306] For example, the electronic device may be a mobile terminal with a shooting function, such as a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. Alternatively, it may be a professional shooting device such as a digital camera, SLR camera / mirrorless camera, action camera, gimbal camera, drone, etc. The specific type of electronic device is not limited in the embodiments of this application.

[0307] In another implementation, the unit that implements the steps of the above method can be configured as one or more processing elements, which can be integrated circuits, such as one or more ASICs, or one or more DSPs, or one or more FPGAs, or combinations of these types of integrated circuits. These integrated circuits can be integrated together to form a chip.

[0308] For example, this application also provides a chip that can be applied to the aforementioned electronic device. The chip includes one or more interface circuits and one or more processors; the interface circuits and processors are interconnected via lines; the processor receives and executes computer instructions from the memory of the electronic device through the interface circuits to implement the methods described in the foregoing embodiments.

[0309] This application also provides a computer program product, including computer-readable code, which, when executed in an electronic device, causes the electronic device to perform the methods described in the foregoing embodiments.

[0310] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0311] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0312] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium.

[0313] Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product, such as a program. This software product is stored in a program product, such as a computer-readable storage medium, and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0314] For example, embodiments of this application may also provide a computer-readable storage medium storing computer program instructions thereon; when the computer program instructions are executed by an electronic device, the electronic device causes the electronic device to perform the method described in the foregoing embodiments.

[0315] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A photographing method characterized by comprising: The method includes: The terminal device determines the first camera module and the second camera module from the configured camera modules; The terminal device adjusts the exposure frame rate of the first camera module to a first exposure frame rate and adjusts the exposure frame rate of the second camera module to a second exposure frame rate; the first exposure frame rate is greater than the second exposure frame rate; The terminal device controls the first camera module to shoot at the first exposure frame rate, and controls the second camera module to shoot in coordination with the first camera module at the second exposure frame rate; The terminal device controls the first camera module to take pictures at the first exposure frame rate, and controls the second camera module to take pictures in coordination with the first camera module at the second exposure frame rate, including: The terminal device controls the first camera module to start shooting the first frame and controls the second camera module to start shooting the second frame. The start time of the second frame is delayed by a first duration compared to the start time of the first frame. The first duration is equal to the difference between the frame length corresponding to the third exposure frame rate and the frame length corresponding to the second exposure frame rate. The third exposure frame rate is the maximum value among all values ​​that are less than or equal to the second exposure frame rate and can be divided evenly by the first exposure frame rate.

2. The method of claim 1, wherein, The method further includes: The terminal device controls the first camera module to send a first signal after a delay of the first duration at the start time of the first frame; The terminal device controls the second camera module to start capturing the second frame, including: The terminal device controls the second camera module to sense the first signal according to the configured interval number of times the first signal is sensed. In response to the sensed first signal, the terminal device controls the second camera module to start capturing the second frame.

3. The method of claim 2, wherein, The method further includes: The terminal device configures the second camera module to sense the first signal at a certain interval based on the first exposure frame rate and the second exposure frame rate, and configures the first camera module to emit the first signal at a time that is delayed by the first duration from the start time of the first frame.

4. The method according to claim 2 or 3, characterized in that, The number of intervals at which the second camera module senses the first signal is equal to the quotient of the first exposure frame rate divided by the third exposure frame rate minus 1.

5. The method according to claim 2 or 3, characterized in that, The method further includes: The terminal device determines the third camera module from the configured camera modules; The terminal device adjusts the exposure frame rate of the third camera module so that the exposure frame rate of the third camera module satisfies the following condition: the second exposure frame rate is an integer multiple of the exposure frame rate of the third camera module. The terminal device controls the third camera module to perform coordinated shooting with the first camera module and the second camera module at an adjusted exposure frame rate; The terminal device controls the third camera module to perform coordinated shooting with the first camera module and the second camera module at an adjusted exposure frame rate, including: The terminal device controls the third camera module to sense the first signal according to the configured interval number of times the first signal is sensed. In response to the first sensed signal, the terminal device controls the third camera module to start capturing the current frame.

6. The method according to claim 5, characterized in that, The method further includes: The terminal device configures the number of intervals for sensing the first signal for the third camera module based on the second exposure frame rate, the exposure frame rate of the third camera module, and the number of intervals for sensing the first signal configured for the second camera module.

7. The method according to claim 5, characterized in that, The number of intervals in which the third camera module senses the first signal is configured is equal to the sum of the number of intervals in which the second camera module senses the first signal plus a first value, where the first value is the value obtained by subtracting 1 from the quotient of the second exposure frame rate divided by the exposure frame rate of the third camera module.

8. The method according to any one of claims 1-3, characterized in that, The method further includes: If a change in scene information is detected in the shooting scene, the terminal device adjusts the exposure frame rate of the first camera module to the fourth exposure frame rate and the exposure frame rate of the second camera module to the fifth exposure frame rate. The terminal device controls the first camera module to shoot at the fourth exposure frame rate, and controls the second camera module to shoot in coordination with the first camera module at the fifth exposure frame rate; The terminal device controls the first camera module to shoot at the fourth exposure frame rate, and controls the second camera module to shoot in coordination with the first camera module at the fifth exposure frame rate, including: The terminal device controls the first camera module to start shooting the third frame and controls the second camera module to start shooting the fourth frame. The start time of the fourth frame is delayed by a second duration compared to the start time of the third frame. The second duration is related to the fourth exposure frame rate and the fifth exposure frame rate.

9. The method according to claim 8, characterized in that, The second duration is related to the fourth exposure frame rate and the fifth exposure frame rate, and includes: The second duration is equal to the difference between the frame length corresponding to the sixth exposure frame rate and the frame length corresponding to the fifth exposure frame rate; the sixth exposure frame rate is the maximum value among all values ​​that are less than or equal to the fifth exposure frame rate and can be divided evenly by the fourth exposure frame rate.

10. The method according to claim 9, characterized in that, The method further includes: The terminal device controls the first camera module to send a first signal after a second delay from the start time of the third frame; The terminal device controls the second camera module to start capturing the fourth frame, including: The terminal device controls the second camera module to sense the first signal according to the configured interval number of times the first signal is sensed. In response to the first sensed signal, the terminal device controls the second camera module to start capturing the fourth frame.

11. The method according to claim 10, characterized in that, The method further includes: The terminal device configures the first camera module to emit a first signal at a time that is delayed by the second duration from the start time of the third frame, based on the fourth exposure frame rate and the fifth exposure frame rate.

12. The method according to claim 11, characterized in that, The fourth exposure frame rate is less than or equal to Z times the fifth exposure frame rate, or Z times the frame length corresponding to the fourth exposure frame rate is greater than or equal to the frame length corresponding to the fifth exposure frame rate. Where Z equals the value of the interval number of the second camera module's sensing of the first signal plus 1.

13. The method according to any one of claims 1-3, characterized in that, The method further includes: The terminal device performs a fusion process on the first frame and the second frame to obtain a first image.

14. A shooting device, characterized in that, The device includes: A configuration module is used to determine a first camera module and a second camera module from the configured camera modules; adjust the exposure frame rate of the first camera module to a first exposure frame rate, and adjust the exposure frame rate of the second camera module to a second exposure frame rate; the first exposure frame rate and the second exposure frame rate are different; The shooting module is used to control the first camera module to shoot at the first exposure frame rate, and to control the second camera module to shoot in coordination with the first camera module at the second exposure frame rate; The shooting module is specifically used to control the first camera module to start shooting the first frame and control the second camera module to start shooting the second frame. The start time of the second frame is delayed by a first duration compared to the start time of the first frame. The first duration is equal to the difference between the frame length corresponding to the third exposure frame rate and the frame length corresponding to the second exposure frame rate. The third exposure frame rate is the maximum value among all values ​​that are less than or equal to the second exposure frame rate and can be divided evenly by the first exposure frame rate.

15. An electronic device, characterized in that, include: A processor, and a memory for storing processor-executable instructions; When the processor is configured to execute the instructions, it causes the electronic device to implement the method as described in any one of claims 1-13.

16. A computer-readable storage medium having computer program instructions stored thereon; characterized in that, When the computer program instructions are executed by the electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-13.

Images