Photographing method, apparatus, device, and storage medium

Abstract

The application provides a photographing method and device, equipment and storage medium, and relates to the field of photographing. The method comprises the following steps: a terminal device determines a first camera module and a second camera module used for photographing from configured camera modules, and configures an exposure frame rate of the first camera module and an exposure frame rate of the second camera module. The number of the second camera modules is one or more. The exposure frame rate of the first camera module is an integer multiple of the exposure frame rate of the second camera module. The terminal device performs photographing through the first camera module and each second camera module. The method can effectively reduce the power consumption generated by the camera module under the premise of ensuring the photographing effect.

Description

Technical Field

[0001] This application relates to the field of photography, and more particularly to a photography method, apparatus, device, and storage medium. Background Technology

[0002] In multi-camera collaborative shooting scenarios, a terminal device (such as a mobile phone) can be equipped with multiple camera modules. During shooting, the terminal device can simultaneously capture multiple images corresponding to each camera module. The terminal device can then fuse these images to obtain the final captured image. To ensure the uniformity of the images captured by multiple camera modules, the terminal device needs to ensure that the timestamps of the start of exposure for each frame of the multiple camera modules remain consistent during the shooting phase.

[0003] Currently, terminal devices can divide multiple camera modules into a master camera module and other slave camera modules. Through hard synchronization control technology, the master camera module and slave camera modules are kept in sync with each frame of the image during the shooting stage by ensuring that the timestamps at the start of exposure are consistent.

[0004] However, in the current method of using hard synchronization control technology to keep the timestamps of the start of exposure of each frame of the main camera module and the slave camera module consistent during the shooting stage, the main camera module and the slave camera module need to maintain the same exposure frame rate, which will result in high power consumption of the terminal device. Summary of the Invention

[0005] This application provides a shooting method, apparatus, device, and storage medium that can dynamically configure master-slave attributes for multiple camera modules of a terminal device based on the current shooting scenario, and dynamically configure an appropriate exposure frame rate for each camera module. This can effectively reduce the power consumption generated by multiple camera modules while ensuring the shooting effect of multiple camera modules.

[0006] In a first aspect, embodiments of this application provide a shooting method that can be applied to a terminal device, the terminal device including multiple (at least two) camera modules. The method includes:

[0007] The terminal device identifies a first camera module and a second camera module from its configured camera modules, and configures the exposure frame rate of the first camera module and the exposure frame rate of the second camera module. The number of second camera modules can be one or more; the exposure frame rate of the first camera module is an integer multiple of the exposure frame rate of the second camera module. The terminal device takes pictures using the first camera module and each of the second camera modules.

[0008] Among them, the first camera module and the second camera module are camera modules that enable multi-camera collaborative shooting.

[0009] In this method, the first camera module can be referred to as the main camera module, and the second camera module can be referred to as the slave camera module. The terminal device dynamically adjusts the exposure frame rate of the main camera module and each slave camera module, ensuring that the exposure frame rate of the main camera module is an integer multiple of the exposure frame rate of the slave camera modules. This allows the slave camera modules to achieve collaborative shooting in hard-synchronized scenes with the main camera module at a lower exposure frame rate (reducing unnecessary exposure frequencies of the slave camera modules). Because the exposure frame rate of the slave camera modules can be lower than that of the main camera module, the power consumption of the slave camera modules can be lower. In other words, this method can effectively reduce the power consumption of the terminal device when performing multi-camera collaborative shooting.

[0010] For example, taking a terminal device that uses a color camera module (main camera module), a monochrome camera module (slave camera module), and a depth-of-field camera module (slave camera module) for collaborative shooting, this method allows the color camera module to shoot at an exposure frame rate of 30 FPS, the monochrome camera module at 15 FPS, and the depth-of-field camera module at 10 FPS. Simultaneously, the frame headers of each frame from the color, monochrome, and depth-of-field camera modules maintain a consistent timestamp (i.e., consistent image output time). Since the exposure frame rates of the monochrome and depth-of-field camera modules are lower than those of the color camera module, they do not need to maintain consistency with the color camera module's exposure frame rate. Therefore, the power consumption of the monochrome and depth-of-field camera modules can be lower.

[0011] In one possible design, before the terminal device takes pictures through the first camera module and each second camera module, the method further includes: for each second camera module, the terminal device configures the number of intervals for sensing the first signal for the second camera module according to the exposure frame rate of the first camera module and the exposure frame rate of the second camera module.

[0012] The terminal device takes pictures through a first camera module and each second camera module, including: during the shooting process, the first camera module emits a first signal at the time corresponding to the frame header of each frame; each second camera module senses the first signal emitted by the first camera module according to the number of intervals in which the first signal is sensed; in response to sensing the first signal emitted by the first camera module, each second camera module starts shooting the current frame.

[0013] In this design, the terminal device configures the second camera module to sense the first signal at intervals based on the exposure frame rate of the first camera module and the exposure frame rate of the second camera module. The first camera module emits the first signal at the moment corresponding to the frame header of each frame. In response to sensing the first signal emitted by the first camera module, each second camera module starts shooting the current frame. This allows the frame headers of each frame of the first camera module and each second camera module to maintain the same timestamp (i.e., consistent image output time) during shooting, thereby achieving collaborative shooting in hard synchronization scenarios.

[0014] In some embodiments, the terminal device determines the first camera module and the second camera module from the camera modules configured on the terminal device, including: the terminal device acquiring scene information of the current shooting scene and capability information of the camera modules configured on the terminal device; the terminal device determining the first camera module and the second camera module from the camera modules configured on the terminal device based on the scene information and the capability information.

[0015] For example, when a terminal device starts the shooting function (such as launching and running a shooting application), it can obtain scene information of the current shooting scene.

[0016] For example, the scene information includes one or more of the following: focal length information, lighting information, and the user's shooting requirements.

[0017] In one possible design, the terminal device acquires scene information of the current shooting scene, including: the terminal device acquires scene information of the current shooting scene based on one or more of the following: the preview image corresponding to the current shooting scene, sensor information (such as light information collected by a light sensor), and the first operation.

[0018] The first operation is to select a shooting mode and / or set shooting parameters on the first interface displayed on the terminal device.

[0019] For example, the terminal device may include a shooting application, which can be launched and run in response to a user's operation to initiate the shooting application. After launching and running the shooting application, the terminal device initiates the shooting function. The terminal device can use the shooting application to call the configured camera module to perform the shooting function. After launching and running the shooting application, the terminal device can display a shooting interface (this shooting interface is the first interface). The shooting interface can include function controls corresponding to various shooting modes that the user can select, such as: large aperture mode, portrait mode, bokeh mode, slow motion mode, high dynamic range mode, night scene mode, etc. The user can select one or more shooting modes by clicking the corresponding function controls on the shooting interface. The terminal device can determine the user's shooting needs in response to the user's selection of a shooting mode. For example, if the user clicks the function control corresponding to portrait mode, the terminal device can determine that the user's shooting needs include: shooting a portrait or shooting in portrait mode.

[0020] In some embodiments, the terminal device can also obtain scene information corresponding to the current shooting scene by recognizing the preview image corresponding to the current shooting scene. For example, the terminal can determine whether the user's shooting requirement is a portrait or a scene by recognizing the preview image corresponding to the current shooting scene, or determine the lighting information of the current shooting scene, etc. Alternatively, the terminal device can also combine the recognition results of the preview image corresponding to the current shooting scene, as well as the detection parameters of hardware such as the light sensor, and the user's operation, to comprehensively determine the scene information of the current shooting scene.

[0021] Optionally, the camera module's capability information may include one or more of the following: maximum exposure frame rate, reasonable focal length for shooting, and output color mode.

[0022] For example, assuming the terminal device is equipped with camera modules including: a color camera module, a monochrome camera module, a depth-of-field camera module, a telephoto camera module, and a wide-angle camera module, then the capability information of the aforementioned camera modules can be as follows:

[0023] 1) The maximum exposure frame rate of the color camera module is 30 frames per second (FPS) or 60 FPS. The reasonable focal length for shooting is between 0.8X and 5X, and the output color mode is color.

[0024] 2) The maximum exposure frame rate of the black and white camera module is 30FPS, the reasonable focal length for shooting is between 0.8X and 5X, and the output color mode is no color or black and white.

[0025] 3) The maximum exposure frame rate of the depth-of-field camera module is 30FPS, the reasonable focal length for shooting is between 0.8X and 5X, the output color mode is color, and it can quickly and at a distance (1-10 meters) obtain more accurate depth-of-field information.

[0026] 4) The maximum exposure frame rate of the telephoto camera module is 30FPS, the reasonable focal length for shooting is 3X or higher, and the output color mode is color.

[0027] 5) The maximum exposure frame rate of the wide-angle camera module is 30FPS, the reasonable focal length for shooting is between 0.1X and 1X, and the output color mode is color.

[0028] In one possible design, the terminal device configures the exposure frame rate of the first camera module and the exposure frame rate of the second camera module, including:

[0029] Based on scene information of the current shooting scene and the capability information of the camera modules configured on the terminal device, the terminal device determines the minimum exposure frame rate for the first camera module and the second camera module, respectively. The terminal device configures the exposure frame rate of the first camera module as a first exposure frame rate, which is greater than or equal to the minimum exposure frame rate of the first camera module. The terminal device configures the exposure frame rate of each second camera module such that the exposure frame rate of each second camera module satisfies the following conditions: the first exposure frame rate is an integer multiple of the exposure frame rate of the second camera module, and the exposure frame rate of the second camera module is greater than or equal to the minimum exposure frame rate of the second camera module. It can be understood that when the first exposure frame rate is an integer multiple of the exposure frame rate of the second camera module, the exposure frame rate of the second camera module is less than or equal to the first exposure frame rate.

[0030] For example, assume there are i second camera modules, where i is a positive integer; the minimum exposure frame rate of the first camera module is S0, the minimum exposure frame rate of the first second camera module is S1, the minimum exposure frame rate of the second second camera module is S2, ..., and the minimum exposure frame rate of the i-th second camera module is Si. Then, the terminal device can adjust (i.e., configure) the exposure frame rate S'0 of the first camera module to be S'0 = S0. The method for adjusting the exposure frame rate of the first to the i-th second camera modules is as follows:

[0031] Based on S'0, adjust the exposure frame rate S'1 of the first second camera module so that S'1 satisfies: S'0=X1*S'1, where X1 is a positive integer, and S1≤S'1≤S'0; adjust the exposure frame rate S'2 of the second second camera module so that S'2 satisfies: S'0=X2*S'2, where X2 is a positive integer, and S2≤S'2≤S'0; ...; and so on, adjust the exposure frame rate S'i of the i-th second camera module so that S'i satisfies: S'0=Xi*S'i, where Xi is a positive integer, and Si≤S'i≤S'0.

[0032] “≤” means less than or equal to.

[0033] In another possible design, the terminal device configures the exposure frame rate of each second camera module such that the exposure frame rate of each second camera module satisfies: the first exposure frame rate is an integer multiple of the exposure frame rate of the second camera module, including:

[0034] The terminal device configures the exposure frame rate of each second camera module so that the first exposure frame rate and the exposure frame rate of any second camera module are proportionally related, and the exposure frame rates of any two second camera modules are also proportionally related.

[0035] That is, the exposure frame rates of any two camera modules in the set consisting of the first camera module and all the second camera modules are proportional to each other.

[0036] For example, assume there are i second camera modules, where i is a positive integer; the minimum exposure frame rate of the first camera module is S0, the minimum exposure frame rate of the first second camera module is S1, the minimum exposure frame rate of the second second camera module is S2, ..., and the minimum exposure frame rate of the i-th second camera module is Si. Then, the terminal device can adjust the exposure frame rate S'0 of the first camera module to S'0 = S0. The method for adjusting the exposure frame rate of the first to the i-th second camera modules is as follows:

[0037] Using S'0 as a reference, adjust the exposure frame rate S'1 of the first second camera module to the exposure frame rate S'i of the i-th second camera module, so that S'1 to S'i satisfy:

[0038] S'0=X1*S'1=X1*X2*S'2=…=X1*X2*…*Xi-1*Xi*S'i; X1 to Xi are integers greater than 0, and Si≤S'i≤S'0.

[0039] In this design, the terminal device dynamically adjusts the exposure frame rate of the first and second camera modules using the aforementioned method, and configures the interval number of pulse signals sensed by each second camera module. This allows the second camera modules to achieve collaborative shooting in hard-synchronized scenarios with the first camera module at a lower exposure frame rate than the first camera module. Since the exposure frame rate of the second camera module can be lower than that of the first camera module, the power consumption of the second camera module can be lower. Furthermore, in this design, the exposure frame rates of any two camera modules in the set consisting of the first camera module and all the second camera modules are proportionally related. This allows the first camera module and all the second camera modules to output multi-camera exposure images with the same timestamp at certain fixed frames. The fewer frames between the output timestamps of the first camera module and all the second camera modules, the better it is for subsequent image fusion. The final image output by the terminal device after image fusion processing can have a better effect.

[0040] In one possible design, the current shooting scene can be divided into (or include) scenes with high time consistency requirements or scenes with low time consistency requirements; the method further includes: the terminal device determining, based on the scene information of the current shooting scene, whether the current shooting scene has high time consistency requirements or low time consistency requirements.

[0041] The terminal device configures the exposure frame rate of each second camera module such that the first exposure frame rate and the exposure frame rate of any second camera module are proportionally multiple of each other, and the exposure frame rates of any two second camera modules are also proportionally multiple of each other, including:

[0042] When the current shooting scene is a scene with high requirements for time consistency, the terminal device configures the exposure frame rate of each second camera module so that the first exposure frame rate and the exposure frame rate of any second camera module are proportional to each other, and the exposure frame rates of any two second camera modules are also proportional to each other.

[0043] In this design, for scenarios with high time consistency requirements, the terminal device is configured with the exposure frame rate of each second camera module so that the exposure frame rates of any two camera modules in the set consisting of the first camera module and all second camera modules are proportional to each other. This can meet the time consistency requirement of the final image capture when shooting with multiple cameras, so as to avoid difficulties in image fusion due to large displacement of objects between different frames.

[0044] For example, in High Dynamic Range (HDR) mode, the subject is typically moving rapidly. Because the subject is in motion, besides the primary camera module requiring a high exposure frame rate, multiple cameras working together necessitate high temporal consistency in the final image capture to prevent difficulties in image fusion due to significant object displacement between frames. Therefore, HDR shooting scenarios can be categorized as having high temporal consistency requirements. Similarly, when the current shooting scene includes moving objects, it can also be classified as a scenario with high temporal consistency requirements. Conversely, scenarios involving stationary objects, or those corresponding to normal mode (non-HDR), can be classified as scenarios with lower temporal consistency requirements.

[0045] For example, when the user selects high dynamic range mode, or when the terminal device identifies a moving object in the current shooting scene based on the preview image, the terminal device (or the decision model within the terminal device) can classify the current shooting scene into a scene with high requirements for temporal consistency based on the scene information. When subsequently adjusting the exposure frame rate of the first and second camera modules, the terminal device can adjust the exposure frame rate of each second camera module so that the exposure frame rates of any two camera modules in the set consisting of the first camera module and all second camera modules are proportionally related to each other.

[0046] In some embodiments, the terminal device configures the exposure frame rate of each second camera module such that the exposure frame rate of each second camera module satisfies: the first exposure frame rate is an integer multiple of the exposure frame rate of the second camera module, and the exposure frame rate of the second camera module is greater than or equal to the minimum exposure frame rate of the second camera module, including:

[0047] For each second camera module, when the range from the lowest exposure frame rate of the second camera module to the first exposure frame rate includes multiple second exposure frame rates that satisfy the condition that the first exposure frame rate is an integer multiple of the second exposure frame rate, the terminal device configures the minimum value among the multiple second exposure frame rates as the exposure frame rate of the second camera module.

[0048] For example, during the process of adjusting the exposure frame rate of each second camera module by the terminal device, for the i-th second camera module, the range between the minimum exposure frame rate of the second camera module and the minimum exposure frame rate of the first camera module may include multiple S'i, where the minimum exposure frame rate of the first camera module is an integer multiple of S'i, and S'i is the aforementioned second exposure frame rate. In this case, the terminal device can configure the minimum value among the multiple S'i as the exposure frame rate of the second camera module. That is, the terminal device can adjust the exposure frame rate S'i of the i-th second camera module to the minimum one.

[0049] For example, assuming the first exposure frame rate S'0 is 30 FPS and the minimum exposure frame rate Si of the i-th second camera module is 10 FPS, the terminal device adjusts the exposure frame rate S'i of the i-th second camera module so that S'i satisfies: S'0=Xi*S'i, where Xi is an integer greater than 0. When Si≤S'i≤S'0, the combination of Xi and S'i can include the three types shown in (1) to (3) below:

[0050] (1) Xi is 1, S'i is 30 FPS;

[0051] (2) Xi is 2, S'i is 15 FPS;

[0052] (3) Xi is 3, S'i is 10 FPS;

[0053] At this point, the terminal device can adjust the exposure frame rate S'i of the i-th second camera module to 10 FPS.

[0054] Optionally, during the process of adjusting the exposure frame rate of each second camera module in the manner described above, for the i-th second camera module, when there are multiple combinations of Xi and S'i corresponding to the i-th second camera module, the terminal device may also adjust the exposure frame rate S'i of the i-th second camera module to any one of the multiple combinations, without any restrictions.

[0055] Optionally, the method further includes: the terminal device controlling the first camera module and each of the second camera modules to maintain a consistent exposure time for the first frame during shooting. That is, the first frame exposure time is kept consistent.

[0056] For example, when the terminal device takes pictures using the first camera module and each of the second camera modules, during the first frame pre-exposure stage when the first camera module and each of the second camera modules start working, the first frame length of the first camera module and each of the second camera modules can be controlled to be the same. This ensures that the image output time of the first frame of the first camera module and each of the second camera modules is consistent, thus ensuring that the first camera module and each of the second camera modules maintain the same frame output time when the first frame is output. Then, starting from the second frame, the first camera module emits a first signal at the time corresponding to the frame header of each frame; each of the second camera modules senses the first signal emitted by the first camera module according to the configured interval number of times the first signal is sensed, and starts shooting the current frame after each time the first signal emitted by the first camera module is sensed.

[0057] Optionally, the method further includes: the terminal device performing fusion processing on the images captured by the first camera module and the second camera module to obtain a first image.

[0058] The first image is the final image captured by the terminal device through multi-camera collaborative shooting.

[0059] Optionally, when the terminal device performs fusion processing on the images captured by the first camera module and each of the second camera modules, it can select the most recently captured frame image from the first camera module and each of the second camera modules for fusion processing.

[0060] For example, when a terminal device captures an image using the color camera module, and then performs fusion processing on the images captured by the color camera module, the monochrome camera module, and the depth camera module, if the monochrome camera module and the depth camera module also sensed the XVS pulse signal and captured their respective images while the color camera module was capturing this image, then the terminal device will fuse the three images currently captured by the color camera module, the monochrome camera module, and the depth camera module. If the monochrome camera module sensed the XVS pulse signal and captured its image while the color camera module was capturing this image, but the depth camera module did not sense the XVS pulse signal, then the terminal device can fuse the current image captured by the color camera module and the monochrome camera module, along with the image captured by the depth camera module last time it sensed the XVS pulse signal.

[0061] Secondly, embodiments of this application provide a shooting device that can be applied to a terminal device to implement the shooting method described in the first aspect. The function of this device can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, the device may include a configuration module, a shooting module, etc.

[0062] The configuration module is used to determine the first camera module and the second camera module from the camera modules configured in the terminal device, and to configure the exposure frame rate of the first camera module and the exposure frame rate of the second camera module; the number of the second camera modules is one or more; the exposure frame rate of the first camera module is an integer multiple of the exposure frame rate of the second camera module.

[0063] The shooting module is used to take pictures through the first camera module and each of the second camera modules.

[0064] Among them, the first camera module and the second camera module are camera modules that enable multi-camera collaborative shooting.

[0065] In one possible design, the configuration module is further configured to, for each second camera module, determine the number of intervals for sensing the first signal based on the exposure frame rate of the first camera module and the exposure frame rate of the second camera module. During shooting, the first camera module emits the first signal at the moment corresponding to the frame header of each frame; each second camera module senses the first signal emitted by the first camera module according to the number of intervals for sensing the first signal; in response to sensing the first signal emitted by the first camera module, each second camera module begins shooting the current frame.

[0066] In some embodiments, the shooting device further includes: an acquisition module, configured to acquire scene information of the current shooting scene and capability information of the camera modules configured in the terminal device; and a configuration module, specifically configured to determine a first camera module and a second camera module from the camera modules configured in the terminal device based on the scene information and the capability information.

[0067] For example, the scene information includes one or more of the following: focal length information, lighting information, and the user's shooting requirements.

[0068] In one possible design, the acquisition module is specifically used to acquire scene information of the current shooting scene based on one or more of the following: the preview image corresponding to the current shooting scene, sensor information (such as light information collected by a light sensor), and the first operation.

[0069] The first operation is to select a shooting mode and / or set shooting parameters on the first interface displayed on the terminal device.

[0070] Optionally, the camera module's capability information may include one or more of the following: maximum exposure frame rate, reasonable focal length for shooting, and output color mode.

[0071] In one possible design, a configuration module is specifically used to determine the minimum exposure frame rates of the first camera module and the second camera module based on scene information of the current shooting scene and the capability information of the camera modules configured on the terminal device. The exposure frame rate of the first camera module is configured as a first exposure frame rate, which is greater than or equal to the minimum exposure frame rate of the first camera module. The exposure frame rate of each second camera module is configured such that the first exposure frame rate is an integer multiple of the exposure frame rate of the second camera module, and the exposure frame rate of the second camera module is greater than or equal to the minimum exposure frame rate of the second camera module (less than or equal to the first exposure frame rate).

[0072] In another possible design, a configuration module is specifically used to configure the exposure frame rate of each second camera module, such that the first exposure frame rate and the exposure frame rate of any second camera module are proportionally related, and the exposure frame rates of any two second camera modules are also proportionally related.

[0073] In one possible design, the current shooting scene can be divided into (or included) scenes with high time consistency requirements, or scenes with low time consistency requirements. The configuration module, specifically used when the current shooting scene has high time consistency requirements, configures the exposure frame rate of each second camera module, ensuring that the first exposure frame rate and the exposure frame rate of any second camera module are proportionally related, and that the exposure frame rates of any two second camera modules are also proportionally related.

[0074] In some embodiments, the configuration module is specifically configured to, for each second camera module, when the range from the lowest exposure frame rate of the second camera module to the first exposure frame rate includes multiple second exposure frame rates that satisfy the condition that the first exposure frame rate is an integer multiple of the second exposure frame rate, configure the minimum value among the multiple second exposure frame rates as the exposure frame rate of the second camera module.

[0075] Optionally, the shooting module is also used to control the first camera module and each second camera module to keep the exposure time of the first frame of the image consistent during shooting.

[0076] Optionally, the shooting device further includes a fusion module for fusing the images captured by the first camera module and the second camera module to obtain a first image.

[0077] The first image is the final image captured by the terminal device through multi-camera collaborative shooting.

[0078] It should be understood that the shooting device can be used to achieve all the functions of the shooting method described in the first aspect above, and will not be repeated here.

[0079] Thirdly, embodiments of this application provide an electronic device, including: a processor, and a memory for storing processor-executable instructions; the processor is configured to, when executing the instructions, cause the electronic device to implement the method as described in the first aspect and any possible implementation thereof.

[0080] Fourthly, embodiments of this application provide a computer-readable storage medium having computer program instructions stored thereon; when the computer program instructions are executed by an electronic device, the electronic device causes the electronic device to implement the method described in the first aspect and any possible implementation thereof.

[0081] Fifthly, embodiments of this application also provide a computer program product, including computer-readable code, which, when executed in an electronic device, causes the electronic device to implement the method described in the first aspect and any possible implementation thereof.

[0082] In the third to fifth aspects mentioned above, the electronic device can be a mobile terminal with shooting function, such as a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., or it can be a professional shooting device such as a digital camera, SLR camera / mirrorless camera, action camera, gimbal camera, drone, etc.

[0083] The beneficial effects of the second to fifth aspects mentioned above can be referred to in the first aspect, and will not be repeated here.

[0084] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0085] Figure 1 This is a schematic diagram illustrating the hard synchronization control principle of two camera modules.

[0086] Figure 2 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application;

[0087] Figure 3 A flowchart illustrating the multi-camera collaborative shooting method provided in this application embodiment;

[0088] Figure 4A schematic diagram illustrating the hard synchronization control principle of three camera modules provided in this application embodiment;

[0089] Figure 5 A schematic diagram illustrating the principle of the multi-camera synchronization control module provided in this application embodiment;

[0090] Figure 6 Another schematic diagram illustrating the hard synchronization control principle of the three camera modules provided in the embodiments of this application;

[0091] Figure 7 A schematic diagram illustrating the hard synchronization control principle of two camera modules provided in an embodiment of this application;

[0092] Figure 8 This is a schematic diagram illustrating the software architecture implementation principle of the terminal device provided in the embodiments of this application;

[0093] Figure 9 This is a schematic diagram of the imaging device provided in the embodiments of this application;

[0094] Figure 10 This is another schematic diagram of the imaging device provided in the embodiments of this application;

[0095] Figure 11 This is another schematic diagram of the imaging device provided in the embodiments of this application. Detailed Implementation

[0096] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two). The character “ / ” generally indicates that the preceding and following objects are in an “or” relationship.

[0097] References to "one embodiment" or "some embodiments" as used in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes both direct and indirect connections, unless otherwise stated.

[0098] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0099] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0100] In multi-camera collaborative shooting scenarios, a terminal device (such as a mobile phone) can be equipped with multiple (two or more) camera modules, also known as camera module sets. When shooting, the terminal device can simultaneously capture multiple images corresponding to each camera module. The terminal device can then fuse these images to obtain the final captured image. The final captured image can have better shooting quality (or output image quality).

[0101] For example, in one possible scenario, the terminal device may include a color camera and a depth camera. When taking a picture, the terminal device can capture a color image using the color camera and a black-and-white image using the depth camera. The terminal device can then fuse the color image from the color camera and the black-and-white image from the depth camera to obtain the final captured image. The final captured image can contain richer depth information.

[0102] When a terminal device takes pictures using multiple camera modules, in order to ensure the uniformity of the images captured by the multiple camera modules, the terminal device needs to control the timestamps of the start of exposure of each frame of the image to be consistent during the shooting stage.

[0103] Currently, terminal devices primarily use hard synchronization control technology to ensure that the exposure start timestamps of each frame from multiple camera modules remain consistent during the shooting phase. The main principle of hard synchronization control is as follows: When a terminal device uses multiple camera modules for shooting, these modules are divided into a master camera module and other slave camera modules. The master camera module emits an XVS pulse signal at the beginning of each frame during the shooting phase. The other slave camera modules sense this XVS pulse signal. Each time a slave camera module senses the XVS pulse signal from the master camera module, it begins exposure for that frame. After completing exposure for the current frame, each slave camera module waits for the next XVS pulse signal from the master camera module until it senses it, at which point it begins exposure for the next frame. Thus, the exposure start timestamps of each frame from multiple camera modules remain consistent during the shooting phase.

[0104] For example, taking a scenario where a terminal device uses two camera modules to achieve multi-camera collaborative shooting as an example, Figure 1 This is a schematic diagram illustrating the hard synchronization control principle of two camera modules. One camera module acts as the master camera module, and the other acts as the slave camera module.

[0105] like Figure 1As shown, the main camera module emits an XVS pulse signal at the moment corresponding to the frame header of each frame during the shooting phase. The frame header can be represented by the start-of-frame delimiter (SOF), and the frame tail by the end-of-frame delimiter (EOF). When the camera module senses the XVS pulse signal, it can begin exposure and image processing for the current frame. The effective frame length of the main camera module is slightly longer than that of the secondary camera module. For example, the frame length of a camera module generally refers to the length between SOF and the next SOF, while the effective frame length is the sum of the length between SOF and EOF and the length of the non-exposure blank lines (vertical blank, vblank, also known as the blank invalid line time) in each frame. The terminal device can adjust the length of the vblank of the main camera module and the secondary camera module so that the effective frame length of the main camera module is slightly longer than that of the secondary camera module. The vblank represents the time from the end of the last line readout of the previous frame to the start of the first line readout of the next frame. After the slave camera module finishes exposing and capturing the image in the current frame, it will wait for the XVS pulse signal sent by the master camera. It will only start exposing and capturing the image in the next frame when it senses the next XVS pulse signal from the master camera module. In this way, the timestamps of the frame headers of the master camera module and the slave camera module are consistent, achieving the effect of capturing images in the same time at the millisecond level.

[0106] In the aforementioned terminal device, which uses hard synchronization control technology to ensure that the timestamps at the start of exposure for each frame of an image from multiple camera modules remain consistent during the shooting phase, the main camera module and the slave camera modules need to maintain the same exposure frame rate. That is, the exposure frame rate of the slave camera module follows that of the main camera module, and within the same shooting preview time, the main camera module and the slave camera module expose the same number of frames.

[0107] However, maintaining the same exposure frame rate for the main camera module and the slave camera module will result in higher power consumption in the terminal device.

[0108] For example, in multi-camera collaborative shooting scenarios, when a terminal device performs multi-image fusion, it generally needs to use the image output from the main camera module as the benchmark. The main camera module will be set to a higher exposure frame rate (i.e., more exposure frames per unit time) to ensure high dynamic range in the shooting effect. The slave camera modules will follow the exposure frame rate of the main camera module, maintaining the same high frame rate. However, maintaining the same exposure frame rate for both the main and slave camera modules will lead to higher camera power consumption in the terminal device, such as overheating or even burning. When the terminal device reaches a certain temperature limit, the camera module may be unable to maintain high-performance operation, affecting the final shooting effect.

[0109] Against this background, embodiments of this application provide a multi-camera collaborative shooting method (or shooting method), applicable to terminal devices with multiple (e.g., at least two) camera modules. In this method, when the terminal device shoots using multiple camera modules, it can dynamically configure master-slave attributes for the multiple camera modules based on the current shooting scene, and dynamically configure an appropriate exposure frame rate for each camera module. This effectively reduces the power consumption generated by the multiple camera modules while ensuring the shooting effect of the multiple camera modules.

[0110] Optionally, in this application embodiment, the terminal device may be a mobile terminal with shooting function such as a mobile phone, tablet computer, wearable device, vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. Alternatively, it may be a professional shooting device such as a digital camera, SLR camera / mirrorless camera, action camera, gimbal camera, drone, etc. This application embodiment does not limit the specific type of terminal device.

[0111] It should be understood that the terminal device has at least two camera modules for multi-camera collaborative shooting.

[0112] For example, taking a mobile phone as the terminal device, Figure 2 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 2As shown, a mobile phone may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, a sensor module 280, buttons 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc.

[0113] Processor 210 may include one or more processing units, such as application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0114] The controller can serve as the nerve center and command center of the mobile phone. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0115] The processor 210 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory can store instructions or data that the processor 210 has just used or that are used repeatedly. If the processor 210 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0116] In some embodiments, the processor 210 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and / or a USB interface, etc.

[0117] The external storage interface 220 can be used to connect an external storage card, such as a Micro SD card, to expand the phone's storage capacity. The external storage card communicates with the processor 210 through the external storage interface 220 to perform data storage functions. For example, it can save files such as pictures, music, and videos to the external storage card.

[0118] Internal memory 221 can be used to store computer executable program code, which includes instructions. Processor 210 executes various functions and data processing of the mobile phone by running the instructions stored in internal memory 221.

[0119] The internal memory 221 may further include a program storage area and a data storage area. The program storage area may store the operating system, at least one application required for a function (such as a camera application), etc. The data storage area may store data created during phone use (such as image data, phonebook data, etc.). Furthermore, the internal memory 221 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0120] The charging management module 240 receives charging input from the charger. While charging the battery 242, the charging management module 240 can also power the mobile phone via the power management module 241. The power management module 241 connects to the battery 242, the charging management module 240, and the processor 210. The power management module 241 can also receive input from the battery 242 to power the mobile phone.

[0121] The wireless communication function of a mobile phone can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile phone can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0122] Mobile phones can perform audio functions, such as music playback and recording, through an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, and an application processor.

[0123] The sensor module 280 may include a pressure sensor 280A, a gyroscope sensor 280B, a barometric pressure sensor 280C, a magnetic sensor 280D, an accelerometer sensor 280E, a proximity sensor 280F, a proximity light sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, and a bone conduction sensor 280M, etc. When the phone is taking a picture, the sensor module 280 can collect scene information of the current shooting scene, such as collecting the temperature of the current shooting scene through the temperature sensor 280J, and collecting the light intensity of the current shooting scene through the ambient light sensor 280L.

[0124] The mobile phone can take pictures using camera 293. Camera 293 can also be called a camera module. In this embodiment, the number of cameras 293 is at least two, and the types of different cameras can be different. For example, camera 293 can be a color camera module, a monochrome camera module, a depth-of-field camera module, a telephoto (or telephoto) camera module, a wide-angle camera module, a macro camera module, etc. This application does not limit the number of cameras 293 or the specific type of each camera 293.

[0125] For example, the camera 293 may include a lens and a sensor. When taking a photo or recording video, the shutter is opened, and light is transmitted through the lens of the camera 293 to the sensor. The sensor can convert the light signal passing through the lens into an electrical signal, and then perform an A / D conversion on the electrical signal to output the corresponding digital signal. This digital signal undergoes subsequent RAW domain processing, ISP processing, and YUV domain processing to obtain the captured photo or video image.

[0126] In one possible design, the sensor's photosensitive element can be a charge-coupled device (CCD), and the sensor also includes an A / D converter. In another possible design, the sensor's photosensitive element can be a complementary metal-oxide-semiconductor (CMOS).

[0127] Display screen 294 is used to display images, videos, etc. Display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the mobile phone may include one or N displays 294, where N is a positive integer greater than 1. For example, display screen 294 can be used to display an application interface.

[0128] The mobile phone implements its display function through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0129] Understandable Figure 2 The structure shown does not constitute a specific limitation on the mobile phone. In some embodiments, the mobile phone may also include... Figure 2 This could mean having more or fewer components, combining some components, separating some components, or having different component arrangements. Or, Figure 2 Some of the components shown can be implemented in hardware, software, or a combination of both.

[0130] Additionally, when the terminal device is another mobile terminal with shooting capabilities, such as a tablet, wearable device, in-vehicle device, AR / VR device, laptop, UMPC, netbook, or PDA, or a professional shooting device such as a digital camera, SLR / mirrorless camera, action camera, gimbal camera, or drone, the specific structure of these other terminal devices can also be referenced. Figure 2 As shown. For example, other terminal devices may be... Figure 2 The components were added or removed based on the given structure, which will not be elaborated here.

[0131] It should also be understood that a terminal device (such as a mobile phone) may run one or more camera applications to enable the camera function. For example, the camera application may include the system-level application "Camera". Alternatively, the camera application may include other applications installed on the terminal device that can be used for taking pictures.

[0132] For example, Figure 3 This is a flowchart illustrating the multi-camera collaborative shooting method provided in an embodiment of this application. Figure 3 As shown, the method may include:

[0133] S301. When the shooting function is activated, the terminal device collects / acquires scene information corresponding to the current shooting scene.

[0134] For example, scene information may include: focal length information of the current shooting scene, lighting information of the current shooting scene, user shooting requirements, etc.

[0135] The focal length information of the current shooting scene can refer to the focal length (i.e., shooting distance) between the terminal device and the object being photographed in the current shooting scene. The focal length determines the distance of the image in focus; the larger the focal length, the smaller the depth of field and the smaller the shooting angle.

[0136] The lighting information of the current shooting scene can refer to the brightness of the light in the current shooting scene, whether there are dynamic changes in brightness (multi-light source scene), etc.

[0137] Optionally, the lighting information of the current shooting scene can be obtained from sensor information collected by sensors configured in the terminal device. For example, the lighting information can be collected by an ambient light sensor (light sensor).

[0138] User shooting needs refer to the shooting modes, shooting parameters, etc. that users select or set on their terminal devices.

[0139] For example, the terminal device may include a shooting application, which can be launched and run in response to a user's operation to initiate the shooting application. After launching and running the shooting application, the terminal device initiates the shooting function. The terminal device can use the shooting application to call the configured camera module to perform the shooting function. After launching and running the shooting application, the terminal device can display a shooting interface through the shooting application. The shooting interface may include function controls corresponding to various shooting modes that the user can select. For example, shooting modes may include: large aperture mode, portrait mode, bokeh mode, slow motion mode, high dynamic range mode, night scene mode, etc. The user can select one or more shooting modes by clicking the corresponding function controls on the shooting interface. The terminal device can determine the user's shooting needs in response to the user's selection of a shooting mode. For example, if the user clicks the function control corresponding to portrait mode, the terminal device can determine that the user's shooting needs include: shooting a portrait or shooting in portrait mode.

[0140] In this context, the shooting interface displayed by the shooting application after the terminal device starts running is also called the first interface. The user's actions of selecting a shooting mode and / or setting shooting parameters on the shooting interface can also be called the first operation.

[0141] In some embodiments, the terminal device can also obtain scene information corresponding to the current shooting scene by recognizing the preview image corresponding to the current shooting scene. For example, the terminal recognition can determine whether the user's shooting needs are for portraits, scenery, or the lighting information of the current shooting scene by recognizing the preview image corresponding to the current shooting scene.

[0142] Alternatively, the terminal device can combine the recognition results of the preview image corresponding to the current shooting scene, the detection parameters of hardware such as the light sensor, and the user's operation to comprehensively determine the scene information of the current shooting scene.

[0143] It should be noted that the scene information corresponding to the current shooting scene mentioned above is only an illustrative example. This application embodiment does not limit the specific type of scene information corresponding to the current shooting scene collected by the terminal device, nor the specific method by which the terminal device collects the scene information of the current shooting scene.

[0144] S302. Based on scene information and the capability information of the configured camera modules, the terminal device determines the main camera module and the slave camera modules from the configured camera modules, and determines the minimum exposure frame rate of the main camera module and each slave camera module respectively.

[0145] In S302, the terminal device determines the master camera module and slave camera module as the camera modules suitable for shooting the current shooting scene.

[0146] The number of camera modules can be one or more. The minimum exposure frame rate of the main camera module is greater than the minimum exposure frame rate of any of the slave camera modules. In this application, the main camera module can be referred to as the first camera module, and the slave camera modules can be referred to as the second camera module.

[0147] In some embodiments, the capability information of the camera modules configured in the terminal device can be preset in the terminal device. For example, each camera module can store its own capability information, and the terminal device can obtain the capability information of each camera module.

[0148] For example, the camera module's capability information may include: maximum exposure frame rate, reasonable focal length for shooting, output color mode, etc.

[0149] For example, assuming the terminal device is equipped with camera modules including: a color camera module, a monochrome camera module, a depth-of-field camera module, a telephoto camera module, and a wide-angle camera module, then the capability information of the aforementioned camera modules can be as follows:

[0150] 1) The maximum exposure frame rate of the color camera module is 30 frames per second (FPS) or 60 FPS. The reasonable focal length for shooting is between 0.8X and 5X, and the output color mode is color.

[0151] 2) The maximum exposure frame rate of the black and white camera module is 30 FPS, the reasonable focal length for shooting is between 0.8X and 5X, and the output color mode is no color or black and white.

[0152] 3) The maximum exposure frame rate of the depth-of-field camera module is 30FPS, the reasonable focal length for shooting is between 0.8X and 5X, the output color mode is color, and it can quickly and at a distance (1-10 meters) obtain more accurate depth-of-field information.

[0153] 4) The maximum exposure frame rate of the telephoto camera module is 30FPS, the reasonable focal length for shooting is 3X or higher, and the output color mode is color.

[0154] 5) The maximum exposure frame rate of the wide-angle camera module is 30 FPS, the reasonable focal length for shooting is between 0.1X and 1X, and the output color mode is color.

[0155] The following example illustrates how the terminal device in S302 determines the main camera module and secondary camera modules suitable for shooting the current scene based on scene information corresponding to the current shooting scene and the capability information of the camera modules configured on the terminal device.

[0156] For example, when the scene information corresponding to the current shooting scene collected by the terminal device in S301 is: large aperture portrait mode, focal length between 1X and 3X, in S302: the terminal device can combine the capability information of the color camera module, monochrome camera module, depth-of-field camera module, telephoto camera module, and wide-angle camera module, and determine that the telephoto camera module and wide-angle camera module are not needed based on the 1X-3X focal length scene; based on the large aperture mode, the color camera module is selected as the main camera module to provide the main image information, and the monochrome camera module is selected as the secondary camera module to provide richer light and shadow information; based on the portrait mode, the depth-of-field camera module is selected as the secondary camera module to provide better bokeh effect. That is, in this scene, the terminal device can select the color camera module as the main camera module, and the monochrome camera module and depth-of-field camera module as secondary camera modules.

[0157] When the scene information collected by the terminal device in S301 corresponds to the current shooting scene as follows: large aperture non-portrait mode, focal length between 1X and 3X, in S302: the terminal device can combine the capability information of the color camera module, monochrome camera module, depth-of-field camera module, telephoto camera module, and wide-angle camera module. Based on the 1X-3X focal length scene, it determines that the telephoto camera module and wide-angle camera module are not needed; based on the large aperture mode, it selects the color camera module as the main camera module to provide the main image information, and selects the monochrome camera module as the secondary camera module to provide richer light and shadow information; based on the non-portrait mode, it determines that the depth-of-field camera module is not needed. That is, in this scene, the terminal device can select the color camera module as the main camera module and the monochrome camera module as the secondary camera module.

[0158] The above example illustrates the scenario where a telephoto and wide-angle camera module are not needed when the focal length is between 1X and 3X. The following explains the scenarios where a telephoto or wide-angle camera module is required.

[0159] Optionally, when the focal length is between 0.5X and 1X, the terminal device can determine whether a wide-angle camera module is needed based on the specific changes in the focal length, and if a wide-angle camera module is needed, it can make a master-slave camera module decision between the wide-angle camera module and the color camera module.

[0160] For example, during the transition from 1X to 0.5X focal length, the terminal device can make decisions regarding the master and slave camera modules based on the specific changes in focal length as follows:

[0161] 1) When the focal length is less than 1X and greater than 0.9X, the terminal device can select the color camera module as the main camera module and determine that a wide-angle camera module is not needed.

[0162] 2) When the focal length is less than or equal to 0.9X and greater than 0.7X, the terminal device can select the color camera module as the main camera module and the wide-angle camera module as the slave camera module.

[0163] 3) When the focal length is less than or equal to 0.7X and greater than 0.5X, the terminal device can select the color camera module as the slave camera module and the wide-angle camera module as the master camera module.

[0164] 4) When the focal length is less than or equal to 0.5X, the terminal device can select the wide-angle camera module as the main camera module and determine that a color camera module is not needed.

[0165] Similarly, when the focal length is between 3X and 5X, the terminal device can determine whether a telephoto camera module is needed based on the specific changes in focal length, and when a telephoto camera module is needed, it can make a master-slave decision between the telephoto camera module and the color camera module.

[0166] For example, during the transition from 3X to 5X focal length, the terminal device can make decisions regarding the master and slave camera modules based on the specific changes in focal length as follows:

[0167] 1) When the focal length is greater than 3X and less than 3.5X, the terminal device can select the color camera module as the main camera module and the telephoto camera module as the slave camera module.

[0168] 2) When the focal length is greater than or equal to 3.5X and less than 5X, the terminal device can select the color camera module as the slave camera module and the telephoto camera module as the master camera module.

[0169] 3) When the focal length is greater than or equal to 5X, the terminal device can select the telephoto camera module as the main camera module and determine that a color camera module is not needed.

[0170] It should be noted that the process described above, in which the terminal device determines the suitable primary and secondary camera modules for shooting the current scene based on scene information corresponding to the current shooting scene and the capability information of the camera modules configured on the terminal device, is merely an illustrative example. It should be understood that when the scene information corresponding to the current shooting scene is of other types, the method by which the terminal device determines the suitable primary and secondary camera modules for shooting the current shooting scene may also vary, and is not limited here.

[0171] In some embodiments, the rules by which the terminal device determines the suitable main camera module and slave camera module for shooting the current shooting scene from among the camera modules configured on the terminal device, based on scene information corresponding to the current shooting scene and the capability information of the camera modules configured on the terminal device, can be manually configured. The terminal device can determine the suitable main camera module and slave camera module for shooting the current shooting scene from among the camera modules configured on the terminal device according to manually configured rules, based on scene information corresponding to the current shooting scene and the capability information of the camera modules configured on the terminal device.

[0172] This application does not limit the specific method by which the terminal device determines the main camera module and the slave camera module suitable for shooting the current shooting scene from the camera modules configured on the terminal device based on the scene information corresponding to the current shooting scene and the capability information of the camera modules configured on the terminal device.

[0173] After the terminal device determines the main camera module and slave camera modules suitable for shooting the current shooting scene, it can determine the minimum exposure frame rate for the main camera module and each slave camera module respectively.

[0174] The following example illustrates how the terminal device determines the minimum exposure frame rate for the main camera module and each slave camera module, using the scene information corresponding to the current shooting scene in the S301 as follows: large aperture portrait mode, focal length between 1X and 3X.

[0175] As mentioned above, the scene information corresponding to the current shooting scene is as follows: in large aperture portrait mode with a focal length between 1X and 3X, the terminal device can determine the color camera module as the primary camera module, and the monochrome camera module and depth-of-field camera module as secondary camera modules. Since the large aperture portrait mode generally does not involve the dynamic movement of the subject, there is no excessive requirement for the frame rate. Therefore, the terminal device can determine the minimum exposure frame rate of the color camera module to be 30FPS to ensure good image output, determine the minimum exposure frame rate of the monochrome camera module to be 10-12FPS to meet the needs of providing light and shadow variations, and determine the minimum exposure frame rate of the depth-of-field camera module to be 7.5FPS or 10FPS to meet the needs of providing distance information in the scene.

[0176] Optionally, the rules by which the terminal device determines the minimum exposure frame rate of the main camera module and each slave camera module can also be manually configured. For example, the manufacturer can configure the rules for determining the minimum exposure frame rate of the main camera module and each slave camera module for the terminal device before it leaves the factory.

[0177] This application does not impose any restrictions on the specific method by which the terminal device determines the minimum exposure frame rate of the main camera module and each slave camera module.

[0178] After determining the main camera module and the slave camera module, and respectively determining the minimum exposure frame rate of the main camera module and each slave camera module, the terminal device can adjust the exposure frame rate of the main camera module and each slave camera module during actual shooting in the manner described in S303-S304 below.

[0179] S303, The terminal device adjusts the exposure frame rate of the main camera module to the lowest exposure frame rate of the main camera module.

[0180] In some embodiments, in S303, the terminal device can adjust the exposure frame rate of the main camera module to a value greater than the minimum exposure frame rate of the main camera module. For example, if the minimum exposure frame rate of the main camera module is 30 FPS, the terminal device can adjust the exposure frame rate of the main camera module to 40 FPS, 60 FPS, etc. It is understandable that the maximum exposure frame rate of the main camera module can be adjusted to the maximum exposure frame rate supported by the main camera module.

[0181] In other words, in S303, the terminal device can adjust the exposure frame rate of the main camera module to a first exposure frame rate, which is greater than or equal to the minimum exposure frame rate of the main camera module and less than the maximum / high exposure frame rate of the main camera module. In this embodiment, the example of the terminal device adjusting the exposure frame rate of the main camera module to its minimum exposure frame rate will be used for illustration, but this is not a limitation.

[0182] S304. The terminal device adjusts the exposure frame rate of each slave camera module so that the exposure frame rate of each slave camera module satisfies the following conditions: the exposure frame rate of the master camera module is an integer multiple of the exposure frame rate of the slave camera module, and the exposure frame rate of the slave camera module is greater than or equal to the minimum exposure frame rate of the slave camera module.

[0183] It is understandable that when the exposure frame rate of the main camera module is an integer multiple of the exposure frame rate of the secondary camera module, the exposure frame rate of the secondary camera module is less than or equal to the exposure frame rate of the main camera module.

[0184] For example, assuming that the number of slave camera modules determined by the terminal device in S302 is i, where i is an integer greater than 0, the minimum exposure frame rate of the main camera module is S0, the minimum exposure frame rate of the first slave camera module is S1, the minimum exposure frame rate of the second slave camera module is S2, ..., and the minimum exposure frame rate of the i-th slave camera module is Si. Then, in S303, the terminal device can adjust the exposure frame rate S'0 of the main camera module to S'0 = S0. The specific method by which the terminal device adjusts the exposure frame rate of each slave camera module in S304 may include:

[0185] Using S'0 as the baseline, adjust the exposure frame rate S'1 of the first slave camera module so that S'1 satisfies: S'0 = X1 * S'1, where X1 is a positive integer, and S1 ≤ S'1 ≤ S'0; adjust the exposure frame rate S'2 of the second slave camera module so that S'2 satisfies: S'0 = X2 * S'2, where X2 is a positive integer, and S2 ≤ S'2 ≤ S'0; ...; and so on, adjust the exposure frame rate S'i of the i-th slave camera module so that S'i satisfies: S'0 = Xi * S'i, where Xi is a positive integer, and Si ≤ S'i ≤ S'0.

[0186] “≤” means less than or equal to.

[0187] Optionally, during the process of adjusting the exposure frame rate of each slave camera module in the manner described above, if there are multiple combinations of Xi and S'i corresponding to the i-th slave camera module, the terminal device can adjust the exposure frame rate S'i of the i-th slave camera module to the smallest of the multiple combinations.

[0188] For example, assuming the minimum exposure frame rate S0 of the main camera module is 30 FPS and the minimum exposure frame rate Si of the i-th slave camera module is 10 FPS, the terminal device adjusts the exposure frame rate S'i of the i-th slave camera module so that S'i satisfies: S'0=Xi*S'i, where Xi is an integer greater than 0. When Si≤S'i≤S'0, the combination of Xi and S'i can include the three types shown in (1) to (3) below:

[0189] (1) Xi is 1, S'i is 30 FPS;

[0190] (2) Xi is 2, S'i is 15 FPS;

[0191] (3) Xi is 3, S'i is 10 FPS;

[0192] At this point, the terminal device can adjust the exposure frame rate S'i of the i-th image from the camera module to 10 FPS.

[0193] That is, for the i-th slave camera module: when the range from the minimum exposure frame rate of the slave camera module to the minimum exposure frame rate of the master camera module includes multiple S'i, and the minimum exposure frame rate of the master camera module is an integer multiple of S'i, the terminal device can configure the minimum value among the multiple S'i as the exposure frame rate of the i-th slave camera module. The aforementioned multiple S'i can be referred to as the second exposure frame rate.

[0194] Of course, during the process of adjusting the exposure frame rate of each slave camera module in the manner described above, for the i-th slave camera module, when there are multiple combinations of Xi and S'i corresponding to the i-th slave camera module, the terminal device can also adjust the exposure frame rate S'i of the i-th slave camera module to any one of the multiple combinations, without any restrictions.

[0195] Taking a terminal device that identifies the main camera module as a color camera module, and the secondary camera modules as a monochrome camera module and a depth-of-field camera module, with a minimum exposure frame rate of 30 FPS for the color camera module, 12 FPS for the monochrome camera module, and 10 FPS for the depth-of-field camera module as an example: In this embodiment, the terminal device can adjust the exposure frame rate of the color camera module to 30 FPS, the exposure frame rate of the monochrome camera module to 15 FPS, and the exposure frame rate of the depth-of-field camera module to 10 FPS as described above. The exposure frame rate ratio of the color camera module, the monochrome camera module, and the depth-of-field camera module is 6:3:2. It can be understood that, at this time, the frame intervals corresponding to the color camera module, the monochrome camera module, and the depth-of-field camera module during shooting are approximately 33 milliseconds (ms), 66 ms, and 100 ms, respectively.

[0196] It should be noted that this application does not restrict the execution order of S303 and S304. For example, S303 can be executed before or after S304, or S303 and S304 can be executed synchronously. For example, when S303 and S304 are executed synchronously, the terminal device can predetermine the target values ​​of the exposure frame rates that need to be configured for the main camera module and the slave camera module (such as the target value of the exposure frame rate of the main camera module being the minimum exposure frame rate of the main camera module), and then configure the exposure frame rates of the main camera module and the slave camera module. The target values ​​of the exposure frame rates configured for the main camera module and the slave camera module satisfy the aforementioned integer multiple relationship.

[0197] After the terminal device adjusts the exposure frame rate of the main camera module and the exposure frame rate of each slave camera module in accordance with the methods described in S303 and S304, the terminal device can execute S305-S307 to complete multi-camera collaborative shooting.

[0198] S305. For each slave camera module: The terminal device configures the interval number of sensing pulse signals for the slave camera module according to the exposure frame rate of the master camera module and the exposure frame rate of the slave camera module.

[0199] The interval number configured for the sensing pulse signal from the camera module refers to the interval at which the camera module senses the pulse signal emitted by the main camera module. For example, if the terminal device configures the interval number for the sensing pulse signal from a certain camera module to be 2, then the camera module will sense the pulse signal emitted by the main camera module every 2 intervals. Sensing a specific pulse signal emitted by the main camera module from the camera module means that the camera module can respond to the pulse signal emitted by the main camera module in this instance and begin capturing the current frame.

[0200] Optionally, the step of configuring the interval number of sensing pulse signals for the slave camera module based on the exposure frame rate of the master camera module and the exposure frame rate of the slave camera module may include: the terminal device configuring the value of the quotient of the exposure frame rate of the master camera module divided by the exposure frame rate of the slave camera module minus 1 as the interval number of sensing pulse signals for the slave camera module.

[0201] For example, if the terminal device determines that the main camera module is a color camera module, and the secondary camera modules are a monochrome camera module and a depth-of-field camera module, and the terminal device adjusts the exposure frame rate of the color camera module to 30 FPS, the exposure frame rate of the monochrome camera module to 15 FPS, and the exposure frame rate of the depth-of-field camera module to 10 FPS, the terminal device can configure the interval number of pulse signals sensed by the monochrome camera module to (30 / 15-1=1) times; and configure the interval number of pulse signals sensed by the depth-of-field camera module to (30 / 10-1=2) times.

[0202] In some embodiments, for different combinations of master and slave camera modules, the terminal device may pre-set a correspondence between the exposure frame rates of the master and slave camera modules and the interval number of times the slave camera module senses pulse signals. This correspondence can be obtained in the manner described above for determining the interval number of times the slave camera module senses pulse signals, and can be pre-configured in the terminal device. The terminal device can query this correspondence based on the exposure frame rates of the master and slave camera modules to determine the interval number of times the slave camera module senses pulse signals.

[0203] S306. The terminal device takes pictures through the main camera module and each slave camera module. During the shooting process, the main camera module emits a pulse signal at the moment corresponding to the frame header of each frame. Each slave camera module senses the pulse signal emitted by the main camera module according to the configured interval number of pulse signal sensing, and starts shooting the current frame after sensing the pulse signal.

[0204] In other words, the camera module can respond to the pulse signal emitted by the main camera module and start capturing the current frame.

[0205] Optionally, prior to S306, the terminal device could configure the frequency of the pulse signal emitted by the main camera module so that the main camera module emits a pulse signal at the moment corresponding to the frame header of each frame.

[0206] In some embodiments, the pulse signal described in this application may be an XVS pulse signal, a VSYNC pulse signal, a FSIN pulse signal, etc. The XVS pulse signal is used as an example below, but this application does not limit the specific type of pulse signal. The aforementioned XVS pulse signal, VSYNC pulse signal, FSIN pulse signal, etc., can all be referred to as the first signal.

[0207] For example, taking the XVS pulse signal as an example, Figure 4 This is a schematic diagram illustrating the hard synchronization control principle of the three camera modules provided in an embodiment of this application. Figure 4As shown, assuming the terminal device determines the main camera module to be a color camera module, and the secondary camera modules to be a monochrome camera module and a depth-of-field camera module; the terminal device adjusts the exposure frame rate of the color camera module to 30 FPS, the exposure frame rate of the monochrome camera module to 15 FPS, and the exposure frame rate of the depth-of-field camera module to 10 FPS; the terminal device configures the interval for the monochrome camera module to sense the XVS pulse signal to be 1 time, and the interval for the depth-of-field camera module to sense the XVS pulse signal to be 2 times; then in S306: the terminal device, when passing through the color camera module, the monochrome camera module, and the depth-of-field camera module... During the shooting process of the RA module and the depth-of-field camera module, the terminal device can control the color camera module to emit XVS pulse signals at 30FPS at the time corresponding to the frame header (SOF) of each frame; control the black and white camera module to sense the XVS pulse signals emitted by the color camera module once every interval, and start shooting the current frame after sensing the XVS pulse signal emitted by the main camera module each time; control the depth-of-field camera module to sense the XVS pulse signals emitted by the color camera module once every two intervals, and start shooting the current frame after sensing the XVS pulse signal emitted by the main camera module each time.

[0208] In other words, after sensing one XVS pulse signal from the color camera module, the monochrome camera module does not perceive the first subsequent XVS pulse signal received from the color camera module, but it does perceive the second subsequent XVS pulse signal received from the color camera module. Similarly, after sensing one XVS pulse signal from the color camera module, the depth-of-field camera module does not perceive the first and second subsequent XVS pulse signals received from the color camera module, but it does perceive the third subsequent XVS pulse signal received from the color camera module.

[0209] Optionally, a counter can be installed inside the slave camera module. The slave camera module can use the counter to count the number of XVS pulse signals received from the master camera module. Based on the counter's count, the slave camera module can determine which XVS pulse signal it has received, whether the configured interval for sensing XVS pulse signals has been reached, and thus further determine whether to sense the XVS pulse signal received from the master camera module.

[0210] S307 The terminal device performs fusion processing on the images captured by the main camera module and each camera module to obtain the final captured image.

[0211] For example, the terminal device performs fusion processing on the main camera module and each image captured by the camera module, and the final captured image can be the first image.

[0212] Optionally, when the terminal device performs fusion processing on the images captured by the main camera module and each secondary camera module, it can select the most recently captured frame from the main camera module and each secondary camera module for fusion processing. For example, when the terminal device performs fusion processing on the images captured by the color camera module, the monochrome camera module, and the depth-of-field camera module after the color camera module captures an image, if the monochrome camera module and the depth-of-field camera module also sensed the XVS pulse signal and captured an image when the color camera module captured this image, then the three images currently captured by the color camera module, the monochrome camera module, and the depth-of-field camera module of the terminal device are fused. If the color camera module captures this frame while the monochrome camera module senses the XVS pulse signal and captures a frame, but the depth camera module does not sense the XVS pulse signal, the terminal device can fuse the current frame image captured by the color camera module and the monochrome camera module, as well as the image captured by the depth camera module last time it sensed the XVS pulse signal.

[0213] Optionally, in S306 above, when the terminal device takes pictures using the main camera module and each slave camera module, during the first frame pre-exposure stage when the main camera module and each slave camera module start working, the first frame length of the main camera module and each slave camera module can be controlled to be the same (i.e., the first frame length of the exposed image is the same) to ensure that the first frame output time of the main camera module and each slave camera module is consistent, thereby ensuring that the main camera module and each slave camera module maintain the same frame output time when the first frame is output. Afterwards, starting from the second frame, the terminal device controls the main camera module and each slave camera module to take pictures in the manner described in S306 above.

[0214] For example, taking the example of a color camera module as the main camera module in the aforementioned embodiments, and using a monochrome camera module and a depth-of-field camera module as examples, assuming that the frame intervals corresponding to the color camera module, monochrome camera module, and depth-of-field camera module during shooting are approximately 33 milliseconds (ms), 66 ms, and 100 ms respectively, the terminal device can control the frame length of the first frame corresponding to the color camera module, monochrome camera module, and depth-of-field camera module to be the same during the first frame pre-exposure stage when the color camera module, monochrome camera module, and depth-of-field camera module start working. For example, the terminal device can compress the duration of the vblank of the first frame corresponding to the monochrome camera module and depth-of-field camera module, controlling the frame length of the first frame corresponding to the color camera module, monochrome camera module, and depth-of-field camera module to be 33 ms. Alternatively, the terminal device can lengthen the duration of the vblank for the first frame corresponding to the color camera module and compress the duration of the vblank for the first frame corresponding to the depth-of-field camera module, controlling the frame length of the first frame for each of the color, monochrome, and depth-of-field camera modules to be 66ms. Or, the terminal device can lengthen the duration of the vblank for the first frame corresponding to each of the color and monochrome camera modules, controlling the frame length of the first frame for each of the color, monochrome, and depth-of-field camera modules to be 100ms, etc.

[0215] This application does not restrict the specific implementation method in which the first frame length of the terminal device controls the main camera module and each slave camera module to be the same.

[0216] Optionally, the terminal device can send the first frame pre-exposure frame length to the main camera module and each slave camera module, and the main camera module and each slave camera module can expose the first frame according to the first frame pre-exposure frame length sent by the terminal device.

[0217] In the multi-camera collaborative shooting method provided in this application embodiment, the terminal device dynamically adjusts the exposure frame rate of the main camera module and each slave camera module, and configures the interval number of pulse signals sensed by each slave camera module. This enables the slave camera modules to achieve collaborative shooting with the main camera module in hard-synchronized scenarios at a lower exposure frame rate than the main camera module (reducing unnecessary exposure frequencies of the slave camera modules). Since the exposure frame rate of the slave camera modules can be lower than that of the main camera module, the power consumption of the slave camera modules can be lower. That is, this method can effectively reduce the power consumption of the terminal device when performing multi-camera collaborative shooting.

[0218] For example, in the example of collaborative shooting by the color camera module (main camera module), monochrome camera module (slave camera module), and depth-of-field camera module (slave camera module) described in the foregoing embodiments, the multi-camera collaborative shooting method provided in this application embodiment allows the color camera module to shoot at an exposure frame rate of 30 FPS, the monochrome camera module at an exposure frame rate of 15 FPS, and the depth-of-field camera module at an exposure frame rate of 10 FPS. Simultaneously, the frame headers of each frame from the color camera module, monochrome camera module, and depth-of-field camera module maintain consistent timestamps (i.e., consistent image output time). Since the exposure frame rates of the monochrome camera module and depth-of-field camera module are lower than those of the color camera module, they do not need to maintain consistency with the color camera module's exposure frame rate. Therefore, the power consumption of the monochrome camera module and depth-of-field camera module can be lower.

[0219] Optionally, in this embodiment, the terminal device may include a multi-camera synchronization control module. This module can be used to implement the functions described in the foregoing embodiments: determining the master camera module and slave camera modules, and respectively determining the minimum exposure frame rate of the master camera module and each slave camera module; adjusting the exposure frame rate of the master camera module and each slave camera module; and configuring the interval number of sensing pulse signals for the slave camera modules based on the exposure frame rate of the master camera module and the exposure frame rate of the slave camera modules. This multi-camera synchronization control module can be executable code in a processor.

[0220] For example, Figure 5 This is a schematic diagram illustrating the principle of the multi-camera synchronization control module provided in an embodiment of this application. Figure 5As shown, the multi-camera synchronization control module can include two parts: a decision model and a control module. The decision model determines the master / slave attributes of the camera modules, the exposure frame rates of the master and slave camera modules, and the interval at which the slave camera modules sense XVS pulse signals. The control module, based on the decision model's results, configures the master / slave attributes of the camera modules, the exposure frame rates of the master and slave camera modules, and controls the master camera module to emit XVS pulse signals and the slave camera modules to sense XVS pulse signals.

[0221] For example, taking a mobile phone as the terminal device, the mobile phone may include the aforementioned multi-camera synchronization control module. After the mobile phone starts running the camera application, it can collect scene information of the current shooting scene (see the foregoing embodiments for details). The decision model can determine the main camera module and slave camera module suitable for shooting the current shooting scene from the camera modules configured on the mobile phone, based on the scene information of the current shooting scene and the capability information of the camera modules configured on the mobile phone. After determining the main camera module and slave camera module, the decision model can determine the minimum exposure frame rate corresponding to the main camera module and slave camera module respectively, and determine the exposure frame rate corresponding to the main camera module and slave camera module respectively (which can be called the target exposure frame rate, see the foregoing embodiments for the specific decision method) according to the minimum exposure frame rate corresponding to the main camera module and slave camera module respectively. In addition, the decision model can also determine the interval number of times the slave camera module senses the XVS pulse signal based on the exposure frame rate corresponding to the main camera module and slave camera module respectively. The control module can determine the master and slave camera modules based on the decision-making model's results, and adjust their exposure frame rates to the target exposure frame rates determined by the model. Then, during collaborative shooting using the master and slave camera modules, the control module can control the master camera module to emit an XVS pulse signal at the beginning of each frame, according to its exposure frame rate. It can also control the slave camera module to sense the XVS pulse signal at the specified intervals determined by the decision-making model. For example, the control module can configure the slave camera module to sense the XVS pulse signal at the specified intervals.

[0222] Optionally, in some other embodiments of this application, in S304 above, the terminal device may also adjust the exposure frame rate of each slave camera module so that the exposure frame rate of the main camera module and all slave camera modules satisfies the following: the exposure frame rate of the main camera module and the exposure frame rate of any slave camera module are proportional to each other, the exposure frame rates of any two slave camera modules are also proportional to each other, and the exposure frame rate of each slave camera module is greater than or equal to the lowest exposure frame rate of the slave camera module and less than or equal to the exposure frame rate of the main camera module.

[0223] For example, assuming that the number of slave camera modules determined by the terminal device in S302 is i, where i is an integer greater than 0, the minimum exposure frame rate of the main camera module is S0, the minimum exposure frame rate of the first slave camera module is S1, the minimum exposure frame rate of the second slave camera module is S2, ..., and the minimum exposure frame rate of the i-th slave camera module is Si. Then, in S303, the terminal device can adjust the exposure frame rate S'0 of the main camera module to S'0 = S0. In this embodiment, the specific method by which the terminal device adjusts the exposure frame rate of each slave camera module in S304 may include:

[0224] Using S'0 as a reference, adjust the exposure frame rate S'1 of the first slave camera module to the exposure frame rate S'i of the i-th slave camera module, so that S'1 to S'i satisfy:

[0225] S'0=X1*S'1=X1*X2*S'2=…=X1*X2*…*Xi-1*Xi*S'i, where X1 to Xi are integers greater than 0, and Si≤S'i≤S'0.

[0226] After the terminal device adjusts the exposure frame rate of each camera module in the manner described above, it can complete multi-camera collaborative shooting in the same manner as described in S305-S307 of the aforementioned embodiments.

[0227] In this embodiment, the terminal device dynamically adjusts the exposure frame rate of the main camera module and each slave camera module using the aforementioned method, and configures the interval number of pulse signals sensed by each slave camera module. This allows the slave camera modules to achieve collaborative shooting in hard-synchronized scenarios with the main camera module at a lower exposure frame rate than the main camera module. Since the exposure frame rate of the slave camera modules can be lower than that of the main camera module, the power consumption of the slave camera modules can be lower. Furthermore, compared to the previous embodiment, in this embodiment, the main camera module and any two slave camera modules are in a proportional relationship, allowing the main camera module and all slave camera modules to output multi-camera exposure images with the same time stamp at certain fixed frames. This is more conducive to subsequent image fusion, resulting in a better quality final image output by the terminal device after image fusion processing.

[0228] Optionally, in this embodiment, during the process of adjusting the exposure frame rate of each slave camera module in the manner described above, for the i-th slave camera module, when there are multiple combinations of Xi and S'i corresponding to the i-th slave camera module, the terminal device can also adjust the exposure frame rate S'i of the i-th slave camera module to the smallest of the multiple combinations. The specific principle is the same as described in the previous embodiments, and will not be repeated here.

[0229] The following example illustrates how the terminal device determines the main camera module to be a color camera module, the slave camera modules to be a monochrome camera module and a depth-of-field camera module, and the pulse signal to be an XVS pulse signal. The example demonstrates how the terminal device adjusts the exposure frame rate of each slave camera module to ensure a proportional relationship between any two camera modules among the main camera module and all slave camera modules.

[0230] For example, Figure 6 Another schematic diagram illustrating the hard synchronization control principle of the three camera modules provided in this application embodiment. (See diagram below.) Figure 6As shown, assuming the minimum exposure frame rate of the color camera module is 30 FPS, the minimum exposure frame rate of the monochrome camera module is 12 FPS, and the minimum exposure frame rate of the depth-of-field camera module is 10 FPS, the terminal device can adjust the exposure frame rate of the color camera module to 30 FPS, and adjust the exposure frame rates of the monochrome and depth-of-field camera modules to 15 FPS. At this point, the exposure frame rates of the color, monochrome, and depth-of-field camera modules satisfy the following condition: the exposure frame rates of any two camera modules are proportionally related to each other. Then, the terminal device can configure the interval for sensing XVS pulse signals by the monochrome and depth-of-field camera modules to be 1 time each. During the shooting process using the color camera module, monochrome camera module, and depth-of-field camera module, the terminal device can control the color camera module to emit an XVS pulse signal at 30 FPS at the time corresponding to the frame header (SOF) of each frame; control the monochrome camera module to sense the XVS pulse signal emitted by the color camera module once every interval, and start shooting the current frame after sensing the XVS pulse signal emitted by the main camera module; control the depth-of-field camera module to sense the XVS pulse signal emitted by the color camera module once every interval, and start shooting the current frame after sensing the XVS pulse signal emitted by the main camera module.

[0231] In other words, after sensing one XVS pulse signal from the color camera module, the monochrome camera module does not perceive the first subsequent XVS pulse signal received from the color camera module, but it does perceive the second subsequent XVS pulse signal received from the color camera module. Similarly, the depth-of-field camera module senses one XVS pulse signal from the color camera module, does not perceive the first subsequent XVS pulse signal received from the color camera module, but it does perceive the second subsequent XVS pulse signal received from the color camera module.

[0232] As can be seen, for the examples where the minimum exposure frame rate is 30 FPS for the color camera module, 12 FPS for the monochrome camera module, and 10 FPS for the depth-of-field camera module:

[0233] In the embodiment where the exposure frame rate of each slave camera module satisfies the requirement that the minimum exposure frame rate of the master camera module is an integer multiple of the exposure frame rate of the slave camera module, after adjustment by the terminal device, the exposure frame rate of the color camera module is 30 FPS, the exposure frame rate of the monochrome camera module is 15 FPS, the exposure frame rate of the depth-of-field camera module is 10 FPS, the output ratio of the three camera modules is 6:3:2, and the output timestamps of the three camera modules remain consistent every 6 frames.

[0234] In the embodiment where any two camera modules are proportionally related to each other, after the terminal device is adjusted, the exposure frame rate of the color camera module is 30 FPS, the exposure frame rate of the black and white camera module and the exposure frame rate of the depth-of-field camera module are both 15 FPS, the output ratio of the three camera modules is 2:1:1, and the output timestamps of the three camera modules are consistent every 2 frames.

[0235] Based on the examples above, where the minimum exposure frame rate for the color camera module is 30 FPS, the minimum exposure frame rate for the monochrome camera module is 12 FPS, and the minimum exposure frame rate for the depth-of-field camera module is 10 FPS, it can be seen that in multi-camera collaborative shooting scenarios, in embodiments where the exposure frame rates of any two camera modules are proportionally related, the number of frames between the output timestamps of the master and slave camera modules that maintain consistency is relatively small. Therefore, in embodiments where the exposure frame rates of any two camera modules are proportionally related, the timing consistency of the master and slave camera modules is better, and the final image output by the terminal device after image fusion processing can be of better quality.

[0236] It should be noted that although the foregoing embodiments of this application use three camera modules as an example, it should be understood that the multi-camera collaborative shooting method provided in this application can be applied to scenarios where other numbers of camera modules, such as two or four, are used for collaborative shooting. This application does not limit the number of camera modules. When the number of camera modules is two (i.e., one main camera module and one slave camera module), the embodiment in which the terminal device adjusts the exposure frame rate of each slave camera module so that the exposure frame rate of each slave camera module meets the requirement that the minimum exposure frame rate of the main camera module is an integer multiple of the exposure frame rate of the slave camera module, achieves the same effect as the embodiment in which the terminal device adjusts the exposure frame rate of each slave camera module so that any two camera modules among the main camera module and all slave camera modules have a proportional relationship.

[0237] For example, Figure 7 This is a schematic diagram illustrating the hard synchronization control principle of two camera modules provided in an embodiment of this application. Figure 7 As shown, assuming a shooting scenario (such as a low-light scene at dusk), the terminal device determines that the main camera module is a color camera module and the slave camera modules are monochrome camera modules. The minimum exposure frame rate of the color camera module is 30 FPS, and the minimum exposure frame rate of the monochrome camera module is 12 FPS. The color camera module has higher shooting performance; the monochrome camera module can capture grayscale information, providing richer brightness variations. Therefore, the terminal device can adjust the exposure frame rate of the color camera module to 30 FPS and the exposure frame rate of the monochrome camera module to 15 FPS, so that the exposure frame rate of the monochrome camera module is twice that of the color camera module, resulting in an exposure frame rate ratio of 2:1. At this point, since there are only color and monochrome camera modules, the main camera module and any two slave camera modules also have a proportional relationship. Then, the terminal device can configure the interval for the monochrome camera module to sense the XVS pulse signal to be once. During shooting using both the color and monochrome camera modules, the terminal device can control the color camera module to emit an XVS pulse signal at 30 FPS at the time corresponding to the frame header (SOF) of each frame; and control the monochrome camera module to sense the XVS pulse signal emitted by the color camera module once every interval, and to start shooting the current frame after each time it senses the XVS pulse signal emitted by the main camera module. That is, after sensing the XVS pulse signal emitted by the color camera module once, the monochrome camera module will not sense the first XVS pulse signal received from the color camera module, but will sense the second XVS pulse signal received from the color camera module.

[0238] Optionally, in some embodiments of this application, after the terminal device starts the shooting function, it can further divide the current shooting scene into a scene with high or low time consistency requirements based on the scene information corresponding to the current shooting scene. When the current shooting scene has low time consistency requirements, the terminal device can adjust the exposure frame rate of each slave camera module so that the exposure frame rate of each slave camera module is an integer multiple of the minimum exposure frame rate of the master camera module. When the current shooting scene has high time consistency requirements, the terminal device can adjust the exposure frame rate of each slave camera module so that the exposure frame rate of the master camera module and the exposure frame rate of any slave camera module are proportionally related, and the exposure frame rates of any two slave camera modules are also proportionally related.

[0239] For example, in High Dynamic Range (HDR) mode, the subject is typically moving rapidly. Because the subject is in motion, besides the main camera module requiring a high exposure frame rate, multiple cameras will need to ensure high temporal consistency in the final image capture to prevent difficulties in image fusion due to significant object displacement between frames. Therefore, HDR shooting scenarios can be categorized as scenarios with high temporal consistency requirements. Similarly, when the current shooting scene contains moving objects, it can also be categorized as a scenario with high temporal consistency requirements. Conversely, scenarios involving stationary objects, or those corresponding to normal mode (non-HDR), can be categorized as scenarios with lower temporal consistency requirements.

[0240] For example, when the user selects high dynamic range mode, or when the terminal device identifies moving objects in the current shooting scene based on the preview image, the terminal device (or the decision model within the terminal device) can classify the current shooting scene into a scene with high requirements for temporal consistency based on the scene information. When subsequently adjusting the exposure frame rate of the master and slave camera modules, the terminal device can adjust the exposure frame rate of each slave camera module, ensuring a proportional relationship between the master camera module and any two slave camera modules.

[0241] Optionally, when the current shooting scene has high requirements for time consistency, the terminal device can appropriately increase the exposure frame rate of the slave camera module when adjusting the exposure frame rate of the slave camera module; or, when determining the minimum exposure frame rate of the main camera module and each slave camera module, the terminal device can appropriately increase the minimum exposure frame rate of the main camera module and each slave camera module, so as to maintain better consistency in the image output time of the main camera module and each slave camera module.

[0242] In some embodiments, the terminal device may have a preset correspondence between scene information and time consistency. The terminal device can query this correspondence to determine whether the current shooting scene has high or low requirements for time consistency based on the scene information corresponding to the current shooting scene. This correspondence may be manually configured based on photographic experience, photographic rules, etc., and is not limited here.

[0243] For example, Figure 8 This is a schematic diagram illustrating the software architecture implementation principle of the terminal device provided in the embodiments of this application. Figure 8 As shown in the embodiments of this application, the software architecture of the terminal device may include: application layer, application framework layer, hardware abstract layer (HAL), kernel layer, and hardware driver layer.

[0244] The application layer can contain multiple applications, such as system applications like email, SMS, calendar, maps, browser, and contact management, as well as photography applications (like camera apps).

[0245] In this embodiment, the terminal device, based on scene information corresponding to the current shooting scene and the capability information of the camera modules configured on the terminal device, determines (decides) the functions of the main camera module and slave camera modules suitable for shooting the current shooting scene, determines the minimum exposure frame rate of the main camera module and each slave camera module, and determines the actual exposure frame rate of the main camera module and each slave camera module. This can be implemented in the application framework layer. The terminal device, based on the decision results of the application framework layer, configures the functions of the main and slave camera modules, as well as the exposure frame rate of the main and slave camera modules and the interval number of pulse signals sensed by the slave camera modules. This can be implemented in the hardware abstraction layer (HAL). The kernel layer can read and write registers according to the configuration results of the hardware abstraction layer (HAL) to control the exposure frame rate of the main and slave camera modules, the frequency of pulse signals emitted by the main camera module, and the interval number of pulse signals sensed by the slave camera modules. The hardware driver layer can call the corresponding camera module to implement the corresponding functions according to the register read and write logic. For example, the camera module can include camera module 1, camera module 2, camera module 3, etc.

[0246] It should be understood that the above embodiments are merely illustrative examples of the multi-camera collaborative shooting method provided in this application. In other possible implementations, certain execution steps may be omitted or added to the above embodiments, or the order of some steps in the above embodiments may be adjusted, and this application does not impose any limitations on these aspects.

[0247] Corresponding to the multi-camera collaborative shooting method described in the foregoing embodiments, this application also provides a shooting device, which can be applied to a terminal device to implement the multi-camera collaborative shooting method described in the foregoing embodiments. The function of this device can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.

[0248] For example, Figure 9 This is a schematic diagram of the imaging device provided in an embodiment of this application. Figure 9 As shown, the device may include: a configuration module 901 and a shooting module 902.

[0249] The configuration module 901 is used to determine the first camera module and the second camera module from the camera modules configured in the terminal device, and configure the exposure frame rate of the first camera module and the exposure frame rate of the second camera module; the number of the second camera modules is one or more; the exposure frame rate of the first camera module is an integer multiple of the exposure frame rate of the second camera module.

[0250] The shooting module 902 is used to take pictures through the first camera module and each of the second camera modules.

[0251] Among them, the first camera module and the second camera module are camera modules that enable multi-camera collaborative shooting.

[0252] In one possible design, the configuration module 901 is further configured to, for each second camera module, configure the interval number for sensing the first signal based on the exposure frame rate of the first camera module and the exposure frame rate of the second camera module. During the shooting process, the first camera module emits the first signal at the moment corresponding to the frame header of each frame; each second camera module senses the first signal emitted by the first camera module according to the interval number for sensing the first signal; in response to sensing the first signal emitted by the first camera module, each second camera module begins shooting the current frame.

[0253] Figure 10 Another schematic diagram of the imaging device provided in an embodiment of this application. Optionally, as shown... Figure 10 As shown,

[0254] In some embodiments, the shooting device further includes an acquisition module 903, used to acquire scene information of the current shooting scene and capability information of the camera module configured in the terminal device.

[0255] The configuration module 901 is specifically used to determine the first camera module and the second camera module from the camera modules configured in the terminal device based on the scene information and the capability information.

[0256] For example, scene information includes one or more of the following: focal length information, lighting information, and the user's shooting requirements.

[0257] In one possible design, the acquisition module 903 is specifically used to acquire scene information of the current shooting scene based on one or more of the following: the preview image corresponding to the current shooting scene, sensor information (such as light information collected by a light sensor), and the first operation.

[0258] The first operation is to select a shooting mode and / or set shooting parameters on the first interface displayed on the terminal device.

[0259] Optionally, the camera module's capability information may include one or more of the following: maximum exposure frame rate, reasonable focal length for shooting, and output color mode.

[0260] In one possible design, the configuration module 901 is specifically used to determine the minimum exposure frame rate of the first camera module and the second camera module based on scene information of the current shooting scene and the capability information of the camera modules configured in the terminal device. The exposure frame rate of the first camera module is configured as a first exposure frame rate, which is greater than or equal to the minimum exposure frame rate of the first camera module. The exposure frame rate of each second camera module is configured such that the first exposure frame rate is an integer multiple of the exposure frame rate of the second camera module, and the exposure frame rate of the second camera module is greater than or equal to the minimum exposure frame rate of the second camera module (less than or equal to the first exposure frame rate).

[0261] In another possible design, the configuration module 901 is specifically used to configure the exposure frame rate of each second camera module, so that the first exposure frame rate and the exposure frame rate of any second camera module are proportional to each other, and the exposure frame rates of any two second camera modules are also proportional to each other.

[0262] In one possible design, the current shooting scene can be divided into (or include) scenes with high time consistency requirements, or scenes with low time consistency requirements. Configuration module 901 is specifically used to configure the exposure frame rate of each second camera module when the current shooting scene has high time consistency requirements, such that the first exposure frame rate and the exposure frame rate of any second camera module are proportionally related, and the exposure frame rates of any two second camera modules are also proportionally related.

[0263] In some embodiments, the configuration module 901 is specifically configured to, for each second camera module: when the range from the lowest exposure frame rate of the second camera module to the first exposure frame rate includes multiple second exposure frame rates that satisfy the condition that the first exposure frame rate is an integer multiple of the second exposure frame rate, configure the minimum value among the multiple second exposure frame rates as the exposure frame rate of the second camera module.

[0264] Optionally, the shooting module 902 is also used to control the first camera module and each second camera module to keep the exposure time of the first frame of the image consistent during shooting.

[0265] Figure 11 This is another schematic diagram of the imaging device provided in an embodiment of this application. Optionally, as shown... Figure 11 As shown, the shooting device also includes a fusion module 904, which is used to fuse the images captured by the first camera module and the second camera module to obtain a first image.

[0266] The first image is the final image captured by the terminal device through multi-camera collaborative shooting.

[0267] This shooting device can be used to realize all the functions of the multi-camera collaborative shooting method described in the foregoing method embodiments, and will not be described in detail here.

[0268] It should be understood that the device may also include other modules or units for implementing the multi-camera collaborative shooting method described in the foregoing embodiments, such as a display unit, which can be used to display the first interface, but these are not shown one by one here.

[0269] It should be understood that the division of units (or modules) in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, all units in the device can be implemented entirely in software through processing element calls; all units can be implemented entirely in hardware; or some units can be implemented in software through processing element calls, while others can be implemented in hardware.

[0270] For example, each unit can be a separate processing element, or it can be integrated into a chip within the device. Alternatively, it can be stored as a program in memory, invoked and executed by a processing element within the device. Furthermore, these units can be integrated in whole or in part, or implemented independently. The processing element described here can also be called a processor, which can be an integrated circuit with signal processing capabilities. In implementation, each step of the above method or each of the above units can be implemented through integrated logic circuits in the processor element or through software invoked by the processing element.

[0271] In one example, the unit in the above device may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0272] For example, when the units in the device can be implemented through a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. Alternatively, these units can be integrated together to form a system-on-a-chip (SOC).

[0273] In one implementation, the units that implement the corresponding steps in the above methods can be implemented in the form of a processing element scheduler. For example, the device may include a processing element and a storage element, wherein the processing element calls a program stored in the storage element to execute the methods described in the above embodiments. The storage element may be a storage element located on the same chip as the processing element, i.e., an on-chip storage element.

[0274] In another implementation, the program used to perform the above methods can be located on a storage element on a different chip than the processing element, i.e., an off-chip storage element. In this case, the processing element calls or loads the program from the off-chip storage element onto the on-chip storage element to call and execute the methods described in the above method embodiments.

[0275] For example, embodiments of this application may also provide an apparatus, such as an electronic device. The electronic device may include a processor and a memory for storing processor-executable instructions; the processor is configured to execute the instructions such that the electronic device performs the method described in the foregoing embodiments. The memory may be located within or outside the electronic device. Furthermore, the processor may include one or more processors.

[0276] For example, the electronic device may be a mobile terminal with a shooting function, such as a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. Alternatively, it may be a professional shooting device such as a digital camera, SLR camera / mirrorless camera, action camera, gimbal camera, drone, etc. The specific type of electronic device is not limited in the embodiments of this application.

[0277] In another implementation, the unit that implements the steps of the above method can be configured as one or more processing elements, which can be integrated circuits, such as one or more ASICs, or one or more DSPs, or one or more FPGAs, or combinations of these types of integrated circuits. These integrated circuits can be integrated together to form a chip.

[0278] For example, this application also provides a chip that can be applied to the aforementioned electronic device. The chip includes one or more interface circuits and one or more processors; the interface circuits and processors are interconnected via lines; the processor receives and executes computer instructions from the memory of the electronic device through the interface circuits to implement the methods described in the foregoing embodiments.

[0279] This application also provides a computer program product, including computer-readable code, which, when executed in an electronic device, causes the electronic device to perform the methods described in the foregoing embodiments.

[0280] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0281] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0282] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium.

[0283] Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product, such as a program. This software product is stored in a program product, such as a computer-readable storage medium, and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0284] For example, embodiments of this application may also provide a computer-readable storage medium having computer program instructions stored thereon; when the computer program instructions are executed by an electronic device, the electronic device causes the electronic device to perform the method described in the foregoing embodiments.

[0285] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A shooting method, characterized in that, The method includes: The terminal device acquires scene information of the current shooting scene, as well as the capability information of the camera module configured on the terminal device; The terminal device determines the first camera module and the second camera module from the camera modules configured in the terminal device based on the scene information and the capability information. The terminal device determines the minimum exposure frame rate of the first camera module and the second camera module based on the scene information and the capability information. The terminal device configures the exposure frame rate of the first camera module to a first exposure frame rate, wherein the first exposure frame rate is greater than or equal to the minimum exposure frame rate of the first camera module. The terminal device configures the exposure frame rate of each second camera module such that the exposure frame rate of each second camera module satisfies the following conditions: the first exposure frame rate is an integer multiple of the exposure frame rate of the second camera module, and the exposure frame rate of the second camera module is greater than or equal to the minimum exposure frame rate of the second camera module; the number of second camera modules is one or more. The terminal device takes pictures through the first camera module and each of the second camera modules.

2. The method according to claim 1, characterized in that, Before the terminal device takes pictures through the first camera module and each of the second camera modules, the method further includes: For each of the second camera modules, the terminal device configures the number of intervals for sensing the first signal for the second camera module based on the exposure frame rate of the first camera module and the exposure frame rate of the second camera module; The terminal device takes pictures through the first camera module and each of the second camera modules, including: During the shooting process, the first camera module emits a first signal at the time corresponding to the frame header of each frame; each second camera module senses the first signal emitted by the first camera module according to the number of intervals in which the first signal is sensed; in response to sensing the first signal emitted by the first camera module, each second camera module begins shooting the current frame.

3. The method according to claim 1, characterized in that, The terminal device configures the exposure frame rate of each of the second camera modules such that the exposure frame rate of each of the second camera modules satisfies the following: the first exposure frame rate is an integer multiple of the exposure frame rate of the second camera module, including: The terminal device configures the exposure frame rate of each of the second camera modules such that the first exposure frame rate and the exposure frame rate of any one of the second camera modules are proportional to each other, and the exposure frame rates of any two of the second camera modules are also proportional to each other.

4. The method according to claim 3, characterized in that, The current shooting scene includes: a scene with high requirements for time consistency, or a scene with low requirements for time consistency; The terminal device configures the exposure frame rate of each of the second camera modules such that the first exposure frame rate and the exposure frame rate of any one of the second camera modules are proportionally multiple of each other, and the exposure frame rates of any two second camera modules are also proportionally multiple of each other, including: When the current shooting scene is a scene with high requirements for time consistency, the terminal device configures the exposure frame rate of each second camera module so that the first exposure frame rate and the exposure frame rate of any second camera module are proportional to each other, and the exposure frame rates of any two second camera modules are also proportional to each other.

5. The method according to any one of claims 1-4, characterized in that, The terminal device configures the exposure frame rate of each of the second camera modules such that the exposure frame rate of each of the second camera modules satisfies the following conditions: the first exposure frame rate is an integer multiple of the exposure frame rate of the second camera module, and the exposure frame rate of the second camera module is greater than or equal to the minimum exposure frame rate of the second camera module, including: For each second camera module, when the range from the lowest exposure frame rate of the second camera module to the first exposure frame rate includes multiple second exposure frame rates that satisfy the first exposure frame rate being an integer multiple of the second exposure frame rate, the terminal device configures the minimum value among the multiple second exposure frame rates as the exposure frame rate of the second camera module.

6. The method according to any one of claims 1-4, characterized in that, The terminal device acquires scene information of the current shooting scene, including: The terminal device obtains scene information of the current shooting scene based on one or more of the preview screen corresponding to the current shooting scene, sensor information, and the first operation; The first operation is to select a shooting mode and / or set shooting parameters on the first interface displayed on the terminal device.

7. The method according to any one of claims 1-4, characterized in that, The scene information includes one or more of the following: focal length information, lighting information, and the user's shooting requirements.

8. The method according to any one of claims 1-4, characterized in that, The capability information includes one or more of the following: maximum exposure frame rate, reasonable focal length for shooting, and output color mode.

9. The method according to any one of claims 1-4, characterized in that, The method further includes: The terminal device controls the first camera module and each of the second camera modules to maintain a consistent exposure time for the first frame of the image during shooting.

10. The method according to any one of claims 1-4, characterized in that, The method further includes: The terminal device performs fusion processing on the images captured by the first camera module and the second camera module to obtain a first image.

11. A shooting device, characterized in that, The device includes: The acquisition module is used to acquire scene information of the current shooting scene and capability information of the camera module configured in the device; A configuration module is configured to: determine a first camera module and a second camera module from the camera modules configured in the device based on the scene information and the capability information; determine the minimum exposure frame rate of the first camera module and the second camera module based on the scene information and the capability information; configure the exposure frame rate of the first camera module as a first exposure frame rate, wherein the first exposure frame rate is greater than or equal to the minimum exposure frame rate of the first camera module; configure the exposure frame rate of each second camera module such that the exposure frame rate of each second camera module satisfies: the first exposure frame rate is an integer multiple of the exposure frame rate of the second camera module, and the exposure frame rate of the second camera module is greater than or equal to the minimum exposure frame rate of the second camera module; the number of second camera modules is one or more. The shooting module is used to take pictures through the first camera module and each of the second camera modules.

12. An electronic device, characterized in that, include: A processor, and a memory for storing processor-executable instructions; When the processor is configured to execute the instructions, it causes the electronic device to implement the method as described in any one of claims 1-10.

13. A computer-readable storage medium having computer program instructions stored thereon; characterized in that, When the computer program instructions are executed by the electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-10.

14. A computer program product comprising computer-readable code, characterized in that, When the computer-readable code is executed in an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-10.

Images