Control Method for Multiple Cameras and Self-Propelled Device

Abstract

The present application provides a control method for multi-cameras, which is used for a self-propelled device. The self-propelled device includes a main camera, at least one slave camera, and a controller. The method includes: starting the main camera to perform image acquisition, and the controller periodically receiving an interrupt signal; in response to receiving the interrupt signal, the controller dynamically adjusts the frame rate of the slave camera according to the frame rate information of the main camera, and controls the slave camera to perform image acquisition at intervals with the adjusted frame rate and the main camera. Through this solution, after the controller receives the interrupt signal, according to the frame rate information of the main camera, by using the characteristic of the variable frame rate of the slave camera for image acquisition, the time interval for the slave camera to acquire images is dynamically adjusted, so as to achieve the effect of multiple cameras collaborating to perform image acquisition.

Description

Technical Field

[0001] This application relates to the technical field of intelligent devices, and particularly to a control method for multiple cameras, a self-propelled device, a storage medium, and an electronic device. Background Art

[0002] With the development of artificial intelligence, intelligent devices in various scenarios have become increasingly common. For example, smart home devices, self-driving delivery trucks, etc. These intelligent devices can move by themselves to complete tasks such as cleaning and transportation. Self-propelled intelligent devices are all equipped with cameras. In order to obtain surrounding images from multiple directions and angles, there are often two or more cameras. For self-propelled devices using multiple cameras, due to crosstalk between line lasers or fill lights, the controller needs to ensure that multiple cameras are alternately exposed and alternately turned on. Therefore, how to effectively coordinate camera photography has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, this application provides a control method for multiple cameras, a self-propelled device, a storage medium, and an electronic device, mainly aiming to solve the technical problem of how to effectively control the coordination of multiple camera photography.

[0004] According to one aspect of this application, a control method for multiple cameras is provided for a self-propelled device. The self-propelled device includes a main camera, at least one sub-camera, and a controller. The method includes: starting the main camera for image acquisition, and the controller periodically receiving an interrupt signal; after receiving the interrupt signal, the controller dynamically adjusts the frame rate of the sub-camera according to the frame rate information of the main camera, and controls the sub-camera to perform image acquisition at intervals with the main camera at the adjusted frame rate.

[0005] According to one aspect of this application, a self-propelled device is provided. The self-propelled device includes a main camera, at least one sub-camera, and a controller; the main camera performs image acquisition at a first frame rate; the controller receives an interrupt signal generated in response to the main camera performing image acquisition, and dynamically adjusts the frame rate of the sub-camera according to the frame rate information of the main camera, and controls the sub-camera to perform image acquisition at intervals with the main camera at the adjusted frame rate.

[0006] According to one aspect of this application, a storage medium is provided. A computer program is stored in the storage medium, wherein the computer program is set to execute the above-mentioned control method for multiple cameras when running.

[0007] According to one aspect of the present application, there is provided an electronic device, including a memory and a processor. It is characterized in that a computer program is stored in the memory, and the processor is configured to run the computer program to execute the above-mentioned multi-camera control method.

[0008] By means of the above technical solution, a multi-camera control method, a self-propelled device, a storage medium and an electronic device provided by the present application use one of multiple cameras as the main camera, and perform image acquisition at a fixed frame rate, while sending a periodic interruption signal to the controller. After receiving the interruption signal, the controller dynamically adjusts the time interval for the slave camera to perform image acquisition according to the frame rate information of the main camera and the characteristic of the slave camera for variable frame rate image acquisition, so as to achieve the effect of multiple cameras collaborating for image acquisition.

[0009] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the description. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the following specifically gives the specific embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0011] Figure 1 It shows a schematic diagram of an implementation scenario of a multi-camera control method provided by an embodiment of the present application;

[0012] Figure 2 It shows another schematic diagram of an implementation scenario of a multi-camera control method provided by an embodiment of the present application;

[0013] Figure 3 It shows a flowchart of a multi-camera control method provided by an embodiment of the present application;

[0014] Figure 4 It shows a flowchart of Example 1 of a multi-camera control method provided by an embodiment of the present application;

[0015] Figure 5 It shows a timing diagram of Example 1 of a multi-camera control method provided by an embodiment of the present application;

[0016] Figure 6 It shows a flowchart of Example 2 of a multi-camera control method provided by an embodiment of the present application;

[0017] Figure 7Shows the timing diagram of Example 2 of a control method for multiple cameras provided by an embodiment of the present application;

[0018] Figure 8 Shows the structural schematic diagram of a self - walking device provided by an embodiment of the present application. Detailed implementation manners

[0019] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0020] See Figure 1 , which is a schematic diagram of an implementation scenario of a control method for multiple cameras provided by an embodiment of the present application. This scenario shows a self - walking device 10, and at least two cameras are configured on the self - walking device 10. Figure 1 Exemplarily shows a main camera 100 and a slave camera 101 in. The main camera 100 and the slave camera 101 cooperate to take pictures, so as to ensure obtaining images in multiple directions and angles, so as to more completely and clearly obtain the surrounding environment of the self - walking device 10, and provide a basis for the self - walking device 10 to avoid obstacles during walking. The self - walking device 10 can be an intelligent device such as a cleaning robot, an autonomous driving delivery vehicle, a companion robot, etc. The embodiments of the present application do not make limitations in this regard. It can be understood that the self - walking devices mentioned in the embodiments of the present application can be all existing and future self - walking intelligent devices. Among them, there can be multiple slave cameras. For example, as Figure 2 shown, the self - walking device 20 includes a main camera 200, a slave camera 201 and a slave camera 202.

[0021] See Figure 3 , which is a flowchart of Example 1 of a control method for multiple cameras provided by an embodiment of the present application. This method is used to coordinately control the shooting of multiple cameras of a self - walking device. Among them, the self - walking device includes a controller, a main camera and at least one slave camera. The control method for multiple cameras includes the following steps S301 - S303.

[0022] S301: Start the main camera to collect images, and the controller periodically receives interrupt signals.

[0023] For a self - driving device using multiple cameras, due to the mutual crosstalk between line lasers or fill lights, the controller needs to ensure that multiple cameras are alternately exposed and alternately turned on. Suppose there are two cameras, both with a frame rate of 30fps. One is a color camera (RGB camera) that can use an infrared fill light, and the other is a grayscale camera using TOF (Time of Flight) or a line laser. The photographing areas of the two cameras overlap. If no synchronization or cross - logic is added on both sides, when taking pictures simultaneously, the infrared fill light of the color camera will interfere with the grayscale camera after it is turned on, and when the time - of - flight camera is turned on or the line laser is turned on, it will also interfere with the color camera. Therefore, the embodiment of this application proposes this solution for how to effectively coordinate the photographing of multiple cameras.

[0024] In one implementation, the main camera is a color camera, and the slave camera is a grayscale camera (such as a line - laser camera or a time - of - flight camera). In another implementation, the main camera is a grayscale camera, and the slave camera is a color camera.

[0025] In one implementation, the interrupt signal can be sent by the main camera to the controller. For example, connect the fsync signal (frame - synchronization signal) of the main camera to the controller, and send the fsync signal to the controller as the interrupt signal. Or, the main camera sends a strobe signal (gating signal) to the controller as the interrupt signal. In another implementation, the controller can receive the interrupt signal through the MIPI (Mobile Industry Processor Interface) interface. For example, receive the start - of - frame signal (sof signal) or end - of - frame signal (eof signal) as the interrupt signal.

[0026] S202: In response to receiving the interrupt signal, the controller dynamically adjusts the frame rate of the slave camera according to the frame - rate information of the main camera, and controls the slave camera to perform image acquisition at the adjusted frame rate at intervals with the main camera.

[0027] In the embodiment of this application, the main camera can be a constant - frame - rate (CFR) camera, and the slave camera is a variable - frame - rate (VFR) camera. Therefore, the main camera shoots at a fixed frame rate. When starting shooting, it sends an interrupt signal to the controller, and the controller sets the offset time of the slave camera according to the interrupt signal, thereby dynamically adjusting the frame rate of the slave camera by utilizing the characteristic of the variable frame rate of the slave camera.

[0028] In one implementation, the dynamic adjustment of the frame rate of the slave camera is achieved through the following steps:

[0029] (1) The controller determines that the frame rate of the main camera is the first frame rate according to the frame - rate information of the main camera;

[0030] (2) Set the offset time based on the first frame rate to dynamically adjust the frame rate of the slave camera with variable frame rate.

[0031] In practice, after a sufficient long time after controlling the master camera, there may be shooting errors. For example, when the cumulative error between cameras reaches a threshold (such as 1 ms), the frame rate of the slave camera can be adjusted again to complete the synchronization requirement. Therefore, in one implementation, the multi-camera control method described above may further include the following steps: 1) The controller records the timestamps of the master camera and the slave camera when collecting the current image; 2) In response to determining that the difference between the timestamps is not less than the preset maximum cumulative error threshold, the controller adjusts the frame rate of the slave camera again according to the frame rate information of the master camera, and controls the slave camera to collect images at intervals with the adjusted frame rate and the master camera.

[0032] It can be seen that in the embodiments of the present application, one of the multiple cameras is used as the master camera to collect images at a fixed frame rate, and at the same time, a periodic interrupt signal (such as fsync, or strobe actively sent by the camera, or sof or eof signal when the master controller mipi receives) is sent to the controller. After receiving the interrupt signal, the controller adjusts the time interval for the slave camera to collect images dynamically according to the frame rate information of the master camera, taking advantage of the characteristic of the variable frame rate of the slave camera for image collection, so as to achieve the effect of multiple cameras collaborating to collect images.

[0033] The following uses two specific examples to schematically illustrate the multi-camera control method provided in the embodiments of the present application.

[0034] In the first example, the self-driving device includes two cameras, one master camera and one slave camera (for the implementation scenario, see Figure 1 ).

[0035] See Figure 4 , which shows a flowchart of a first example of a multi-camera control method provided in the embodiments of the present application, including the following steps S401-S405.

[0036] S401: Start the master camera to collect images at a fixed first frame rate, and the controller periodically receives interrupt signals.

[0037] S402: In response to receiving the interrupt signal, the controller records the timestamps of the master camera and the slave camera when shooting the first frame of images respectively, and calculates the difference between the timestamps.

[0038] S403: Set the offset time for the slave camera to capture images relative to the master camera according to the difference between the first frame rate and the time stamp. In one implementation, through the offset time, determine that the image capture interval between the master camera and the slave camera is half of the time interval for the master camera to capture two adjacent frames of images.

[0039] S404: Control the slave camera to offset the above offset time based on the second frame rate and perform the second frame image capture.

[0040] S405: Control the slave camera to start capturing images at the second frame rate starting from the third frame. The second frame rate can be the same as or different from the first frame rate, as long as it is ensured that starting from the third frame, the master camera captures images at the first frame rate, the slave camera captures images at the second frame rate, and the two cameras capture images at an alternating or fixed interval.

[0041] For example, take a color camera as the master camera and connect the fsync signal of the camera to the main controller. After enabling at the beginning, the controller records the time stamps when the master and slave cameras capture the first frame of images respectively. Starting from the second frame, the trigger signal of the master camera only records the time stamp. The controller corrects the frame rate of the slave camera according to the difference between the time stamps of the master and slave cameras, and realizes the effect of the slave camera taking pictures alternately with the master camera starting from the third frame of image. When the third frame of the slave camera outputs an image, continue to correct the frame rate to achieve fixed frame rate image output later, and complete the synchronization logic between the two cameras.

[0042] See Figure 5 , for the shooting timing diagrams of the two cameras.

[0043] Assume that in this example, the frame rate of the master camera is 30fps. After the master camera and the slave camera are enabled respectively, the master camera starts to capture images at time t1, and the slave camera starts to capture images at time t1'. With a frame rate of 30fps, the time interval for capturing two adjacent frames of images is approximately 33.3ms.

[0044] At time t2', adjust the frame rate of the slave camera to,

[0045] T_sync = t2 + 33.3 – t2' + 33.3 / 2

[0046] After the slave camera adjusts the image output time for the first time, when outputting an image again at t3', adjust the frame rate to 33.3ms again.

[0047] In this example, the time interval between the main camera capturing two adjacent frames of images is 33.3 ms, and the offset time set for the slave camera is 33.3 / 2 (i.e., offset time = half of the time interval between the main camera capturing two adjacent frames of images). The advantage of this setting is that when ensuring the main camera shoots at a fixed frame rate, the slave camera is controlled to shoot at half of the time interval. For example, Figure 5 in, after t3, it can be ensured that the main and slave cameras shoot crosswise at a fixed interval (33.3 / 2).

[0048] In the second example, the self-propelled device includes three cameras, one main camera and two slave cameras: the first slave camera and the second slave camera (for the implementation scenario, see Figure 2 ).

[0049] See Figure 6 , which shows a flowchart of the first example of a multi-camera control method provided by an embodiment of the present application, including the following steps S601 - S605.

[0050] S601: Start the main camera to perform image acquisition at a fixed first frame rate, and the controller periodically receives an interrupt signal.

[0051] S602: In response to receiving the interrupt signal, the controller records the timestamps of the main camera, the first slave camera, and the second slave camera when each captures the first frame of image, and calculates the first timestamp difference between the main camera and the first slave camera and the second timestamp difference between the main camera and the second slave camera.

[0052] S603: According to the first frame rate of the main camera and the first timestamp difference, set the first offset time for the first slave camera to perform image acquisition relative to the main camera, and according to the first frame rate of the main camera and the second timestamp difference, set the second offset time for the second slave camera to perform image acquisition relative to the main camera. Among them, the first offset time and the second offset time are different. In one implementation, through the set first offset time and second offset time, it is determined that the time interval between the main camera, the first slave camera, and the second slave camera for capturing two adjacent frames of images is one-third of the time interval between the main camera capturing two adjacent frames of images.

[0053] S604: Control the first slave camera to perform the second frame of image acquisition by offsetting the first offset time based on the second frame rate, and control the second slave camera to perform the second frame of image acquisition by offsetting the second offset time based on the third frame rate.

[0054] S605: Control the first slave camera and the second slave camera to start fixing image acquisition at the second frame rate and the third frame rate respectively starting from the third frame image. Among them, the second frame rate and the third frame rate can be the same as the first frame rate or different, as long as it is ensured that starting from the third frame, the main camera performs image acquisition at the first frame rate, the first slave camera performs image acquisition at the second frame rate, the second slave camera performs image acquisition at the third frame rate, and the three cameras perform image acquisition at cross or fixed intervals.

[0055] For example, use a color camera as the main camera and connect the fsync signal of the camera to the main controller. After enabling at the beginning, the controller records the timestamps when the main and slave cameras respectively capture the first frame of image. Starting from the second frame, the trigger signal of the main camera still only records the timestamp. The controller corrects the frame rate of the slave camera according to the difference in timestamps between the main and slave cameras to achieve the effect of the slave camera taking pictures crosswise with the main camera when the third frame of the slave camera outputs the image. When the third frame of the slave camera outputs the image, continue to correct the frame rate to achieve fixed frame rate output later and complete the synchronization logic between the two cameras.

[0056] See Figure 6 , which is the shooting timing diagram of the three cameras.

[0057] Assume that in this example, the frame rate of the main camera is 30fps. After the main camera and the two slave cameras are enabled respectively, the main camera starts image acquisition at time t1, the first slave camera starts image acquisition at time t1', and the second slave camera starts image acquisition at time t1". With a frame rate of 30fps, the time interval for acquiring two adjacent frames of images is approximately 33.3ms.

[0058] At time t2', adjust the frame rate of the first slave camera to

[0059] T_sync = t2 + 33.3 – t2' + 33.3 / 3

[0060] At time t2", adjust the frame rate of the second slave camera to:

[0061] T_sync = t2 + 33.3 – t2' + 33.3*2 / 3

[0062] After the first adjustment of the output time of the first slave camera, when the first slave camera outputs the image again at t3', adjust the frame rate to 33.3ms. Similarly, after the first adjustment of the output time of the second slave camera, when the second slave camera outputs the image again at t3", adjust the frame rate to 33.3ms.

[0063] In this example, the time interval between two adjacent frames captured by the main camera is 33.3 ms, and the offset time set for the slave camera is 33.3 / 3 (i.e., the offset time = 1 / 3 of the above time interval). The advantage of such a setting is that when ensuring that the main camera shoots at a fixed frame rate, the first slave camera shoots at 1 / 3 of the duration, and the second slave camera shoots at 2 / 3 of the duration. For example, Figure 7 In [reference], after t3, it can be ensured that the main camera, the first slave camera, and the second slave camera shoot crosswise at a fixed interval (33.3 / 3).

[0064] It can be seen that by setting the offset time of the slave camera, it is possible to ensure crosswise or interval shooting between the main camera and the slave cameras. Generally speaking, the offset time can be set according to the shooting frame rate of the main camera and the number of cameras. For example, set: offset time = time interval between two adjacent frames captured by the main camera / number of cameras. In the above two examples, when there are two cameras, the offset time = 33.3 / 2, and when there are three cameras, the offset time = 33.3 / 3, thereby ensuring that these multiple cameras shoot crosswise at a basically fixed interval.

[0065] The embodiment of the present application has significant effects compared with the existing solutions. In the existing solutions, the module itself supports the frame synchronization mechanism, but there are restrictions on the hardware. For example, it must be the same model module, must have the same frame rate, and can only perform frame synchronization with simultaneous exposure, and crosswise or interval synchronization cannot be achieved. In the embodiment of the present application, the main camera shoots at a fixed frame rate, and the controller controls the shooting offset of the slave camera, so as to utilize the characteristic of the variable frame rate of the slave camera to realize the crosswise or interval shooting logic when taking pictures with multiple cameras; the embodiment of the present application has relatively low requirements for the cameras. As long as the slave camera can dynamically adjust the frame rate and the exposure time is controllable, the effect of crosswise or interval shooting between multiple cameras can be achieved.

[0066] Corresponding to the above control method for multiple cameras, the embodiment of the present application also provides a self-driving device. Refer to Figure 8 which shows a schematic structural diagram of the self-driving device provided by the embodiment of the present application.

[0067] The self-driving device includes a main camera 801, at least one slave camera 802, and a controller 803;

[0068] The main camera 801 performs image acquisition at a first frame rate;

[0069] The controller 803 periodically receives an interrupt signal generated in response to the image acquisition of the main camera 801, and dynamically adjusts the frame rate of the slave camera 802 according to the frame rate information of the main camera 801, and controls the slave camera 802 to perform image acquisition at intervals with the main camera 801.

[0070] In one implementation,

[0071] The controller 803 is further configured to record the timestamps of the current frames acquired by the main camera 801 and the slave camera 802, and determine that the difference between the timestamps is not less than a preset maximum cumulative error threshold. The controller 802 then adjusts the frame rate of the slave camera 802 again according to the frame rate information of the main camera 801, and controls the slave camera 802 to perform image acquisition at the adjusted frame rate at intervals with the main camera 801.

[0072] In one implementation,

[0073] The controller 803 uses the frame synchronization signal or the gating signal sent by the main camera 801 as the interrupt signal, or the controller 803 uses the frame start signal or the frame end signal sent by the main camera 801 as the interrupt signal.

[0074] In one implementation, the main camera 801 is a color camera and the slave camera 802 is a grayscale camera; or the main camera 801 is a grayscale camera and the slave camera 802 is a color camera.

[0075] In one implementation,

[0076] The controller 803 is specifically configured to determine the frame rate of the main camera as a first frame rate according to the frame rate information of the main camera, set an offset time based on the first frame rate, and dynamically adjust the frame rate of the slave camera 802 with variable frame rate.

[0077] In one implementation,

[0078] The controller 803 records the timestamps of the first frames acquired by the main camera 801 and the slave camera 802 respectively, calculates the difference between the timestamps, and sets an offset time for the slave camera 802 to perform image acquisition relative to the main camera 801 according to the first frame rate and the difference between the timestamps;

[0079] The slave camera 802 performs second-frame image acquisition by offsetting the offset time based on a second frame rate, and starting from the third frame, performs image acquisition at the second frame rate fixedly.

[0080] In one implementation,

[0081] The image acquisition time interval between the main camera 801 and the slave camera 802 is half of the time interval between the main camera 801 acquiring two adjacent frames of images.

[0082] In one implementation,

[0083] The slave cameras 802 include a first slave camera and a second slave camera;

[0084] The controller 803 is specifically configured to determine, according to the frame rate information of the master camera 801, that the frame rate of the master camera 801 is a first frame rate; based on the first frame rate, set a first offset time and a second offset time for the first slave camera and the second slave camera respectively, and dynamically adjust the frame rates of the first slave camera and the second slave camera with variable frame rates; wherein, the values of the first offset time and the second offset time are different.

[0085] In one implementation,

[0086] The controller 803 records the timestamps of the master camera 801, the first slave camera, and the second slave camera when each acquires the first frame of image, and calculates a first timestamp difference between the master camera 801 and the first slave camera and a second timestamp difference between the master camera and the second slave camera;

[0087] According to the first frame rate of the master camera 801 and the first timestamp difference, set a first offset time for the first slave camera to acquire images relative to the master camera 801, and according to the first frame rate of the master camera 801 and the second timestamp difference, set a second offset time for the second slave camera to acquire images relative to the master camera 801;

[0088] The first slave camera acquires a second frame of image by offsetting the first offset time based on a second frame rate, and starting from the third frame, acquires images at the second frame rate fixedly;

[0089] The second slave camera acquires a second frame of image by offsetting the second offset time based on a third frame rate, and starting from the third frame, acquires images at the third frame rate fixedly.

[0090] In one implementation,

[0091] The time interval for image acquisition among the master camera 801, the first slave camera, and the second slave camera is one-third of the time for the master camera to acquire images.

[0092] An embodiment of the present application further provides a storage medium, in which a computer program is stored, and wherein the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0093] Optionally, in this embodiment, the above storage medium may be configured to store a computer program for executing the following steps:

[0094] (1) Start the main camera for image acquisition, and the controller periodically receives interruption signals;

[0095] (2) In response to receiving the interruption signal, the controller dynamically adjusts the frame rate of the secondary camera according to the frame rate information of the main camera, and controls the secondary camera to perform image acquisition at intervals with the adjusted frame rate relative to the main camera.

[0096] Optionally, in this embodiment, the above storage medium may include, but is not limited to: various media such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), external hard drives, magnetic disks, or optical discs that can store computer programs.

[0097] An embodiment of the present application further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0098] Optionally, the above electronic device may further include a transmission device and input / output devices. Among them, the transmission device is connected to the above processor, and the input / output devices are connected to the above processor.

[0099] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0100] (1) Start the main camera for image acquisition, and the controller periodically receives interruption signals;

[0101] (2) In response to receiving the interruption signal, the controller dynamically adjusts the frame rate of the secondary camera according to the frame rate information of the main camera, and controls the secondary camera to perform image acquisition at intervals with the adjusted frame rate relative to the main camera.

[0102] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated herein.

[0103] The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0104] In the above embodiments of the present application, the descriptions of the respective embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0105] In several embodiments provided by this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed with each other can be through some interfaces. The indirect couplings or communication connections of units or modules can be in electrical or other forms.

[0106] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0107] In addition, in each embodiment of this application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0108] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of this application. The aforementioned storage medium includes: various media such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0109] The above is only the preferred embodiment of this application. It should be noted that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of this application.

Claims

1. A control method for multiple cameras, characterized in that, For a self - propelled device, the self - propelled device includes a main camera, at least one slave camera and a controller, and the slave camera is a variable - frame - rate camera; the method includes: Start the main camera for image acquisition, and the controller periodically receives an interrupt signal; In response to receiving the interrupt signal, the controller dynamically adjusts the frame rate of the slave camera according to the frame rate information of the main camera, and controls the slave camera to perform image acquisition at an adjusted frame rate at intervals with the main camera; The controller dynamically adjusts the frame rate of the slave camera according to the frame rate information of the main camera, including: the controller records the timestamps of the first - frame image and the second - frame image captured by the main camera and the slave camera respectively, and corrects the time when the slave camera captures the third - frame image according to the difference between the timestamps of the main camera and the slave camera. Wherein, the time interval from the time t2' when the slave camera captures the second - frame image to the time t3' when the slave camera captures the third - frame image is determined as: T_sync = t2 + the time interval for the main camera to capture adjacent images - t2' + offset time, t2 is the time when the main camera captures the second - frame image, the offset time = the time interval for the main camera to capture adjacent two - frame images / the number of cameras, and after the time t3' when the slave camera captures the third - frame image, the frame rate of the slave camera is adjusted to the frame rate of the main camera again.

2. The method according to claim 1, wherein It further includes: The controller records the timestamps of the current images captured by the main camera and the slave camera; In response to determining that the difference between the timestamps is not less than a preset maximum cumulative error threshold, the controller adjusts the frame rate of the slave camera again according to the frame rate information of the main camera, and controls the slave camera to perform image acquisition at an adjusted frame rate at intervals with the main camera.

3. The method according to claim 1, characterized in that, The starting the main camera for image acquisition and the controller periodically receiving an interrupt signal includes: Start the main camera for image acquisition; The controller uses the frame - synchronization signal or strobe signal sent by the main camera as the interrupt signal, or the controller uses the frame - start signal or frame - end signal sent by the main camera as the interrupt signal.

4. The method according to claim 1, characterized in that, The main camera is a color camera and the slave camera is a grayscale camera; or the main camera is a grayscale camera and the slave camera is a color camera.

5. The method according to claim 1, characterized in that It further includes: Through the offset time, control the image acquisition time interval between the main camera and the slave camera to be half of the time interval for the main camera to capture adjacent two - frame images.

6. The method according to any one of claims 1-4, characterized in that, The slave camera includes a first slave camera and a second slave camera; The dynamically adjusting the frame rate of the slave camera according to the frame rate information of the main camera includes: Determine the frame rate of the main camera as a first frame rate according to the frame rate information of the main camera; Based on the first frame rate, set a first offset time and a second offset time for the first slave camera and the second slave camera respectively, and dynamically adjust the frame rates of the variable - frame - rate first slave camera and second slave camera; wherein, the values of the first offset time and the second offset time are different.

7. The method according to claim 6, characterized in that, Based on the first frame rate, set a first offset time and a second offset time for the first slave camera and the second slave camera respectively, and dynamically adjust the frame rates of the variable-frame-rate first slave camera and the second slave camera, including: The controller records the timestamps of the main camera, the first slave camera, and the second slave camera when they each capture the first frame of an image, and calculates the first timestamp difference between the main camera and the first slave camera and the second timestamp difference between the main camera and the second slave camera; According to the first frame rate of the main camera and the difference in the first timestamp, set a first offset time for the first slave camera to capture an image relative to the main camera, and according to the first frame rate of the main camera and the difference in the second timestamp, set a second offset time for the second slave camera to capture an image relative to the main camera; Control the first slave camera to capture a second frame of an image by offsetting the first offset time based on a second frame rate, and control the second slave camera to capture a second frame of an image by offsetting the second offset time based on a third frame rate; Control the first slave camera and the second slave camera to respectively capture images at the second frame rate and the third frame rate starting from the third frame of an image.

8. The method according to claim 7, wherein It further includes: According to the first offset time and the second offset time, determine that the time interval for image capture among the main camera, the first slave camera, and the second slave camera is one-third of the time interval for the main camera to capture two adjacent frames of an image.

9. A self-propelled device, characterized in that, The self-propelled device includes a main camera, at least one slave camera, and a controller; The main camera captures images at a first frame rate; The controller receives an interrupt signal generated in response to the main camera capturing an image, and dynamically adjusts the frame rate of the slave camera according to the frame rate information of the main camera, and controls the slave camera to capture images at intervals with the main camera at the adjusted frame rate; Specifically, the controller records the timestamps of the main and slave cameras when they capture the first and second frames of an image respectively, and corrects the time when the slave camera captures the third frame of an image according to the difference in timestamps between the main and slave cameras. The time interval from the time t2' when the slave camera captures the second frame of an image to the time t3' when it captures the third frame of an image is determined as: T_sync = t2 + the time interval for the main camera to capture two adjacent images - t2' + the offset time, where t2 is the time when the main camera captures the second frame of an image, the offset time = the time interval for the main camera to capture two adjacent frames of an image / the number of cameras, and after the time t3' when the slave camera captures the third frame of an image, the frame rate of the slave camera is adjusted to the frame rate of the main camera.

10. The self-propelled device according to claim 9, wherein The controller is further configured to record the timestamps of the current frames acquired by the main camera and the slave camera, and in response to determining that the difference between the timestamps is not less than a preset maximum cumulative error threshold, the controller adjusts the frame rate of the slave camera again according to the frame rate information of the main camera, and controls the slave camera to perform image acquisition at the adjusted frame rate at intervals with the main camera.

11. The self-propelled device according to claim 9, wherein The controller uses the frame synchronization signal or the strobe signal sent by the main camera as the interrupt signal, or the controller uses the frame start signal or the frame end signal sent by the main camera as the interrupt signal.

12. The self-propelled device according to claim 11, characterized in that, The main camera is a color camera and the slave camera is a grayscale camera; or the main camera is a grayscale camera and the slave camera is a color camera.

13. The self-propelled device according to claim 9, characterized in that, The time interval between image acquisitions between the main camera and the slave camera is half of the time interval between adjacent two-frame acquisitions of the main camera.

14. The self-propelled device according to any one of claims 9-12, characterized in that, The slave camera includes a first slave camera and a second slave camera; The controller is specifically configured to determine, according to the frame rate information of the main camera, that the frame rate of the main camera is a first frame rate; based on the first frame rate, set a first offset time and a second offset time for the first slave camera and the second slave camera respectively, and dynamically adjust the frame rates of the variable-frame-rate first slave camera and the second slave camera, where the values of the first offset time and the second offset time are different.

15. The self-propelled device according to claim 14, wherein The controller records the timestamps of the first frames acquired by the main camera, the first slave camera, and the second slave camera respectively, and calculates a first timestamp difference between the main camera and the first slave camera and a second timestamp difference between the main camera and the second slave camera; According to the first frame rate of the main camera and the difference of the first timestamp, set a first offset time for the first slave camera to perform image acquisition relative to the main camera, and according to the first frame rate of the main camera and the difference of the second timestamp, set a second offset time for the second slave camera to perform image acquisition relative to the main camera; The first slave camera offsets the first offset time based on a second frame rate to perform second-frame image acquisition, and starting from the third frame, fixedly performs image acquisition at the second frame rate; The second slave camera offsets the second offset time based on a third frame rate to perform second-frame image acquisition, and starting from the third frame, fixedly performs image acquisition at the third frame rate.

16. The self-propelled device according to claim 15, characterized in that, The time interval between image acquisitions among the main camera, the first slave camera, and the second slave camera is one-third of the time interval between adjacent two-frame acquisitions of the main camera.

17. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is set to execute the method described in any one of claims 1 to 8 when running.

18. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method described in any one of claims 1 to 8.

Images