Multi-sensor synchronized time transfer visual-inertial odometry system and time transfer method thereof

Abstract

This application discloses a multi-sensor synchronized visual inertial odometry system and its synchronization method. The system includes: a computing platform, an MCU, a data bus, a communication interface, and two or more sensors. The computing platform is connected to the MCU via the data bus, and the MCU is connected to the sensors via the communication interface. The computing platform is also connected to both the communication interface and the sensors. The computing platform is used to initialize the MCU's parameters, acquire sensor data measured by the sensors, and perform preset processing. The MCU is used to control the sensor's triggering method according to the initialization parameters. The sensors are used to acquire sensor data. This application can reduce synchronization time errors and improve sensor compatibility, and can be widely applied in the field of multi-sensor synchronized time synchronization.

Description

Technical Field

[0001] This application relates to the field of multi-sensor synchronous time synchronization, and in particular to a visual inertial odometer system and time synchronization method for multi-sensor synchronous time synchronization. Background Technology

[0002] Currently, sensor synchronization methods for visual inertial odometry systems include software synchronization and hardware synchronization. However, software synchronization suffers from problems such as large synchronization time errors and data delays, while hardware synchronization is limited by issues such as synchronization frequency, sensor selection, and the number of connected sensors. Summary of the Invention

[0003] In view of this, this application provides a visual inertial odometer system and time synchronization method with multiple sensors to reduce synchronization time errors and improve sensor compatibility.

[0004] One aspect of this application provides a multi-sensor synchronized time visual inertial odometry system, comprising:

[0005] Computing platform, MCU, data bus, communication interface, and two or more sensors;

[0006] The computing platform is connected to the MCU via the data bus, the MCU is connected to the sensor via the communication interface, and the computing platform is also connected to both the communication interface and the sensor.

[0007] The computing platform is used to initialize the MCU by setting parameters, acquire sensor data measured by the sensor, and perform preset processing.

[0008] The MCU is used to control the triggering mode of the sensor according to the initialization parameters;

[0009] The sensor is used to acquire sensor data.

[0010] Optionally, the computing platform uses an ARM processor or a PC host.

[0011] Optionally, the data bus is a UART;

[0012] The communication interface uses GPIO.

[0013] Optionally, the sensor includes a camera, a ranging radar, an inertial measurement unit, and a positioning sensor.

[0014] Optionally, the camera includes a monocular camera and a multi-view camera;

[0015] The ranging radar includes TOF and lidar;

[0016] The inertial measurement unit includes: a gyroscope, an accelerometer, and a magnetometer;

[0017] The positioning sensors include RTK, GPS, and BeiDou navigation and positioning sensors.

[0018] Another aspect of this application provides a time synchronization method for a visual inertial odometry system, comprising:

[0019] Receive initialization parameters sent by the computing power platform, the initialization parameters including the triggering methods of each sensor;

[0020] Based on the initialization parameters, sensor frame data statistics are performed to obtain frame statistics data.

[0021] The corresponding sensor is triggered synchronously based on the frame statistics data and the triggering method;

[0022] The trigger frame information of the sensor is determined from the frame statistics data, and the trigger frame information is transmitted to the computing platform.

[0023] Optionally, the initialization parameters include the sensor's active triggering mode and passive triggering mode, the sensor frame interval triggering parameters, and the sensor's single-condition triggering mode and multi-condition triggering mode;

[0024] The initialization parameters sent by the receiving computing power platform include the triggering methods of each sensor, including:

[0025] The system receives information from the computing platform regarding the active and passive triggering modes of the sensor, the sensor frame interval triggering parameters, and the single-condition and multi-condition triggering modes of the sensor.

[0026] Optionally, the step of performing sensor frame data statistics based on the initialization parameters to obtain frame statistics data includes:

[0027] The number of times the sensor has been triggered is obtained by counting the number of interrupts by the MCU, and the received frame data is compared with the data received by the computing platform to perform sensor frame data statistics and obtain frame statistics data.

[0028] Optionally, the step of synchronously triggering the corresponding sensor based on the frame statistics and the triggering method includes:

[0029] The output frame rate of each sensor is determined based on the frame statistics data.

[0030] The synchronization frame rate is determined based on the output frame rate of each sensor.

[0031] The corresponding sensor is triggered synchronously with the specified synchronization frame rate and the specified triggering method.

[0032] Optionally, determining the trigger frame information of the sensor from the frame statistics includes:

[0033] Based on the synchronization frame rate, determine the trigger frames for synchronously triggering each sensor, mark the trigger frames as keyframes, and obtain the trigger frame information.

[0034] This application initializes the MCU with parameters through a computing platform. The MCU then configures the triggering method and triggering frame rate of each sensor according to the initialization parameters, thereby achieving synchronous triggering of each sensor and ensuring that different sensors collect data at the same time point, which greatly reduces the synchronization time error. At the same time, the MCU can also package the triggering frames of synchronous triggering to obtain triggering frame information, and then send the triggering frame information to the computing platform, so that the computing platform can distinguish the triggering frames and perform synchronous data calculation. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 An example structural diagram of a visual inertial odometry system with multi-sensor synchronous time synchronization provided in this application embodiment;

[0037] Figure 2 This is a flowchart illustrating a time synchronization method for a visual inertial odometry system provided in an embodiment of this application. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0039] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0040] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0042] Existing sensor synchronization triggering has the following problems:

[0043] 1. Time uncertainty: Existing sensor triggering may be affected by external factors, resulting in a certain degree of uncertainty in the triggering time. This may lead to inconsistent data acquisition times or time delays, thereby affecting the accuracy and consistency of the data.

[0044] 2. Data asynchrony: When existing sensors are triggered, data from different sensors may be collected at different times, which may lead to inconsistencies between the data, making it difficult to align and analyze the data.

[0045] 3. Difficulty in achieving multi-sensor collaboration: In applications that require multiple sensors to work together, existing sensor triggers may struggle to achieve accurate data synchronization and collaborative operation, which could lead to data inconsistency or difficulty in effective data fusion and analysis.

[0046] Therefore, this application uses an MCU combined with hardware and software to synchronize various sensors, which can effectively solve the time delay error (TD) problem generated during the synchronization process, can connect to multiple sensors for data synchronization, and supports passive and active synchronization trigger signals.

[0047] The following section provides a detailed description of the multi-sensor synchronized time visual inertial odometry system provided in this application. For details, please refer to [link / reference needed]. Figure 1 , Figure 1 An example structural diagram of a multi-sensor synchronized visual inertial odometry system is shown. The visual inertial odometry system includes: a computing platform, an MCU, a data bus, a communication interface, and two or more sensors.

[0048] The computing platform is connected to the MCU via the data bus, the MCU is connected to the sensor via the communication interface, and the computing platform is also connected to both the communication interface and the sensor.

[0049] The computing platform is used to initialize the MCU by setting parameters, acquire sensor data measured by the sensor, and perform preset processing.

[0050] The MCU is used to control the triggering mode of the sensor according to the initialization parameters;

[0051] The sensor is used to acquire sensor data.

[0052] Furthermore, the computing platform uses an ARM processor or a PC host.

[0053] Furthermore, the data bus adopts UART; the communication interface adopts GPIO.

[0054] Furthermore, the sensors include a camera, a ranging radar, an inertial measurement unit, and a positioning sensor.

[0055] Furthermore, the camera includes a monocular camera and a multi-view camera; the ranging radar includes TOF and lidar; the inertial measurement unit (IMU) includes a gyroscope, an accelerometer, and a magnetometer; and the positioning sensor includes an RTK sensor, a GPS sensor, and a BeiDou navigation and positioning sensor. RTK refers to Real-time kinematic, a real-time dynamic carrier phase differential technology.

[0056] It should be noted that the visual inertial odometry system of this application embodiment can also be connected to other optional sensors, which can be selected according to the actual situation. The sensors mentioned above are only examples and do not actually limit the sensors of this application embodiment.

[0057] Specifically, triggering through IMU management can achieve the following beneficial effects:

[0058] 1. High-precision time synchronization: IMUs typically have high sampling frequencies, providing microsecond-level time synchronization. This means that more accurate data acquisition and time alignment can be achieved through IMU synchronization triggering.

[0059] 2. Data Consistency: The IMU can provide synchronous triggering of data from multiple sensors, ensuring that data from different sensors are collected at the same point in time. This helps improve data consistency and accuracy, especially in applications requiring multiple sensors to work together.

[0060] 3. Reduce sampling bias: Synchronous triggering via IMU can reduce sampling bias between different sensors. Sampling bias between sensors can lead to data inconsistencies or time delays, and using IMU for synchronous triggering can solve these problems to some extent.

[0061] In summary, IMU synchronous triggering can provide higher time accuracy and data consistency, helping to solve the problems of time uncertainty and data asynchrony caused by ordinary sensor triggering. This makes IMU synchronous triggering a significant advantage in applications requiring high-precision data acquisition and multi-sensor collaborative operation.

[0062] Specifically, the visual inertial odometry system will be explained with a concrete example below.

[0063] Synchronous triggering plays a crucial role in visual inertial odometry systems, as the time-to-action (TD) generated from the input data directly affects the accuracy of the output results. The synchronous triggering method in this application effectively solves the problems of excessively large and inconsistent TD values.

[0064] An ARM processor / PC host can serve as a computing platform, used to process data transmitted from sensors and perform communication data processing, such as sensor parameter settings and MCU trigger parameter settings. The MCU can control the entire triggering logic, performing low-data-volume calculations, such as frame statistics and GPIO status acquisition and control, thus requiring high real-time performance. The MCU hardware synchronous trigger can connect to various sensors, including monocular and binocular cameras, TOF, LiDAR, multi-axis IMUs, RTK, GPS, and other optional sensors.

[0065] The working principle of the visual inertial odometry system according to the embodiments of this application is then explained: The ARM processor / PC host can first initialize the parameters to the MCU via UART. The initialization parameter setting process can include setting the active and passive triggering methods and the number of keyframes for each sensor. For example, currently, RTK is used as the active triggering condition, with a triggering interval of 10ms. The MCU uses the timestamp received from the RTK satellite as the interrupt triggering condition to passively trigger other sensors. The ARM processor / PC host will receive packaged RTK data from the MCU during both triggering and non-triggering processes (packaged means that keyframe marking information is added). The data at the time of triggering will be marked with keyframe information for timestamp purposes, that is, as long as it is a keyframe, it is synchronous trigger frame data.

[0066] The MCU interrupt I / O trigger supports both passive and active modes, enabling multiple sensors to be triggered simultaneously, so that data is output to the ARM processor / PC host at the same time. The MCU communicates with the ARM processor / PC host through UART. The ARM processor / PC host can configure synchronization parameters for the MCU and obtain the current sensor status information from the MCU through UART.

[0067] Explanation of the difference between active and passive triggering: Active triggering means that only trigger signals are output (including level signals and soft triggering models), while passive triggering means that only trigger signals can be received.

[0068] Then, the time synchronization method of the visual inertial odometry system provided in this application is applied to the MCU in the aforementioned visual inertial odometry system, with reference to... Figure 2 The time synchronization method may include S200 to S230, as follows:

[0069] S200: Receive initialization parameters sent by the computing platform, the initialization parameters including the triggering methods of each sensor.

[0070] Furthermore, the initialization parameters include the sensor's active triggering mode and passive triggering mode, the sensor frame interval triggering parameters, and the sensor's single-condition triggering mode and multi-condition triggering mode;

[0071] The initialization parameters sent by the receiving computing power platform include the triggering methods of each sensor, including:

[0072] The system receives information from the computing platform regarding the active and passive triggering modes of the sensor, the sensor frame interval triggering parameters, and the single-condition and multi-condition triggering modes of the sensor.

[0073] S210: Perform sensor frame data statistics based on the initialization parameters to obtain frame statistics data.

[0074] Furthermore, S210 may include: obtaining the number of times the sensor has been triggered by counting the number of interrupts by the MCU, and communicating with the computing platform to compare the received frame data, so as to perform sensor frame data statistics and obtain frame statistics data.

[0075] S220: Synchronously trigger the corresponding sensor according to the frame statistics data and the triggering method.

[0076] Furthermore, S220 may include:

[0077] The output frame rate of each sensor is determined based on the frame statistics data.

[0078] The synchronization frame rate is determined based on the output frame rate of each sensor.

[0079] The corresponding sensor is triggered synchronously with the specified synchronization frame rate and the specified triggering method.

[0080] S230: Determine the trigger frame information of the sensor from the frame statistics data, and transmit the trigger frame information to the computing platform.

[0081] Further, determining the trigger frame information of the sensor from the frame statistics may include:

[0082] Based on the synchronization frame rate, determine the trigger frames for synchronously triggering each sensor, mark the trigger frames as keyframes, and obtain the trigger frame information.

[0083] The application process of the time synchronization method in this application will be illustrated with specific examples below.

[0084] The computing platform initializes the MCU's synchronization parameters via UART. The MCU starts working according to the synchronization parameters and triggers each sensor according to frame statistics and trigger conditions, marking key frame and non-key frame information. Then, the MCU sends the key frame and non-key frame information to the computing platform via UART. At the same time as the trigger, the sensor sends the collected data to the computing platform.

[0085] Specifically, this may include the following steps:

[0086] Step 1: The computing platform configures initialization parameters to the MCU via UART. This may include initialization synchronization methods and synchronization parameters, such as configuring active / passive triggering methods, configuring sensor frame interval triggering, and configuring single-condition / multi-condition triggering.

[0087] Step 2: The MCU performs sensor frame data statistics based on the initialization parameters. Specifically, this may include: the MCU counting the number of times the data has been triggered via its own interrupt and communicating with the computing platform, and then comparing the received data frames to achieve sensor frame data statistics.

[0088] Step 3: The MCU triggers the corresponding sensor in an active / passive triggering mode based on the statistically obtained frame data and the triggering conditions.

[0089] Step 4: The MCU sends the current trigger frame information to the computing platform via UART.

[0090] Step 5: The sensor collects data based on the synchronization trigger signal generated in step 3 and outputs the data to the computing platform.

[0091] Furthermore, this application embodiment can also provide another more detailed application example for illustration.

[0092] In the use of visual inertial odometry systems, the optimal TD value is often calculated through estimation and optimization, followed by data synchronization and alignment. However, this method introduces errors in each estimation due to various uncertainties. The embodiments of this application utilize MCU hardware triggering to completely solve these problems. Specific example: Taking a binocular inertial visual odometry system as an example, its sensor typically consists of two cameras and one IMU. The interfaces are described below:

[0093] The camera's image output interface is MIPI-CSI; trigger interface is GPIO; parameter setting interface is I2C; IMU data interface is I2C; trigger interface is GPIO. This embodiment incorporates an MCU for frame calculation and trigger management of the specific data link. The camera's trigger GPIO and the IMU's I2C / trigger GPIO are respectively connected to the MCU. The computing platform is connected to the MCU's UART and the camera's MIPI-CSI. Optionally, assuming the MCU is currently set to be actively triggered by the IMU and passively triggered by the camera, the IMU's output frame rate is 480Hz, while the camera's maximum output frame rate is 120Hz. In this embodiment, the MCU's frame calculation can trigger the camera every four frames of 480Hz data, thus enabling the IMU to output 480Hz and the camera to output 120Hz to the computing platform. However, since the computing platform does not know whether the current IMU frame and the camera are key frames captured at the same time, the key frame marking function of the MCU in this embodiment can enable the computing platform to distinguish the current key frame information. The working principle of key frame marking includes the MCU performing frame calculation and marking key frames when the camera trigger conditions are met, marking key frame information in the current IMU data to facilitate the computing platform's distinction.

[0094] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0095] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional technology for an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0096] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0098] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0099] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0100] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0101] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0102] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A visual inertial odometry system with multi-sensor synchronous time synchronization, characterized in that, include: Computing platform, MCU, data bus, communication interface, and two or more sensors; The computing platform is connected to the MCU via the data bus, the MCU is connected to the sensor via the communication interface, and the computing platform is also connected to both the communication interface and the sensor. The computing platform is used to initialize the MCU by setting parameters, acquire sensor data measured by the sensor, and perform preset processing. The MCU is used to control the triggering mode of the sensor according to the initialization parameters; The sensor is used to acquire sensor data; The system is used to perform a time synchronization method for a visual inertial odometry system, the time synchronization method comprising: Receive initialization parameters sent by the computing power platform, the initialization parameters including the triggering methods of each sensor; Based on the initialization parameters, sensor frame data statistics are performed to obtain frame statistics data. The corresponding sensor is triggered synchronously based on the frame statistics data and the triggering method; The trigger frame information of the sensor is determined from the frame statistics data, and the trigger frame information is transmitted to the computing platform; The step of synchronously triggering the corresponding sensor based on the frame statistics data and the triggering method includes: The output frame rate of each sensor is determined based on the frame statistics data. The synchronization frame rate is determined based on the output frame rate of each sensor. The corresponding sensor is triggered synchronously with the specified synchronization frame rate and the specified triggering method; The sensor includes a camera and an inertial measurement unit; the keyframe marking function of the MCU enables the computing platform to distinguish the current keyframe information. The working principle of keyframe marking includes the MCU performing frame calculation and marking the keyframe when the camera trigger condition is met, and marking the keyframe information in the current inertial measurement unit data to facilitate the computing platform to distinguish.

2. The visual inertial odometry system with multi-sensor synchronous time synchronization according to claim 1, characterized in that, The computing platform uses an ARM processor or a PC host.

3. The visual inertial odometry system with multi-sensor synchronous time synchronization according to claim 1, characterized in that, The data bus uses UART; The communication interface uses GPIO.

4. A multi-sensor synchronized time-synchronized visual inertial odometry system according to any one of claims 1 to 3, characterized in that, The sensors also include ranging radar and positioning sensors.

5. A multi-sensor synchronized time-synchronized visual inertial odometry system according to claim 4, characterized in that, The cameras include monocular cameras and multi-view cameras; The ranging radar includes TOF and lidar; The inertial measurement unit includes: a gyroscope, an accelerometer, and a magnetometer; The positioning sensors include RTK sensors, GPS sensors, and BeiDou navigation and positioning sensors.

6. The visual inertial odometry system with multi-sensor synchronous time synchronization according to claim 1, characterized in that, The initialization parameters include the sensor's active and passive triggering modes, the sensor's frame interval triggering parameters, and the sensor's single-condition and multi-condition triggering modes. The initialization parameters sent by the receiving computing power platform include the triggering methods of each sensor, including: The system receives information from the computing platform regarding the active and passive triggering modes of the sensor, the sensor frame interval triggering parameters, and the single-condition and multi-condition triggering modes of the sensor.

7. A multi-sensor synchronized time-synchronized visual inertial odometry system according to claim 1, characterized in that, The step of performing sensor frame data statistics based on the initialization parameters to obtain frame statistics data includes: The number of times the sensor has been triggered is obtained by counting the number of interrupts by the MCU, and the received frame data is compared with the data received by the computing platform to perform sensor frame data statistics and obtain frame statistics data.

8. A visual inertial odometry system with multi-sensor synchronous time synchronization according to claim 1, characterized in that, Determining the trigger frame information of the sensor from the frame statistics includes: Based on the synchronization frame rate, determine the trigger frames for synchronously triggering each sensor, mark the trigger frames as keyframes, and obtain the trigger frame information.

Images