Multi-sensor time synchronization method, device, equipment, medium and product

Abstract

The embodiment of the invention discloses a multi-sensor time synchronization method, device and equipment, a medium and a product, and the method comprises the steps: for any image sensor, determining first time based on a readout time sequence signal of the image sensor; the first time is the time when a system on chip (SOC) receives a first pixel; obtaining the exposure time of the image in real time by using a vehicle-mounted image signal processor; determining correction time corresponding to the image sensor based on the first time, the exposure time and the interruption triggering time so as to complete time synchronization; the interruption triggering time is the time when the image sensor receives an interruption detection triggering signal. According to the technical scheme, the correction time of the image sensors is determined through the first time, the exposure time and the interruption triggering time, so that time synchronization of the image sensors is realized, the detection precision of the sensors is improved, and the problem of data inconsistency caused by time deviation when the image sensors work cooperatively is avoided.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of time synchronization technology, and in particular to a multi-sensor time synchronization method, apparatus, device, medium, and product. Background Art

[0002] In modern complex systems, such as autonomous vehicles, sensor fusion technology is crucial. This technology integrates data from different sensors to provide comprehensive environmental perception. For example, radar detects the distance and speed of objects, LiDAR provides precise three-dimensional spatial information, and cameras capture high-resolution visual images. However, due to the varying operating principles and design parameters of these sensors, they may collect data at different time intervals.

[0003] The importance of time synchronization is further emphasized during the collection and processing of sensor data. Each sensor may have its own clock source, which may deviate due to manufacturing variations, temperature changes, or other environmental factors. This deviation can cause timestamps recorded by different sensors to differ, even at the same sampling frequency. Furthermore, data transmission from the sensor to the processing unit may encounter unpredictable network or processing delays, which further affect the data's time accuracy. Summary of the Invention

[0004] The embodiments of the present disclosure provide a multi-sensor time synchronization method, apparatus, device, medium, and product, which achieve time synchronization of various image sensors and improve the detection accuracy of the sensors.

[0005] In a first aspect, a multi-sensor time synchronization method is provided, comprising:

[0006] For any image sensor, a first time is determined based on a readout timing signal of the image sensor; the first time is the time when the system on chip (SOC) receives the first pixel;

[0007] Using the on-board image signal processor to obtain the image exposure time in real time;

[0008] A correction time corresponding to the image sensor is determined based on the first time, the exposure time, and an interruption trigger time to complete time synchronization; the interruption trigger time is the time when the image sensor receives an interruption detection trigger signal.

[0009] In a second aspect, a multi-sensor time synchronization device is provided, comprising:

[0010] A first time determination module is configured to determine, for any image sensor, a first time based on a readout timing signal of the image sensor; the first time being the time when the system on chip (SOC) receives a first pixel;

[0011] An exposure time determination module, used to obtain the exposure time of an image in real time using an on-board image signal processor;

[0012] A correction time determination module is used to determine the correction time corresponding to the image sensor based on the first time, the exposure time and the interrupt trigger time to complete time synchronization; the interrupt trigger time is the time when the image sensor receives the interrupt detection trigger signal.

[0013] According to a third aspect, an electronic device is provided, including:

[0014] at least one processor; and,

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the multi-sensor time synchronization method as described in the first aspect above.

[0017] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the multi-sensor time synchronization method as described in the first aspect is implemented.

[0018] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the multi-sensor time synchronization method as described in the first aspect.

[0019] The disclosed embodiments disclose a multi-sensor time synchronization method, apparatus, device, medium, and product. The method includes: for any image sensor, determining a first time based on a readout timing signal of the image sensor; the first time being the time when the system-on-chip (SOC) receives the first pixel; using an on-board image signal processor to obtain the exposure time of the image in real time; determining a correction time corresponding to the image sensor based on the first time, the exposure time, and an interrupt trigger time to complete time synchronization; the interrupt trigger time being the time when the image sensor receives an interrupt detection trigger signal. This technical solution determines the correction time of the image sensor through the first time, the exposure time, and the interrupt trigger time to achieve time synchronization of each image sensor, improves the detection accuracy of the sensor, and avoids the problem of data inconsistency caused by time deviation when each image sensor works in coordination.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the embodiments of the present disclosure. Other features of the embodiments of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 This is a flowchart of a multi-sensor time synchronization method provided in the first embodiment of the present disclosure;

[0023] Figure 2 is a schematic diagram of an image exposure control process provided by the first embodiment of the present disclosure;

[0024] Figure 3 This is a structural diagram of a multi-sensor time synchronization device provided in the second embodiment of the present disclosure;

[0025] Figure 4 This is a structural diagram of an electronic device provided in Example 3 of the present disclosure. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solutions of the embodiments of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of them. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the embodiments of the present disclosure.

[0027] It should be noted that the terms "first", "second", etc. in the description and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Example 1

[0029] Figure 1 This is a flowchart of a multi-sensor time synchronization method provided in the first embodiment of the present disclosure. This embodiment is applicable to the case of time synchronization of multiple image sensors. The method can be executed by a multi-sensor time synchronization device. The multi-sensor time synchronization device can be implemented in the form of hardware and / or software. The multi-sensor time synchronization device can be configured in electronic devices, including but not limited to computers, computers, terminals, servers and other devices with data processing capabilities. Figure 1 As shown, the method includes:

[0030] S110 : For any image sensor, determine a first time based on a readout timing signal of the image sensor; the first time is the time when the system on chip (SOC) receives a first pixel.

[0031] In this embodiment, when multiple image sensors are time-synchronized, for any image sensor, the readout timing signal of the image sensor can be adjusted based on the frame synchronization signal received by the image sensor, so that the start time of the readout operation is synchronized with the frame synchronization signal, and the arrival of each frame synchronization signal will trigger the image camera to start reading out a new frame of image; the adjusted readout timing signal can trigger the output operation of the image sensor, so that the image sensor outputs pixels to the system on chip (SOC), and then the first time can be determined based on the readout timing signal of the image sensor, and the first time can be the time when the SOC receives the first pixel.

[0032] The readout timing signal of the image sensor may be an adjusted readout timing signal. For example, the readout timing signal inside the image sensor may be adjusted according to a frame synchronization signal received by the image sensor.

[0033] S120: Obtain the exposure time of the image in real time using the onboard image signal processor.

[0034] Specifically, the on-vehicle image signal processor (ISP) can be used to automatically adjust the exposure parameters of the camera (such as exposure time, gain, etc.) through the automatic exposure control (AEC) algorithm, and adjust the exposure settings of the on-vehicle camera in real time, including the resolution and exposure time of the image. Multiple exposure technology is used to accurately control the shooting process of each frame of the image, so that effective image data and embedded row data can be obtained in real time. The effective image data can be image data that can maintain high quality under different lighting conditions, and the embedded row data can store the exposure time of the image. The image exposure time refers to the length of time that the photosensitive element of the image sensor collects light when capturing an image. Exposure time can be one of the important parameters that affect image brightness and quality.

[0035] Figure 2 A schematic diagram of an image exposure control process provided in this embodiment is shown as follows: Figure 2 As shown, the frame synchronization signal can be represented as FSYNC, the image sensor's readout timing signal (internal signal) can be represented as internal signal, and the VSYNC signal is typically a pulse signal. When it goes high, it indicates the start of a new image row. The exposure process is represented by two trapezoids in the figure. The first trapezoid represents the first exposure, and the second trapezoid represents the second exposure. The delay is the delay between the internal signal and the VSYNC signal. This delay ensures that the exposure process starts at the correct time. The red dashed lines indicate the time when the exposure starts. The first exposure starts at the first red dashed line, and the second exposure starts at the second red dashed line. The FSYNC signal indicates the start of each frame, while the VSYNC signal indicates the start of each image row. The internal signal is generated after the FSYNC signal and triggers the start of exposure. The delay ensures that the exposure process starts at the correct time. By precisely controlling the exposure time, image brightness and quality can be guaranteed.

[0036] S130 , determining a correction time corresponding to the image sensor based on the first time, the exposure time, and the interruption triggering time to complete time synchronization; the interruption triggering time is the time when the image sensor receives the interruption detection triggering signal.

[0037] In this embodiment, after determining the first time and exposure time, a correction time corresponding to the image sensor can be determined based on the first time, exposure time, and interrupt trigger time to achieve time synchronization. The correction time can be the time obtained after the image sensor is calibrated. The interrupt trigger time can be the time when the image sensor receives an interrupt detection trigger signal. The interrupt detection trigger signal can be received via a general-purpose input / output (GPIO) pin of the SOC.

[0038] It should be noted that during the time synchronization process, the Precision Time Protocol (PTP) or the Generalized Precision Time Protocol (GPTP) can be used as the master clock for management, and time synchronization can be performed through the network, so that the error in the time transmission process is less than 0.5ms.

[0039] This embodiment provides a multi-sensor time synchronization method, including: for any image sensor, determining a first time based on a readout timing signal from the image sensor; the first time being the time when the system-on-chip (SOC) receives the first pixel; utilizing an on-board image signal processor to obtain the image exposure time in real time; and determining a corresponding correction time for the image sensor based on the first time, the exposure time, and an interrupt trigger time to achieve time synchronization; the interrupt trigger time being the time when the image sensor receives an interrupt detection trigger signal. This technical solution achieves time synchronization among the image sensors, improves sensor detection accuracy, and avoids data inconsistencies caused by time deviations when the image sensors work together.

[0040] As an optional implementation manner of this embodiment, the multi-sensor time synchronization method provided in this embodiment further includes, before determining the first time based on the readout timing signal of the image sensor:

[0041] 1) Sending each frame synchronization signal in a frame synchronization signal set to a corresponding image sensor respectively; the frame synchronization signal set includes at least two frame synchronization signals.

[0042] Specifically, the pulse-per-second signal can be converted to obtain a frame synchronization signal set, which can include at least two frame synchronization signals. Each frame synchronization signal can be sent to each image sensor. The frame synchronization signal can be a signal used to ensure that the receiving end can correctly identify the starting position of the data frame.

[0043] 2) For any image sensor, a readout timing signal is adjusted based on a frame synchronization signal received by the image sensor so that a time difference between the readout timing signal and the frame synchronization signal is less than a preset threshold.

[0044] Specifically, each image sensor can receive a frame synchronization signal. For any image sensor, the frame synchronization signal received by the image sensor can be used as a reference, and the readout timing signal of the image sensor can be adjusted based on the frame synchronization signal so that the time difference between the readout timing signal and the frame synchronization signal is less than a preset threshold. The preset threshold can be a pre-set threshold, and illustratively, the preset threshold can be + / - 1 [line].

[0045] As an optional implementation manner of this embodiment, before sending each frame synchronization signal in the frame synchronization signal set to the corresponding image sensor, the method further includes:

[0046] 1) Receiving a pulse-per-second signal; the pulse-per-second signal is generated by a laser radar.

[0047] In this embodiment, the pulse per second signal can be generated by a laser radar. The pulse per second signal can be transmitted to the SOC through a universal input / output interface. The SOC can receive the pulse per second signal. The pulse per second signal (PPS) can be a pulse signal with a period of 1 second, which is generally used to provide a precise time reference.

[0048] 2) Using the system on chip (SOC), based on a set ratio relationship, converting the pulse-per-second signal into at least two frame synchronization signals, wherein the set ratio relationship is a conversion ratio between the pulse-per-second signal and the frame synchronization signal.

[0049] Specifically, after receiving the pulse-second signal, the SOC can utilize its frequency multiplication management module to implement frequency multiplication management of the pulse-second signal. This can be understood as processing the pulse-second signal through the frequency multiplication management module within the SOC. The SOC frequency multiplication management module can convert the pulse-second signal into at least two frame synchronization signals based on a set ratio. The set ratio refers to the conversion ratio between the pulse-second signal and the frame synchronization signal. For example, the set ratio is one pulse-second signal corresponding to n frame synchronization signals.

[0050] For example, if n is 100, then a pulse-per-second signal of 1 per second can be processed to generate 100 frame synchronization signals, thereby improving the accuracy of time synchronization. At the same time, the error generated in this conversion process is close to the microsecond level.

[0051] As an optional implementation of this embodiment, the multi-sensor time synchronization method provided by this embodiment further includes:

[0052] 1) Obtain the calibration time corresponding to each image sensor.

[0053] It can be known that after the correction time of each image sensor is determined, the correction time corresponding to each image sensor can be obtained.

[0054] 2) For each correction time, determine whether the correction time and the preset standard time meet the preset difference condition; if both are satisfied, determine that the laser radar and each image sensor are time synchronized; the preset standard time is the time when the laser radar generates the second pulse signal.

[0055] In this embodiment, after obtaining each correction time, the difference between each correction time and a preset standard time can be calculated, where the preset standard time can be the time when the laser radar generates a second pulse signal. It can be determined whether the difference between the correction time and the preset standard time meets a preset difference condition, where the preset difference condition can be a pre-set difference condition. For example, the preset difference condition can be that the difference between the correction time and the preset standard time is less than 100 μs.

[0056] Specifically, if the difference between the calibration time corresponding to each image sensor and the preset standard time meets the preset difference condition, the LiDAR and each image sensor can be considered to be time synchronized. If not, it can be considered that each image sensor and the LiDAR are not time synchronized, and this technical solution can be executed again until the LiDAR and each image sensor are time synchronized.

[0057] As an optional implementation manner of this embodiment, determining the correction time corresponding to the image sensor based on the first time, the exposure time, and the interruption trigger time includes:

[0058] 1) Determine an exposure center time based on the first time and the exposure time.

[0059] Specifically, after the first time and the exposure time are determined, the exposure center time may be determined based on the first time and the exposure time. Exemplarily, the exposure center time may be the difference between the first time and the exposure time.

[0060] Exemplarily, the first time may be expressed as Tsof, the exposure time may be expressed as B, and the exposure center time may be expressed as Texp, and then: Tsof-B=Texp.

[0061] 2) determining whether the exposure center time and the interruption triggering time meet a preset condition; if so, using the interruption triggering time as the correction time of the image sensor.

[0062] It can be seen that after obtaining the exposure center time, it is possible to determine whether the exposure center time and the interrupt trigger time meet a preset condition. The preset condition can be a pre-set judgment condition. For example, the preset condition can be that the difference between the exposure center time and the interrupt trigger time is less than a preset value, where the preset value can be 1ms. For example, the exposure center time can be expressed as Texp, and the interrupt trigger time can be expressed as Tgpio. The preset condition can be expressed as: Tgpio – Texp < 1ms.

[0063] Continuing with the above description, if the exposure center time and the interruption triggering time meet the preset conditions, the interruption triggering time can be used as the correction time of the image sensor.

[0064] As an optional implementation of this embodiment, the method further includes:

[0065] If not, the process returns to the step of obtaining the exposure time of the image in real time using the onboard image signal processor, and obtains the exposure time again.

[0066] It should be explained that if the exposure center time and the interrupt trigger time meet the preset conditions, we can return to the step of obtaining the exposure time of the image in real time using the on-board image signal processor, and use the on-board image signal processor to obtain the exposure time again.

[0067] This embodiment also provides an application example of a multi-sensor time synchronization method, including:

[0068] Step 1: During the time synchronization process, PTP or GPTP is used as the master to manage the time, and the time synchronization is performed through the network with an error of less than 0.5ms.

[0069] Step 2: The sensor-lidar generates a pulse per second (pps) signal, which is transmitted to the SoC through the general-purpose input / output (GPIO) pin for frequency multiplication management. The pulse per second signal is converted into at least two frame synchronization signals (fsync) with an error close to us. The conversion relationship can be expressed as 1pps = nfsync.

[0070] Step 3: Refer to the internal signal (readout timing signal) inside the image sensor of the vehicle camera and, based on the fsync signal in step 2, accurately control the readout time of each vehicle camera, thereby adjusting the exposure time trigger of the image sensor of each camera and ignoring the serdes transmission time to obtain Tsof (first time);

[0071] Step 4: Based on the AEC calculation value of the ISP, the long and short multiple exposure mode of the large and small pixels of the next frame of the vehicle camera is counted in real time to accurately control the output image and ensure the image quality;

[0072] Step 5: Based on the characteristics of the camera, the embedded data processing is introduced to add the embedded data information for each image sensor, thereby obtaining the exposure time information of each image sensor and calculating the current exposure center time Texp;

[0073] Step 6: According to the gpio trigger interrupt detection in step 2, obtain the current time Tgpio;

[0074] Step 7: According to steps 3 to 6, select an appropriate exposure center time so that the exposure center time Texp meets the preset conditions: Tgpio – Texp < 1ms, and the accuracy reaches us, and use Tgpio as the precise trigger time (correction time). If the obtained Texp does not meet this formula, you can perform step 5 again to re-trigger the exposure, obtain a new image exposure time, and recalculate the exposure center time Texp.

[0075] Step 8: Different sensors are executed based on steps 2 to 7 to obtain the precise trigger time T1gpio to Tngpio (correction time) of each sensor, with the error guaranteed to be within 1ms.

[0076] Step 9: Compare the Tgpio time of each camera with the lidar's pps time, Tlidar, to confirm that the time difference is within 100 μs. This achieves spatiotemporal synchronization of the data. If the time difference is not within 100 μs, repeat the above solution until the sensor's precise trigger time meets the requirement of less than 100 μs.

[0077] The above technical solution avoids the problem of data inconsistency caused by time deviation when the image sensors work together, realizes the time synchronization of multiple image sensors, and improves the detection accuracy of each sensor.

[0078] Example 2

[0079] Figure 3: is a structural diagram of a multi-sensor time synchronization device provided by the second embodiment of the present disclosure; Figure 3 As shown, the device includes: a first time determination module 210, an exposure time determination module 220, and a correction time determination module 230.

[0080] The first time determination module 210 is configured to determine a first time for any image sensor based on a readout timing signal of the image sensor; the first time is the time when the system on chip (SOC) receives the first pixel;

[0081] An exposure time determination module 220 is configured to obtain the exposure time of an image in real time using an onboard image signal processor;

[0082] The correction time determination module 230 is used to determine the correction time corresponding to the image sensor based on the first time, the exposure time and the interrupt trigger time to complete time synchronization; the interrupt trigger time is the time when the image sensor receives the interrupt detection trigger signal.

[0083] A second embodiment of the present disclosure provides a multi-sensor time synchronization device, which realizes time synchronization of various image sensors and improves the detection accuracy of the sensors.

[0084] Furthermore, the correction time determination module 230 is further configured to:

[0085] determining an exposure center time based on the first time and the exposure time;

[0086] Determine whether the exposure center time and the interruption trigger time meet a preset condition; if so, use the interruption trigger time as the correction time of the image sensor.

[0087] Furthermore, the correction time determination module 230 is further configured to:

[0088] If not, return to the step of using the onboard image signal processor to re-acquire the exposure time.

[0089] Furthermore, the device further comprises:

[0090] A frame synchronization signal sending module, configured to send each frame synchronization signal in a frame synchronization signal set to a corresponding image sensor; the frame synchronization signal set includes at least two frame synchronization signals;

[0091] The adjustment module is configured to adjust a readout timing signal for any image sensor based on a frame synchronization signal received by the image sensor, so that a time difference between the readout timing signal and the frame synchronization signal is smaller than a preset threshold.

[0092] Furthermore, the device further comprises:

[0093] A receiving module, configured to receive a pulse-per-second signal generated by a laser radar;

[0094] The conversion module is used to convert the second pulse signal into at least two frame synchronization signals based on a set ratio relationship using the system on chip SOC, wherein the set ratio relationship is a conversion ratio between the second pulse signal and the frame synchronization signal.

[0095] Furthermore, the device further comprises:

[0096] A correction time acquisition module is used to obtain the correction time corresponding to each image sensor;

[0097] A determination module is used to determine, for each correction time, whether the correction time and the preset standard time meet a preset difference condition; if both are satisfied, it is determined that the laser radar and each image sensor have achieved time synchronization; the preset standard time is the time when the laser radar generates the second pulse signal.

[0098] The multi-sensor time synchronization device provided by the embodiments of the present disclosure can execute the multi-sensor time synchronization method provided by any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0099] Example 3

[0100] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are provided for example only and are not intended to limit the implementation of the embodiments of the present disclosure described and / or claimed herein.

[0101] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0102] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0103] The processor 11 may be any general-purpose and / or specialized processing component with processing and computing capabilities. Examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microprocessor, etc. The processor 11 executes the various methods and processes described above, such as the multi-sensor time synchronization method.

[0104] In some embodiments, the multi-sensor time synchronization method can be implemented as a computer program that is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the multi-sensor time synchronization method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the multi-sensor time synchronization method in any other appropriate manner (e.g., by means of firmware).

[0105] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0106] The computer programs for implementing the methods of the embodiments of the present disclosure may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0107] In the context of the embodiments of the present disclosure, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0108] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0109] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0110] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0111] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the embodiments of the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of the embodiments of the present disclosure can be achieved, and this document is not limited here.

[0112] The above specific implementations do not constitute a limitation on the scope of protection of the embodiments of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the embodiments of the present disclosure shall be included within the scope of protection of the embodiments of the present disclosure.

[0113] An embodiment of the present disclosure further provides a computer program product, including a computer program and / or instructions, which, when executed by a processor, implements the multi-sensor time synchronization method provided in any embodiment of the present application.

[0114] During implementation, the computer program product may be written in one or more programming languages ​​or a combination thereof to write computer program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] Note that the above are only preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present disclosure. Therefore, although the present disclosure is described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present disclosure, and the scope of the present disclosure is determined by the scope of the appended claims.

Claims

1. A multi-sensor time synchronization method, characterized in that: The method comprises: For any image sensor, a first time is determined based on a readout timing signal of the image sensor; the first time is the time when the system on chip (SOC) receives the first pixel; Using the on-board image signal processor to obtain the image exposure time in real time; A correction time corresponding to the image sensor is determined based on the first time, the exposure time, and an interruption trigger time to complete time synchronization; the interruption trigger time is the time when the image sensor receives an interruption detection trigger signal.

2. The method according to claim 1, characterized in that The determining the correction time corresponding to the image sensor based on the first time, the exposure time, and the interruption trigger time includes: determining an exposure center time based on the first time and the exposure time; Determine whether the exposure center time and the interruption trigger time meet a preset condition; if so, use the interruption trigger time as the correction time of the image sensor.

3. The method according to claim 2, characterized in that The method further comprises: If not, the process returns to the step of obtaining the exposure time of the image in real time using the onboard image signal processor, and obtains the exposure time again.

4. The method according to claim 1, wherein Before determining the first time based on the readout timing signal of the image sensor, the method further includes: Sending each frame synchronization signal in a frame synchronization signal set to a corresponding image sensor respectively; the frame synchronization signal set includes at least two frame synchronization signals; For any image sensor, a readout timing signal is adjusted based on a frame synchronization signal received by the image sensor, so that a time difference between the readout timing signal and the frame synchronization signal is smaller than a preset threshold.

5. The method according to claim 4, characterized in that Before sending each frame synchronization signal in the frame synchronization signal set to the corresponding image sensor, the method further includes: receiving a pulse-per-second signal generated by a laser radar; The system on chip (SOC) is used to convert the pulse-per-second signal into at least two frame synchronization signals based on a set proportional relationship, wherein the set proportional relationship is a conversion ratio between the pulse-per-second signal and the frame synchronization signal.

6. The method according to claim 5, characterized in that The method further comprises: Obtain the calibration time corresponding to each image sensor; For each correction time, determine whether the correction time and the preset standard time meet the preset difference condition; if both are satisfied, determine that the laser radar and each image sensor have achieved time synchronization; the preset standard time is the time when the laser radar generates the second pulse signal.

7. A multi-sensor time synchronization device, characterized in that: include: A first time determination module, configured to determine, for any image sensor, a first time based on a readout timing signal of the image sensor; The first time is the time when the system on chip SOC receives the first pixel; An exposure time determination module, used to obtain the exposure time of an image in real time using an on-board image signal processor; A correction time determination module is used to determine the correction time corresponding to the image sensor based on the first time, the exposure time and the interrupt trigger time to complete time synchronization; the interrupt trigger time is the time when the image sensor receives the interrupt detection trigger signal.

8. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the multi-sensor time synchronization method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the multi-sensor time synchronization method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the multi-sensor time synchronization method according to any one of claims 1 to 6.

Images