Multi-sensor synchronization control method, device, unmanned system and medium

Abstract

The application provides a multi-sensor synchronous control method and device, an unmanned system and a medium, comprising: determining a delay duration of a to-be-synchronized sensor based on coordinates of a preset reference point corresponding to the to-be-synchronized sensor and control characteristics of the to-be-synchronized sensor; determining a trigger time of the to-be-synchronized sensor based on a preset target sampling time and the delay duration; and triggering a synchronization clock signal corresponding to the to-be-synchronized sensor by using the trigger time, so that the to-be-synchronized sensor samples data of the preset reference point at the target sampling time. In this way, different trigger times can be set for each sensor, and all sensors can sample corresponding reference points at the preset target sampling time, realizing synchronous sampling of all sensors on the same object, improving the accuracy of the sensing result of the sensing system, and further improving the stability and safety of the unmanned system.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically providing a multi-sensor synchronous control method, device, autonomous driving system, and medium. Background Technology

[0002] Advanced Driving Assistance Systems (ADAS) are attracting increasing attention, and with advancements in sensors and information technology, the user experience is also being further enhanced.

[0003] In existing ADAS systems, due to the different scanning methods of various sensors, the time corresponding to the sensor sampling data is not uniform, and the corresponding position of the same object is inconsistent in different sensors. As a result, the perception system performs target identification based on the data results of each sensor, and then performs fusion verification to obtain the final perception result with low accuracy, which in turn reduces the stability and safety of the autonomous driving system. Summary of the Invention

[0004] To overcome the above-mentioned shortcomings, this invention is proposed to provide a multi-sensor synchronous control method, device, autonomous driving system, and medium that solves, or at least partially solves, the technical problem that the accuracy of the final perception result obtained by the perception system based on the data results of each sensor is low, thereby reducing the stability and safety of the autonomous driving system.

[0005] In a first aspect, the present invention provides a method for synchronous control of multiple sensors, the method comprising:

[0006] Based on the coordinates of the preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized, the delay time of the sensor to be synchronized is determined; wherein, the delay time is the duration between the trigger time of the sensor to be synchronized and the preset target sampling time of the sensor to be synchronized;

[0007] Based on the preset target sampling time and the delay duration, the trigger time of the sensor to be synchronized is determined;

[0008] Using the triggering time, the synchronization clock signal corresponding to the sensor to be synchronized is triggered based on a pre-provided clock signal source, so that the sensor to be synchronized can sample data from the preset reference point at the target sampling time.

[0009] Furthermore, in the multi-sensor synchronization control method described above, the synchronization clock signal is obtained as follows:

[0010] Based on a pre-provided clock signal source, the clock signal source is divided using a frequency divider with the same sampling frequency as the sensor to be synchronized, to obtain the synchronization clock signal; or

[0011] Based on a pre-provided clock signal source, the local clock of the sensor to be synchronized is calibrated at a fixed frequency to obtain the synchronization clock signal.

[0012] Based on the coordinates of a preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized, the delay duration of the sensor to be synchronized is determined, including:

[0013] Based on the coordinates of the preset reference point, the first delay duration of the sensor to be synchronized is determined;

[0014] Based on the control characteristics of the sensor to be synchronized, the second delay duration of the sensor to be synchronized is determined;

[0015] The sum of the first delay duration and the second delay duration is taken as the delay duration.

[0016] Furthermore, in the multi-sensor synchronization control method described above, determining the delay duration of the sensor to be synchronized based on the coordinates of a preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized includes:

[0017] Based on the coordinates of the preset reference point, the first delay duration of the sensor to be synchronized is determined;

[0018] Based on the control characteristics of the sensor to be synchronized, the second delay duration of the sensor to be synchronized is determined;

[0019] The sum of the first delay duration and the second delay duration is taken as the delay duration.

[0020] Furthermore, in the multi-sensor synchronization control method described above, determining the first delay duration of the sensor to be synchronized based on the coordinates of the preset reference point includes:

[0021] Based on the preset correlation between coordinates and delay duration, the first delay duration corresponding to the coordinates of the preset reference point is determined;

[0022] Based on the control characteristics of the sensor to be synchronized, the second delay duration of the sensor to be synchronized is determined, including:

[0023] The second delay duration is determined based on the response time and / or sampling frequency of the sensor to be synchronized.

[0024] Furthermore, in the multi-sensor synchronization control method described above, before determining the delay duration of the sensor to be synchronized based on the coordinates of a preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized, the method further includes:

[0025] In the coordinate system of the autonomous driving system in which the sensors to be synchronized are set, a common point of the sampling range of multiple sensors in at least one specified direction is selected as a preset reference point in each specified direction;

[0026] The sampling range of the sensor is determined based on the sensor's installation angle and position.

[0027] Furthermore, in the multi-sensor synchronization control method described above, before determining the delay of the triggering time of the sensor to be synchronized based on the coordinates of the preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized, the method further includes:

[0028] In the coordinate system of the sensor to be synchronized, a point at a specified location is selected as the preset reference point.

[0029] Furthermore, the aforementioned multi-sensor synchronous control method further includes:

[0030] If the preset reference point is a point at a specified location in the coordinate system of the sensor to be synchronized, the data sampled by the sensor to be synchronized is corrected according to the extrinsic parameters of the sensor to be synchronized.

[0031] Furthermore, in the aforementioned multi-sensor synchronization control method, the process of acquiring the clock signal source includes:

[0032] A high-precision temperature drift compensation crystal oscillator is used to generate a periodic waveform with a fixed frequency.

[0033] The periodic waveform is used as the clock signal source.

[0034] In a second aspect, the present invention provides a multi-sensor synchronous control device, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by the processor to perform the multi-sensor synchronous control method described in any of the preceding claims.

[0035] In a third aspect, an autonomous driving system is provided, characterized in that it includes a multi-sensor synchronous control device as described above.

[0036] In a fourth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the multi-sensor synchronous control method described in any of the preceding technical solutions.

[0037] Solution 1. A multi-sensor synchronous control method, characterized in that it includes:

[0038] Based on the coordinates of the preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized, the delay time of the sensor to be synchronized is determined; wherein, the delay time is the duration between the trigger time of the sensor to be synchronized and the preset target sampling time of the sensor to be synchronized;

[0039] Based on the preset target sampling time and the delay duration, the trigger time of the sensor to be synchronized is determined;

[0040] Using the triggering time, the synchronization clock signal corresponding to the sensor to be synchronized is triggered based on a pre-provided clock signal source, so that the sensor to be synchronized can sample data from the preset reference point at the target sampling time.

[0041] Solution 2. The multi-sensor synchronization control method according to Solution 1, characterized in that the synchronization clock signal is obtained in the following manner:

[0042] Based on a pre-provided clock signal source, the clock signal source is divided using a frequency divider with the same sampling frequency as the sensor to be synchronized, to obtain the synchronization clock signal; or

[0043] Based on a pre-provided clock signal source, the local clock of the sensor to be synchronized is calibrated at a fixed frequency to obtain the synchronization clock signal.

[0044] Solution 3. The multi-sensor synchronization control method according to Solution 1, characterized in that, based on the coordinates of a preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized, the delay time of the sensor to be synchronized is determined, including:

[0045] Based on the coordinates of the preset reference point, the first delay duration of the sensor to be synchronized is determined;

[0046] Based on the control characteristics of the sensor to be synchronized, the second delay duration of the sensor to be synchronized is determined;

[0047] The sum of the first delay duration and the second delay duration is taken as the delay duration.

[0048] Solution 4. The multi-sensor synchronization control method according to Solution 3, characterized in that, based on the coordinates of the preset reference point, determining the first delay duration of the sensor to be synchronized includes:

[0049] Based on the preset correlation between coordinates and delay duration, the first delay duration corresponding to the coordinates of the preset reference point is determined;

[0050] Based on the control characteristics of the sensor to be synchronized, the second delay duration of the sensor to be synchronized is determined, including:

[0051] The second delay duration is determined based on the response time and / or sampling frequency of the sensor to be synchronized.

[0052] Solution 5. The multi-sensor synchronization control method according to Solution 1, characterized in that, before determining the delay time of the sensor to be synchronized based on the coordinates of the preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized, it further includes:

[0053] In the coordinate system of the autonomous driving system in which the sensors to be synchronized are set, a common point of the sampling range of multiple sensors in at least one specified direction is selected as a preset reference point in each specified direction;

[0054] The sampling range of the sensor is determined based on the sensor's installation angle and position.

[0055] Solution 6. The multi-sensor synchronization control method according to Solution 1, characterized in that, before determining the delay time of the triggering time of the sensor to be synchronized based on the coordinates of the preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized, it further includes:

[0056] In the coordinate system of the sensor to be synchronized, a point at a specified location is selected as the preset reference point.

[0057] Solution 7. The multi-sensor synchronous control method according to Solution 6, characterized in that it further includes:

[0058] If the preset reference point is a point at a specified location in the coordinate system of the sensor to be synchronized, the data sampled by the sensor to be synchronized is corrected according to the extrinsic parameters of the sensor to be synchronized.

[0059] Solution 8. The multi-sensor synchronization control method according to Solution 1, characterized in that the process of acquiring the clock signal source includes:

[0060] A high-precision temperature drift compensation crystal oscillator is used to generate a periodic waveform with a fixed frequency.

[0061] The periodic waveform is used as the clock signal source.

[0062] Scheme 9. A multi-sensor synchronous control device, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by the processor to perform the multi-sensor synchronous control method as described in any one of Schemes 1 to 8.

[0063] Solution 10. An unmanned driving system, characterized in that it includes a multi-sensor synchronous control device as described in Solution 9.

[0064] Scheme 11. A computer-readable storage medium storing a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by a processor to perform the multi-sensor synchronous control method described in any one of Schemes 1 to 8.

[0065] The above-described technical solutions of the present invention have at least one or more of the following beneficial effects:

[0066] In implementing the technical solution of the present invention, the delay duration of the sensor to be synchronized is determined based on the coordinates of the preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized; the triggering time of the sensor to be synchronized is determined based on the preset target sampling time and the delay duration; the synchronization clock signal corresponding to the sensor to be synchronized is triggered using the triggering time, so that the sensor to be synchronized samples data from the preset reference point at the preset target sampling time. In this way, different triggering times can be set for each sensor, and all sensors can sample the corresponding reference point at the preset target sampling time, realizing synchronous sampling of the same object by all sensors, improving the accuracy of the perception results of the perception system, and thus improving the stability and safety of the autonomous driving system. Attached Figure Description

[0067] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Furthermore, similar numbers in the drawings are used to denote similar components, wherein:

[0068] Figure 1 This is a schematic flowchart of the main steps of a control method for a smart home device according to an embodiment of the present invention;

[0069] Figure 2 This is a main structural block diagram of a multi-sensor synchronous control system according to an embodiment of the present invention;

[0070] Figure 3It is a specific timing control diagram for sampling by two different cameras;

[0071] Figure 4 This is the timing control diagram for millimeter-wave radar;

[0072] Figure 5 This is a timing diagram of the multi-sensor synchronous control of the present invention;

[0073] Figure 6 This is a main structural block diagram of a multi-sensor synchronous control device according to an embodiment of the present invention. Detailed Implementation

[0074] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0075] In the description of this invention, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, memory, and may also include software components, such as program code, or a combination of software and hardware. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Non-transitory computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular terms "a" or "this" can also include plural forms.

[0076] In existing ADAS systems, due to the different scanning methods of various sensors, the time corresponding to the sensor sampling data is not uniform, and the corresponding position of the same object is inconsistent in different sensors. As a result, the perception system performs target identification based on the data results of each sensor, and then performs fusion verification to obtain the final perception result with low accuracy, which in turn reduces the stability and safety of the autonomous driving system.

[0077] Therefore, in order to solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0078] See appendix Figure 1 , Figure 1This is a schematic flowchart illustrating the main steps of a control method for a smart home device according to an embodiment of the present invention. Figure 1 As shown, the multi-sensor synchronous control method in this embodiment of the invention mainly includes the following steps 101-103.

[0079] Step 101: Determine the delay duration of the sensor to be synchronized based on the coordinates of the preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized.

[0080] In a specific implementation, the delay time of the sensor to be synchronized is the time between the trigger time of the sensor to be synchronized and the preset target sampling time of the sensor to be synchronized. That is, after the ADAS issues a trigger signal, the sensor to be synchronized can sample at the preset target sampling time after this delay time.

[0081] In a specific implementation process, ADAS has multiple sensors. A corresponding reference point can be set for each sensor according to actual needs, so that each sensor can sample data at the preset reference point.

[0082] In a specific implementation process, the preset reference point can be set in the following two ways:

[0083] The first method involves selecting a common point in the coordinate system of the autonomous driving system where the sensors to be synchronized are located, along with the common points of the sampling ranges of multiple sensors in at least one specified direction, as a preset reference point for each specified direction. The sampling range of the sensors can include the sensor's coverage area and / or its line-of-sight range. In other words, a preset reference point can be selected for a specified direction, where multiple sensors can all scan the area. This can include four specified directions (front, rear, left, and right) around the vehicle to scan a 360° area around the vehicle.

[0084] In a specific implementation, the sampling range of the sensor can be determined based on the installation angle and position of the sensor. In this way, when the installation angle and position of the sensor change due to vehicle vibration or other factors, the sampling range of the sensor may change, thereby updating the preset reference point, obtaining the coordinates of the updated preset reference point, and obtaining a more accurate delay time of the sensor to be synchronized, so as to reduce the error in synchronization accuracy.

[0085] The second method involves selecting a point at a specified location as the preset reference point within the coordinate system of the sensor to be synchronized. This method eliminates the influence of sensor extrinsic parameters, and allows for the selection of approximate reference points for sensors of the same type, thus saving resources. However, during the final synchronization process, when transforming to the coordinate system of the autonomous driving system, there is a problem of inaccurate spatial alignment of synchronization points due to differences in sensor positions. In subsequent perception fusion, sensor data is corrected and processed based on extrinsic parameters.

[0086] In a specific implementation process, the coordinates of a preset reference point corresponding to the sensor to be synchronized that needs to be controlled can be obtained, and then the first delay duration of the sensor to be synchronized can be determined using the coordinates of the preset reference point.

[0087] Specifically, the delay duration varies depending on the coordinates of the preset reference point corresponding to the sensor to be synchronized. Therefore, a correlation between coordinates and delay duration can be pre-set for the sensor to be synchronized based on theoretical calculations. After obtaining the coordinates of the preset reference point, the corresponding delay duration can be obtained as the first delay duration for the sensor to be synchronized through this correlation. If the preset reference point is set using the first method described above, the converted coordinates of the preset reference point within the line-of-sight range in each direction can be obtained for each preset reference point and the sensor's extrinsic parameters.

[0088] In a specific implementation, the second delay duration of the sensor to be synchronized can be determined based on the control characteristics of the sensor to be synchronized, and then the sum of the first delay duration and the second delay duration can be used as the delay duration.

[0089] Specifically, different sensors have different control characteristics such as response time and / or sampling frequency, and therefore receive control signals from the ADAS controller at different times. Thus, the second delay duration can be determined based on the response time and / or sampling frequency of the sensor to be synchronized. This response time is the time period corresponding to the sensor's control closed-loop circuit.

[0090] Step 102: Determine the trigger time of the sensor to be synchronized based on the preset target sampling time and the delay duration;

[0091] In a specific implementation process, a time for simultaneous data acquisition by all sensors can be set as the preset target sampling time. In this way, after obtaining the delay time of the sensor to be synchronized, the delay time is pushed forward based on the preset target sampling time to obtain the corresponding time as the trigger time of the sensor to be synchronized, that is, the time when the ADAS controller issues the control signal.

[0092] Step 103: Using the trigger time, trigger the synchronization clock signal corresponding to the sensor to be synchronized, so that the sensor to be synchronized can sample data from the preset reference point at the target sampling time.

[0093] In a specific implementation, the ADAS controller can trigger the synchronization clock signal corresponding to the sensor to be synchronized based on the trigger time of the sensor to be synchronized, thereby sending a control signal to the sensor to be synchronized. After a delay, the sensor to be synchronized receives the control signal at a preset target sampling time, and then the sensor to be synchronized samples data from the preset reference point.

[0094] In one specific implementation, the synchronization clock signal is obtained as follows:

[0095] The synchronization clock signal can be obtained by dividing the pre-provided clock signal source using a frequency divider with the same sampling frequency as the sensor to be synchronized. Alternatively, the synchronization clock signal can be obtained by using the Generalized Precision Time Protocol (gPTP) time synchronization mechanism of Ethernet to correct the local clock of the sensor to be synchronized at a fixed frequency, based on the pre-provided clock signal source. This ensures that the time axes of all sensors are consistent.

[0096] Specifically, a crystal oscillator can be used as a clock signal source, or other devices or circuits capable of generating clock signals can be used. The crystal oscillator employs a high-precision temperature drift compensation method, whereby an additional temperature compensation circuit within the crystal reduces the amount of frequency variation caused by changes in ambient temperature.

[0097] Specifically, a high-precision temperature drift compensation crystal oscillator can be used to generate a periodic waveform with a fixed frequency; this periodic waveform is used as the clock signal source. The periodic waveform with a fixed frequency can include square waves, triangle waves, sine waves, etc., and this embodiment does not impose specific limitations.

[0098] The multi-sensor synchronization control method of this embodiment determines the delay duration of the sensor to be synchronized based on the coordinates of a preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized; it determines the triggering time of the sensor to be synchronized based on a preset target sampling time and the delay duration; and it uses the triggering time to trigger the synchronization clock signal corresponding to the sensor to be synchronized, so that the sensor to be synchronized can sample data from the preset reference point at the preset target sampling time. In this way, different triggering times can be set for each sensor, and all sensors can sample the corresponding reference point at the preset target sampling time, realizing synchronous sampling of the same object by all sensors, improving the accuracy of the perception results of the perception system, and thus improving the stability and safety of the autonomous driving system.

[0099] The technical solution of the present invention will be described in detail below with specific examples:

[0100] See Figure 2 , Figure 2 This is a main structural block diagram of a multi-sensor synchronous control system according to an embodiment of the present invention. Figure 2 As shown, the multi-sensor synchronous control system of this embodiment may include a vehicle time system 20, ADAS 21, phase control module 22, camera 23, lidar 24, millimeter-wave radar 25, inertial measurement unit 26, ultrasonic sensor 27, control chip 28, etc.

[0101] For camera 23, the trigger time of camera 23 is the trigger time of camera 23 and the delay value used to configure the sensor register. Camera 23 and ADAS 21 are connected and transmitted via hard wire. The response time of camera 23 can be ignored. The first delay time is calculated by back-calculating based on the coordinates of the reference point corresponding to camera 23, thereby obtaining the trigger time of camera 23.

[0102] It should be noted that when the phase control module 22 outputs instructions, it has already performed verification and calculation based on information such as link transmission delay and exposure delay. The camera 23 does not need to perform additional calculations and compensations. It can directly configure and write the settings into the register and then take effect synchronously. Figure 3 This is a specific timing control diagram for sampling from two different cameras. For example... Figure 3 As shown, the two parallelograms represent the scan results from the two cameras respectively, and their centers are always aligned, indicating that both cameras 23 are acquiring data from the preset reference point at the same time. See lines x and y.

[0103] For LiDAR 24, its trigger time is the scanning time corresponding to the reference point. LiDAR 24 and ADAS 21 transmit data via Ethernet. This solution relies on the scanning method of LiDAR 24. It requires obtaining the correlation between coordinates and delay time based on the scanning method of LiDAR 24, deducing the first delay time from the coordinates of the reference point corresponding to LiDAR 24, and simultaneously combining the closed-loop characteristics of LiDAR's motor control (response time and control frequency, etc.) to give the second delay time of LiDAR 24, thus obtaining the trigger time of LiDAR 24.

[0104] For millimeter-wave radar 25, the trigger time is the midpoint of the electromagnetic wave transmission duration. Millimeter-wave radar 25 transmits data with ADAS 21 via an Ethernet link. The trigger time obtained by ADAS 25 is similar to that obtained by lidar 24; please refer to the relevant records above for details, which will not be repeated here. Synchronization can be achieved by adjusting the electromagnetic wave transmission time and duration to control the midpoint. Figure 4 This is the timing control diagram for millimeter-wave radar. (For example...) Figure 4 As shown, (a) is the timing sequence of unsynchronized control, (b) is the timing sequence of early triggering, and (c) is the timing sequence of delayed triggering.

[0105] For the inertial measurement unit 26, its trigger period reaches 200Hz, which is an order of magnitude higher than the frequency of other sensors. Frequency correction is performed using a PPS signal provided by the global clock; however, active synchronization is not triggered. Theoretically, the maximum deviation between the data from the inertial measurement unit 26 and the sampling synchronization point of other sensors under the global clock is 5ms. This can be compensated at the algorithm level based on the timestamp of the inertial measurement unit 26 to achieve sensor fusion. The inertial measurement unit 26 and ADAS 21 are also connected via hardwired transmission. The response time of the inertial measurement unit 26 is negligible, and the first delay time is calculated by back-calculating the coordinates of the reference point corresponding to the inertial measurement unit 26, thus obtaining the trigger time of the inertial measurement unit 26.

[0106] It should be noted that the response time of the inertial measurement unit 26 can also be taken as the second delay time. Then, by combining the coordinates of the reference point corresponding to the inertial measurement unit 26 to deduce the first delay time, the trigger time of the inertial measurement unit 26 can be obtained.

[0107] For the ultrasonic sensor 27, the time axis can be synchronized with ADAS 21 based on gPTP by the control chip 28, and the triggering process of the driving layer can be completed by receiving the trigger time information via Ethernet. Alternatively, the triggering process of the driving layer can be completed by controlling the transmission command through the control chip 28.

[0108] It should be noted that for the lidar 24, millimeter-wave radar 25, and ultrasonic sensor 27, since they transmit data with ADAS 21 via Ethernet, they are greatly affected by their respective control characteristics. Therefore, the phase control module 22 can also correct the triggering time of each sensor in the next cycle based on the sampling time of each sensor in the previous cycle, so as to further ensure the synchronous sampling accuracy with other sensors.

[0109] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.

[0110] Figure 5 This is a timing diagram of the multi-sensor synchronous control of the present invention, as shown below. Figure 5 As shown, for the first camera A1, the second camera A2, the first millimeter-wave radar B1, the first millimeter-wave radar B2, and the lidar, they can achieve synchronous sampling of time and space under the unified time axis corresponding to the pre-provided clock signal source Fsync.

[0111] It should be noted that the sampling frequencies of the first millimeter-wave radar B1, the first millimeter-wave radar B2, and the lidar are different from those of the first camera A1 and the second camera A2. Therefore, at a specific moment, the first camera A1, the second camera A2, the first millimeter-wave radar B1, the first millimeter-wave radar B2, and the lidar can achieve synchronous sampling. However, the first camera A1 and the second camera A2 have the same sampling frequency and can perform synchronous sampling at every moment.

[0112] Those skilled in the art will understand that all or part of the processes in the method of the above embodiment of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0113] Furthermore, the present invention also provides a multi-sensor synchronous control device.

[0114] See appendix Figure 6 , Figure 6 This is a main structural block diagram of a multi-sensor synchronous control device according to an embodiment of the present invention. Figure 6 As shown, the multi-sensor synchronous control device in this embodiment of the invention includes a processor 60 and a storage device 61.

[0115] The storage device 61 can be configured to store a program for executing the multi-sensor synchronization control method of the above-described method embodiments, and the processor 60 can be configured to execute the program in the storage device 61, which includes, but is not limited to, a program for executing the multi-sensor synchronization control method of the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The multi-sensor synchronization control device can be a control device comprising various electronic devices.

[0116] Furthermore, the present invention also provides an autonomous driving system, which includes the multi-sensor synchronous control device described in the above embodiments.

[0117] Furthermore, the present invention also provides a computer-readable storage medium. In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program for executing the multi-sensor synchronization control method of the above-described method embodiments. This program can be loaded and run by a processor to implement the above-described multi-sensor synchronization control method. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The computer-readable storage medium can be a storage device comprising various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0118] Furthermore, it should be understood that since the various modules are only provided to illustrate the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the processor's software, hardware, or a combination of software and hardware. Therefore, the number of modules shown in the figures is merely illustrative.

[0119] Those skilled in the art will understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principles of the present invention; therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.

[0120] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for synchronous control of multiple sensors, characterized in that, include: In the coordinate system of the autonomous driving system, a common point of the sampling range of multiple sensors to be synchronized in at least one specified direction is selected as a preset reference point in each specified direction, and the sensors to be synchronized are set in the autonomous driving system. Based on the coordinates of a preset reference point corresponding to the sensor to be synchronized and the control characteristics of the sensor to be synchronized, the delay duration of the sensor to be synchronized is determined, including: determining a first delay duration corresponding to the coordinates of the preset reference point based on a preset correlation between coordinates and delay duration; determining a second delay duration based on the response duration and / or sampling frequency of the sensor to be synchronized; and using the sum of the first delay duration and the second delay duration as the delay duration; wherein, the delay duration is the duration between the triggering time of the sensor to be synchronized and the preset target sampling time of the sensor to be synchronized; Based on the preset target sampling time and the delay duration, the trigger time of the sensor to be synchronized is determined; Using the triggering time, the synchronization clock signal corresponding to the sensor to be synchronized is triggered based on a pre-provided clock signal source, so that the sensor to be synchronized can sample data from the preset reference point at the target sampling time.

2. The multi-sensor synchronous control method according to claim 1, characterized in that, The synchronization clock signal is obtained in the following manner: Based on a pre-provided clock signal source, the clock signal source is divided using a frequency divider that has the same sampling frequency as the sensor to be synchronized, to obtain the synchronization clock signal; or Based on a pre-provided clock signal source, the local clock of the sensor to be synchronized is calibrated at a fixed frequency to obtain the synchronization clock signal.

3. The multi-sensor synchronous control method according to claim 1, characterized in that, The sampling range of the sensor is determined based on the sensor's installation angle and position.

4. The multi-sensor synchronous control method according to claim 1, characterized in that, The process of acquiring the clock signal source includes: A high-precision temperature drift compensation crystal oscillator is used to generate a periodic waveform with a fixed frequency. The periodic waveform is used as the clock signal source.

5. A multi-sensor synchronous control device, comprising a processor and a storage device, wherein the storage device is adapted to store multiple lines of program code, characterized in that, The program code is adapted to be loaded and run by the processor to perform the multi-sensor synchronous control method according to any one of claims 1 to 4.

6. An unmanned driving system, characterized in that, This includes the multi-sensor synchronous control device as described in claim 5.

7. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the multi-sensor synchronous control method according to any one of claims 1 to 4.

Images