Sensor synchronization equipment, method and device, controller and storage medium

Abstract

The embodiment of the invention discloses sensor synchronization equipment, method and device, a controller and a storage medium. The device is applied to an automatic driving vehicle and at least comprises a controller, a plurality of laser sensors and a plurality of cameras, the controller is in time synchronization with the laser sensors and the cameras, and the preset scanning time of each laser sensor corresponds to at least one camera; the controller is used for determining the next scanning moment of each laser sensor; determining each target camera corresponding to the next scanning moment according to the corresponding relationship between each preset scanning moment of each laser sensor and each camera; for each target camera, determining the delay duration of the target camera, and sending an exposure instruction to the target camera at the delay duration before the next scanning moment; and each target camera is used for receiving the exposure instruction sent by the controller and executing the exposure operation when receiving the exposure instruction. According to the embodiment of the invention, space-time synchronization of each camera and each laser sensor can be realized.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and more specifically, to a sensor synchronization device, method, apparatus, controller and storage medium. Background Art

[0002] In autonomous driving technology, multiple sensors are usually installed in autonomous driving vehicles, such as laser sensors, visual sensors (cameras), etc., to comprehensively perceive the data collected by multiple sensors, and then plan the driving path of the autonomous driving vehicle based on the perception results.

[0003] However, in known technologies, the laser sensors installed in autonomous vehicles usually rotate and scan continuously in a certain period, and each laser sensor is in the same phase at the same time, and each camera also takes pictures continuously at a set frequency; and the scanning frequency of each laser sensor and the photo taking frequency of each camera are not exactly the same. In other words, the moment when each laser sensor scans and obtains point cloud data may be different from the moment when the camera takes the picture. Since the autonomous vehicle is always in motion, the difference between the scanning moment of the laser sensor and the moment when the camera takes the picture may result in the final point cloud data and photo data being inconsistent in time and space, which will affect the accuracy of the perception results. Therefore, how to synchronize the time and space of multiple sensors in an autonomous vehicle has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a sensor synchronization device, method, apparatus, controller and storage medium to perform spatiotemporal synchronization of multiple sensors in an autonomous driving vehicle. The specific technical solution is as follows.

[0005] In a first aspect, an embodiment of the present application provides a sensor synchronization device, which is applied to an autonomous driving vehicle, and the device at least includes: a controller, a plurality of laser sensors, and a plurality of cameras, wherein the controller is time-synchronized with each of the laser sensors and each of the cameras, and each preset scanning moment of each of the laser sensors corresponds to at least one camera;

[0006] The controller is used to determine the next scanning moment of each of the laser sensors; determine each target camera corresponding to the next scanning moment according to the corresponding relationship between each preset scanning moment of each of the laser sensors and each camera; determine the delay time of each target camera, and send an exposure instruction to the target camera at the delay time before the next scanning moment;

[0007] Each of the target cameras is used to receive an exposure instruction sent by the controller, and perform an exposure operation upon receiving the exposure instruction.

[0008] From the above content, it can be seen that the sensor synchronization device provided in the embodiment of the present application can realize time synchronization for the controller, each camera, and each laser sensor, and can pre-build the correspondence between the laser sensor scanning time and each camera. Therefore, when the laser sensor scans the point cloud, the controller can control the camera corresponding to the laser sensor scanning area to expose, thereby ensuring that the point cloud data scanned by the laser sensor and the image captured by the camera are synchronized in time and space.

[0009] Optionally, the controller is specifically used to obtain a preset signal delay duration; for each target camera, determine the exposure delay duration of the target camera, and determine the sum of the signal delay duration and the exposure delay duration as the delay duration of the target camera.

[0010] Optionally, the controller is specifically used to determine, for each target camera, the sending time of the last exposure instruction sent to the target camera, and the shooting time when the image returned by the target camera is received; and determine the difference between the shooting time and the sending time as the exposure delay duration of the target camera.

[0011] In this embodiment, the exposure delay duration corresponding to each target camera can be determined in real time, so that the spatiotemporal synchronization of each camera and each laser sensor can be achieved more accurately.

[0012] Optionally, the controller is specifically configured to obtain a preset exposure delay duration of each camera as the exposure delay duration of each target camera.

[0013] Optionally, the multiple laser sensors include: a forward laser sensor, a left laser sensor, a rearward laser sensor, and a right laser sensor; the multiple cameras include: a front left camera, a front right camera, a left camera, a rear left camera, a rear right camera, and a right camera; each of the laser sensors points to the front of the autonomous driving vehicle at 0 milliseconds, and the time required to rotate 360 ​​degrees clockwise is 100 milliseconds;

[0014] The correspondence between the preset scanning moments of each laser sensor and each camera includes: 0 milliseconds corresponds to the front left camera and the front right camera; 25 milliseconds corresponds to the left camera; 50 milliseconds corresponds to the rear left camera and the rear right camera; 75 milliseconds corresponds to the right camera.

[0015] Optionally, the device further includes: a synchronous clock source; the synchronous clock source is connected to the controller and each of the laser sensors.

[0016] In a second aspect, an embodiment of the present application provides a sensor synchronization method, which is applied to a controller of an autonomous driving vehicle, wherein a plurality of laser sensors and a plurality of cameras are also installed in the autonomous driving vehicle, wherein the controller is time-synchronized with each of the laser sensors and each of the cameras, and each preset scanning moment of each of the laser sensors corresponds to at least one camera, and the method comprises:

[0017] Determining the next scanning time of each of the laser sensors;

[0018] Determine the target cameras corresponding to the next scanning moment according to the correspondence between the preset scanning moments of each laser sensor and the cameras;

[0019] For each target camera, the delay time of the target camera is determined, and an exposure instruction is sent to the target camera at the delay time before the next scanning moment.

[0020] Optionally, the step of determining the delay duration of the target camera includes:

[0021] Get the preset signal delay time;

[0022] Determine the exposure delay duration of the target camera;

[0023] The sum of the signal delay time and the exposure delay time is determined as the delay time of the target camera.

[0024] Optionally, the step of determining the exposure delay duration of the target camera includes:

[0025] Determine the sending time of the last exposure instruction sent to the target camera and the shooting time of the image returned by the target camera;

[0026] The difference between the shooting time and the sending time is determined as the exposure delay duration of the target camera.

[0027] Optionally, the step of determining the exposure delay duration of the target camera includes:

[0028] The preset exposure delay duration of each camera is obtained as the exposure delay duration of the target camera.

[0029] Optionally, the multiple laser sensors include: a forward laser sensor, a left laser sensor, a rearward laser sensor, and a right laser sensor; the multiple cameras include: a front left camera, a front right camera, a left camera, a rear left camera, a rear right camera, and a right camera; each of the laser sensors points to the front of the autonomous driving vehicle at 0 milliseconds, and the time required to rotate 360 ​​degrees clockwise is 100 milliseconds;

[0030] The correspondence between the preset scanning moments of each laser sensor and each camera includes: 0 milliseconds corresponds to the front left camera and the front right camera; 25 milliseconds corresponds to the left camera; 50 milliseconds corresponds to the rear left camera and the rear right camera; 75 milliseconds corresponds to the right camera.

[0031] In a third aspect, an embodiment of the present application provides a sensor synchronization device, which is applied to a controller of an autonomous driving vehicle, wherein a plurality of laser sensors and a plurality of cameras are installed in the autonomous driving vehicle, wherein the controller is time-synchronized with each of the laser sensors and each of the cameras, and each preset scanning moment of each of the laser sensors corresponds to at least one camera, and the device comprises:

[0032] A scanning time determination module, used to determine the next scanning time of each of the laser sensors;

[0033] A target camera determination module, used to determine each target camera corresponding to the next scanning moment according to the corresponding relationship between each preset scanning moment of each laser sensor and each camera;

[0034] The camera control module is used to determine the delay time of each target camera and send an exposure instruction to the target camera at the delay time before the next scanning moment.

[0035] Optionally, the camera control module includes:

[0036] The signal delay duration acquisition submodule is used to obtain the preset signal delay duration;

[0037] An exposure delay duration determination submodule is used to determine the exposure delay duration of the target camera;

[0038] The delay duration determination submodule is used to determine the sum of the signal delay duration and the exposure delay duration as the delay duration of the target camera.

[0039] Optionally, the exposure delay duration determination submodule is specifically used to:

[0040] Determine the sending time of the last exposure instruction sent to the target camera and the shooting time of the image returned by the target camera;

[0041] The difference between the shooting time and the sending time is determined as the exposure delay duration of the target camera.

[0042] Optionally, the exposure delay duration determination submodule is specifically used to:

[0043] The preset exposure delay duration of each camera is obtained as the exposure delay duration of the target camera.

[0044] Optionally, the multiple laser sensors include: a forward laser sensor, a left laser sensor, a rearward laser sensor, and a right laser sensor; the multiple cameras include: a front left camera, a front right camera, a left camera, a rear left camera, a rear right camera, and a right camera; each of the laser sensors points to the front of the autonomous driving vehicle at 0 milliseconds, and the time required to rotate 360 ​​degrees clockwise is 100 milliseconds;

[0045] The correspondence between the preset scanning moments of each laser sensor and each camera includes: 0 milliseconds corresponds to the front left camera and the front right camera; 25 milliseconds corresponds to the left camera; 50 milliseconds corresponds to the rear left camera and the rear right camera; 75 milliseconds corresponds to the right camera.

[0046] In a fourth aspect, an embodiment of the present application provides a controller comprising at least one processor, wherein the processor is coupled to a memory, wherein the memory stores a program or instruction running on the processor, and when the program or instruction is executed by the processor, the sensor synchronization method as described in the second aspect above is implemented.

[0047] In a fifth aspect, an embodiment of the present application provides a readable storage medium having a program or instruction stored thereon, wherein the program or instruction, when executed by a processor, implements the sensor synchronization method as described in the second aspect above.

[0048] In a sixth aspect, the present application provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the sensor synchronization method as described in the second aspect above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work.

[0050] Figure 1 A schematic diagram of the structure of a sensor synchronization device provided in an embodiment of the present application;

[0051] Figure 2 This is a schematic diagram of the orientation of the laser sensor at different times in the embodiment of the present application;

[0052] Figure 3 A schematic diagram of the relationship between the controller triggering camera time and the actual scanning time of the laser sensor in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of a flow chart of a sensor synchronization method provided in an embodiment of the present application;

[0054] Figure 5 A schematic diagram of the structure of a sensor synchronization device provided in an embodiment of the present application;

[0055] Figure 6 A schematic diagram of the structure of a controller provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0058] Figure 1 A structural schematic diagram of a sensor synchronization device provided in an embodiment of the present application, which can be applied to autonomous driving vehicles to ensure the spatiotemporal synchronization of various sensors during the driving of the autonomous driving vehicle, thereby improving the accuracy of the perception results when perceiving the data collected by each sensor.

[0059] like Figure 1As shown, the synchronization device may include a controller, multiple laser sensors and multiple cameras. For example, the figure shows a situation including 4 laser sensors and 6 cameras. Among them, the circle represents the laser sensor, the triangle represents the camera, and the line in the triangle represents the camera direction. The direction of each camera is fixed. Taking the driving direction of the autonomous driving vehicle as the front, each laser sensor can be named: front laser sensor (front-lidar), left laser sensor (left-lidar), rear laser sensor (rear-lidar) and right laser sensor (right-lidar); each camera can be named: front left camera (front-left-camera), front right camera (front-right-camera), left camera (left-camera), rear left camera (rear-left-camera), rear right camera (rear-right-camera) and right camera (right-camera).

[0060] In addition, a connection is established between the controller and each camera, as shown by the dotted line in the figure, for signal transmission. For example, the controller can control the camera exposure time, etc. Specifically, the controller can send a signal to each camera based on the camera trigger module to control the exposure time of each camera. Time synchronization needs to be achieved between the controller and each laser sensor. For example, a connection can be established between the controller and each laser sensor so that the controller can synchronize the time of each laser sensor.

[0061] In one implementation, the synchronization device may further include a synchronization clock source, which is connected to the controller and each laser sensor, as shown by the solid line between the synchronization clock source and the controller, and the dotted line between the synchronization clock source and each laser sensor. For example, the synchronization clock source and each laser sensor may be connected via Ethernet.

[0062] Specifically, the synchronous clock source can perform GPS (Global Positioning System) timing for the controller and each laser sensor, that is, the synchronous clock source can synchronize the time obtained based on the global positioning system to the controller and each laser sensor, thereby achieving time synchronization between the controller and each laser sensor. In addition, the controller controls the exposure of each camera based on its own time, thereby achieving time synchronization between each camera and each laser sensor.

[0063] The initial phase of each laser sensor, that is, at 0 milliseconds, points directly in front of the autonomous vehicle, and the time required to rotate 360 ​​degrees clockwise is 100 milliseconds. In other words, after 100 milliseconds, each laser sensor completes the collection of a circle of point cloud data. Specifically, Figure 2 As shown, based on the driver's position in the autonomous driving vehicle, at 0 milliseconds, each laser sensor points to the front of the vehicle, at 25 milliseconds, each laser sensor points to the left side of the vehicle, at 50 milliseconds, each laser sensor points to the rear of the vehicle, at 75 milliseconds, each laser sensor points to the right side of the vehicle, and at 100 milliseconds, each laser sensor points to the front of the vehicle, returning to the initial phase.

[0064] It can be seen that the scanning angles of each laser sensor are different at different times. In the embodiment of the present application, in order to achieve the spatiotemporal synchronization of each laser sensor and each camera, that is, when each laser sensor scans a certain area at a certain moment, the camera also happens to capture the image of the area, the corresponding relationship between each preset scanning time of each laser sensor and each camera can be pre-set, for example, it can be set as follows: 0 milliseconds corresponds to the front left camera and the front right camera; 25 milliseconds corresponds to the left camera; 50 milliseconds corresponds to the rear left camera and the rear right camera; 75 milliseconds corresponds to the right camera.

[0065] That is to say, at 0 milliseconds, when each laser sensor points forward and scans the area in front of the vehicle, the front left camera and the front right camera can be controlled to be exposed at the same time to capture the image in front of the vehicle; at 25 milliseconds, when each laser sensor points to the left side of the vehicle and scans the area on the left side of the vehicle, the left camera can be controlled to be exposed at the same time to capture the image on the left side of the vehicle; at 50 milliseconds, when each laser sensor points backward and scans the area behind the vehicle, the rear left camera and the rear right camera can be controlled to be exposed at the same time to capture the image on the rear of the vehicle; at 75 milliseconds, when each laser sensor points to the right side of the vehicle and scans the area on the right side of the vehicle, the right camera can be controlled to be exposed at the same time to capture the image on the right side of the vehicle. In this way, in each 100 millisecond cycle, a circle of point cloud data collected by each laser sensor and an image taken by each camera can be obtained, for a total of 6 images. Moreover, the exposure time and exposure area of ​​each image are consistent with the scanning time and area of ​​each laser sensor.

[0066] Furthermore, when the controller controls the exposure time of the camera, the delay of the camera needs to be considered. Specifically, the time from when the controller sends a signal to when the camera generates an image may include: signal delay and camera exposure delay. The signal delay is usually a fixed value, and the controller may also pre-set a certain value as the camera exposure delay.

[0067] Therefore, in an embodiment of the present application, during the actual driving process of the autonomous driving vehicle, the controller can determine the next scanning moment of each laser sensor; determine the target cameras corresponding to the next scanning moment according to the corresponding relationship between the preset scanning moments of each laser sensor and each camera; for each target camera, determine the delay time of the target camera, and send an exposure instruction to the target camera at the delay time before the next scanning moment; each target camera is used to receive the exposure instruction sent by the controller, and perform an exposure operation when receiving the exposure instruction.

[0068] For example, the controller can determine the current moment and use the next exposure time point closest to the current moment as the next scanning moment among each exposure time point, namely 25 milliseconds, 50 milliseconds, 75 milliseconds, and 100 milliseconds (0 milliseconds). For example, when the current moment is 10 milliseconds, the next scanning moment can be determined to be 25 milliseconds; when the current moment is 30 milliseconds, the next scanning moment can be determined to be 50 milliseconds.

[0069] Figure 3 The diagram shows the relationship between the controller triggering camera time and the actual scanning time of the laser sensor. Figure 3 As shown, taking 0 milliseconds as an example, when the laser sensor's next scanning time is 0 milliseconds, the controller can issue an exposure command at the delay time before 0 milliseconds, and the time from the exposure command to the two cameras generating an image is the delay time, that is, the two cameras generate an image at 0 milliseconds. The laser sensor also scans and obtains point cloud data at 0 milliseconds, thereby achieving spatiotemporal synchronization between the laser sensor and the camera.

[0070] From the above content, it can be seen that the sensor synchronization device provided in the embodiment of the present application can realize time synchronization for the controller, each camera, and each laser sensor, and can pre-build the correspondence between the laser sensor scanning time and each camera. Therefore, when the laser sensor scans the point cloud, the controller can control the camera corresponding to the laser sensor scanning area to expose, thereby ensuring that the point cloud data scanned by the laser sensor and the image captured by the camera are synchronized in time and space.

[0071] It is understandable that for different cameras, due to differences in structure, parameters, etc., the corresponding exposure delay duration may not be exactly the same. Alternatively, due to factors such as light intensity, the exposure delay of the same camera at different times may also vary to a certain extent.

[0072] As an implementation method of an embodiment of the present application, in order to ensure accurate spatiotemporal synchronization of each laser sensor and each camera, the controller can first determine the exposure delay duration of each target camera before sending an exposure command to it, and then determine the sum of the pre-stored signal delay duration and the determined exposure delay duration as the delay duration of the target camera.

[0073] Specifically, the controller can determine, for each target camera, the sending time of the last exposure instruction sent to the target camera and the shooting time of the image returned by the target camera; and determine the difference between the shooting time and the sending time as the exposure delay duration of the target camera.

[0074] In this embodiment, the exposure delay duration corresponding to each target camera can be determined in real time, so that the spatiotemporal synchronization of each camera and each laser sensor can be achieved more accurately.

[0075] like Figure 4 As shown, it shows a flow chart of a sensor synchronization method provided by an embodiment of the present application. The method is applied to a controller of an autonomous driving vehicle. The autonomous driving vehicle is also equipped with multiple laser sensors and multiple cameras. The controller is time-synchronized with each laser sensor and each camera. Each preset scanning moment of each laser sensor corresponds to at least one camera. The method may include the following steps:

[0076] S410: Determine the next scanning time of each laser sensor;

[0077] S420: Determine the target cameras corresponding to the next scanning moment according to the correspondence between the preset scanning moments of each laser sensor and the cameras;

[0078] S430: For each target camera, determine the delay time of the target camera, and send an exposure instruction to the target camera at the delay time before the next scanning moment.

[0079] From the above content, it can be seen that the sensor synchronization method provided in the embodiment of the present application can realize time synchronization for the controller, each camera, and each laser sensor, and can pre-build the correspondence between the laser sensor scanning time and each camera. Therefore, when the laser sensor scans the point cloud, the controller can control the camera corresponding to the laser sensor scanning area to expose, thereby ensuring that the point cloud data scanned by the laser sensor and the image captured by the camera are synchronized in time and space.

[0080] Optionally, the step of determining the delay duration of the target camera includes:

[0081] Get the preset signal delay time;

[0082] Determine the exposure delay duration of the target camera;

[0083] The sum of the signal delay time and the exposure delay time is determined as the delay time of the target camera.

[0084] Optionally, the step of determining the exposure delay duration of the target camera includes:

[0085] Determine the sending time of the last exposure instruction sent to the target camera and the shooting time of the image returned by the target camera;

[0086] The difference between the shooting time and the sending time is determined as the exposure delay time of the target camera.

[0087] Optionally, the step of determining the exposure delay duration of the target camera includes:

[0088] The preset exposure delay duration of each camera is obtained as the exposure delay duration of the target camera.

[0089] Optionally, the plurality of laser sensors include: a forward laser sensor, a left laser sensor, a rearward laser sensor, and a right laser sensor; the plurality of cameras include: a front left camera, a front right camera, a left camera, a rear left camera, a rear right camera, and a right camera; each laser sensor points to the front of the autonomous driving vehicle at 0 milliseconds, and the time required for a clockwise rotation of 360 degrees is 100 milliseconds;

[0090] The corresponding relationship between each preset scanning time of each laser sensor and each camera includes: 0 milliseconds corresponds to the front left camera and the front right camera; 25 milliseconds corresponds to the left camera; 50 milliseconds corresponds to the rear left camera and the rear right camera; 75 milliseconds corresponds to the right camera.

[0091] like Figure 5 As shown, it shows a schematic diagram of the structure of a sensor synchronization device provided in an embodiment of the present application, the device is applied to a controller of an autonomous driving vehicle, and the autonomous driving vehicle is also equipped with multiple laser sensors and multiple cameras. The controller is time-synchronized with each of the laser sensors and each of the cameras, and each preset scanning time of each of the laser sensors corresponds to at least one camera. The device includes:

[0092] A scanning time determination module 510, used to determine the next scanning time of each of the laser sensors;

[0093] A target camera determination module 520 is used to determine each target camera corresponding to the next scanning moment according to the correspondence between each preset scanning moment of each laser sensor and each camera;

[0094] The camera control module 530 is used to determine the delay time of each target camera, and send an exposure instruction to the target camera at the delay time before the next scanning moment.

[0095] From the above content, it can be seen that the sensor synchronization device provided in the embodiment of the present application can achieve time synchronization for the controller, each camera, and each laser sensor, and can pre-build the correspondence between the laser sensor scanning time and each camera. Therefore, when the laser sensor scans the point cloud, the controller can control the camera corresponding to the laser sensor scanning area to expose, thereby ensuring that the point cloud data scanned by the laser sensor and the image captured by the camera are synchronized in time and space.

[0096] Optionally, the camera control module 530 includes:

[0097] The signal delay duration acquisition submodule is used to obtain the preset signal delay duration;

[0098] An exposure delay duration determination submodule is used to determine the exposure delay duration of the target camera;

[0099] The delay duration determination submodule is used to determine the sum of the signal delay duration and the exposure delay duration as the delay duration of the target camera.

[0100] Optionally, the exposure delay duration determination submodule is specifically used to:

[0101] Determine the sending time of the last exposure instruction sent to the target camera and the shooting time of the image returned by the target camera;

[0102] The difference between the shooting time and the sending time is determined as the exposure delay duration of the target camera.

[0103] Optionally, the exposure delay duration determination submodule is specifically used to:

[0104] The preset exposure delay duration of each camera is obtained as the exposure delay duration of the target camera.

[0105] Optionally, the multiple laser sensors include: a forward laser sensor, a left laser sensor, a rearward laser sensor, and a right laser sensor; the multiple cameras include: a front left camera, a front right camera, a left camera, a rear left camera, a rear right camera, and a right camera; each of the laser sensors points to the front of the autonomous driving vehicle at 0 milliseconds, and the time required to rotate 360 ​​degrees clockwise is 100 milliseconds;

[0106] The correspondence between the preset scanning moments of each laser sensor and each camera includes: 0 milliseconds corresponds to the front left camera and the front right camera; 25 milliseconds corresponds to the left camera; 50 milliseconds corresponds to the rear left camera and the rear right camera; 75 milliseconds corresponds to the right camera.

[0107] The above device embodiment corresponds to the method embodiment and has the same technical effect as the method embodiment. For specific description, please refer to the method embodiment. The device embodiment is obtained based on the method embodiment. For specific description, please refer to the method embodiment part, which will not be repeated here.

[0108] Next, a controller provided by an embodiment of the present application is introduced. Figure 6 , Figure 6 A schematic diagram of a structure of a controller provided in an embodiment of the present application, the controller includes at least one processor 51, the processor 51 is coupled to a memory 52, the memory 52 stores a program or instruction running on the processor 51, and the program or instruction is executed by the processor to implement the following Figure 4 A sensor synchronization method.

[0109] The present application also provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the following is achieved: Figure 4 A sensor synchronization method.

[0110] The present application also provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the computer program implements the following Figure 4 A sensor synchronization method.

[0111] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present application.

[0112] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A sensor synchronization device, characterized in that: The device is applied to an autonomous driving vehicle, and the device comprises at least: a controller, a plurality of laser sensors, and a plurality of cameras, wherein the controller is time-synchronized with each of the laser sensors and each of the cameras, and each preset scanning time of each of the laser sensors corresponds to at least one camera; The controller is used to determine the next scanning moment of each of the laser sensors; determine each target camera corresponding to the next scanning moment according to the corresponding relationship between each preset scanning moment of each of the laser sensors and each camera; determine the delay time of each target camera, and send an exposure instruction to the target camera at the delay time before the next scanning moment; Each of the target cameras is used to receive an exposure instruction sent by the controller, and perform an exposure operation upon receiving the exposure instruction.

2. The device according to claim 1, characterized in that The controller is specifically used to obtain a preset signal delay duration; for each target camera, determine the exposure delay duration of the target camera, and determine the sum of the signal delay duration and the exposure delay duration as the delay duration of the target camera.

3. The device according to claim 2, characterized in that The controller is specifically used to determine, for each target camera, the sending time of the last exposure instruction sent to the target camera and the shooting time of the image returned by the target camera; and determine the difference between the shooting time and the sending time as the exposure delay duration of the target camera.

4. The device according to claim 2, characterized in that The controller is specifically used to obtain the preset exposure delay time length of each camera as the exposure delay time length of each target camera.

5. The device according to any one of claims 1 to 4, characterized in that: The plurality of laser sensors include: a forward laser sensor, a left laser sensor, a rearward laser sensor, and a right laser sensor; the plurality of cameras include: a front left camera, a front right camera, a left camera, a rear left camera, a rear right camera, and a right camera; each of the laser sensors points to the front of the autonomous driving vehicle at 0 milliseconds, and the time required to rotate 360 ​​degrees clockwise is 100 milliseconds; The correspondence between the preset scanning moments of each laser sensor and each camera includes: 0 milliseconds corresponds to the front left camera and the front right camera; 25 milliseconds corresponds to the left camera; 50 milliseconds corresponds to the rear left camera and the rear right camera; 75 milliseconds corresponds to the right camera.

6. The device according to any one of claims 1 to 4, characterized in that: The device further comprises: a synchronous clock source; the synchronous clock source is connected to the controller and each of the laser sensors.

7. A sensor synchronization method, characterized in that: The method is applied to a controller of an autonomous driving vehicle, wherein a plurality of laser sensors and a plurality of cameras are installed in the autonomous driving vehicle, wherein the controller is time-synchronized with each of the laser sensors and each of the cameras, and each preset scanning moment of each of the laser sensors corresponds to at least one camera, and the method comprises: Determining the next scanning time of each of the laser sensors; Determine the target cameras corresponding to the next scanning moment according to the correspondence between the preset scanning moments of each laser sensor and the cameras; For each target camera, the delay time of the target camera is determined, and an exposure instruction is sent to the target camera at the delay time before the next scanning moment.

8. The method according to claim 7, characterized in that The step of determining the delay duration of the target camera comprises: Get the preset signal delay time; Determine the exposure delay duration of the target camera; The sum of the signal delay time and the exposure delay time is determined as the delay time of the target camera.

9. The method according to claim 8, characterized in that The step of determining the exposure delay duration of the target camera comprises: Determine the sending time of the last exposure instruction sent to the target camera and the shooting time of the image returned by the target camera; The difference between the shooting time and the sending time is determined as the exposure delay duration of the target camera.

10. The method according to claim 8, characterized in that The step of determining the exposure delay duration of the target camera comprises: The preset exposure delay duration of each camera is obtained as the exposure delay duration of the target camera.

11. The method according to any one of claims 7 to 10, characterized in that: The plurality of laser sensors include: a forward laser sensor, a left laser sensor, a rearward laser sensor, and a right laser sensor; the plurality of cameras include: a front left camera, a front right camera, a left camera, a rear left camera, a rear right camera, and a right camera; each of the laser sensors points to the front of the autonomous driving vehicle at 0 milliseconds, and the time required to rotate 360 ​​degrees clockwise is 100 milliseconds; The correspondence between the preset scanning moments of each laser sensor and each camera includes: 0 milliseconds corresponds to the front left camera and the front right camera; 25 milliseconds corresponds to the left camera; 50 milliseconds corresponds to the rear left camera and the rear right camera; 75 milliseconds corresponds to the right camera.

12. A sensor synchronization device, characterized in that: The device is applied to a controller of an autonomous driving vehicle, wherein a plurality of laser sensors and a plurality of cameras are installed in the autonomous driving vehicle, wherein the controller is time-synchronized with each of the laser sensors and each of the cameras, and each preset scanning time of each of the laser sensors corresponds to at least one camera, and the device comprises: A scanning time determination module, used to determine the next scanning time of each of the laser sensors; A target camera determination module, used to determine each target camera corresponding to the next scanning moment according to the corresponding relationship between each preset scanning moment of each laser sensor and each camera; The camera control module is used to determine the delay time of each target camera and send an exposure instruction to the target camera at the delay time before the next scanning moment.

13. The device according to claim 12, characterized in that The camera control module comprises: The signal delay duration acquisition submodule is used to obtain the preset signal delay duration; An exposure delay duration determination submodule is used to determine the exposure delay duration of the target camera; The delay duration determination submodule is used to determine the sum of the signal delay duration and the exposure delay duration as the delay duration of the target camera.

14. The device according to claim 13, characterized in that The exposure delay duration determination submodule is specifically used for: Determine the sending time of the last exposure instruction sent to the target camera and the shooting time of the image returned by the target camera; The difference between the shooting time and the sending time is determined as the exposure delay duration of the target camera.

15. The device according to claim 13, characterized in that The exposure delay duration determination submodule is specifically used for: The preset exposure delay duration of each camera is obtained as the exposure delay duration of the target camera.

16. The device according to any one of claims 12 to 15, characterized in that: The plurality of laser sensors include: a forward laser sensor, a left laser sensor, a rearward laser sensor, and a right laser sensor; the plurality of cameras include: a front left camera, a front right camera, a left camera, a rear left camera, a rear right camera, and a right camera; each of the laser sensors points to the front of the autonomous driving vehicle at 0 milliseconds, and the time required to rotate 360 ​​degrees clockwise is 100 milliseconds; The correspondence between the preset scanning moments of each laser sensor and each camera includes: 0 milliseconds corresponds to the front left camera and the front right camera; 25 milliseconds corresponds to the left camera; 50 milliseconds corresponds to the rear left camera and the rear right camera; 75 milliseconds corresponds to the right camera.

17. A controller, characterized in that: The method comprises at least one processor, wherein the processor is coupled to a memory, wherein the memory stores a program or instruction running on the processor, and when the program or instruction is executed by the processor, the steps of the sensor synchronization method according to any one of claims 7 to 11 are implemented.

18. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the sensor synchronization method according to any one of claims 7 to 11 are implemented.

Images