Multi-sensor spatiotemporal synchronization and traffic dispatching device and method based on vehicle-road collaboration

Abstract

The present invention discloses a multi-sensor spatiotemporal synchronization and traffic scheduling device and method based on vehicle-road collaboration, which relates to the field of road traffic technology. It changes the data aggregation method of the subsystem, constrains the distributed subsystems under the same time axis, and provides a prerequisite for the data fusion of multiple sensing devices. By equipping a device for multi-sensor spatiotemporal synchronization and traffic-driven scheduling based on a vehicle-road collaborative global network, the hard-touch exposure control of the camera and the coordinated cooperation of the camera and the fill light under different lighting environments are realized, providing frame synchronization guarantee for the perception data fusion of the mechanical laser radar and the camera, and ensuring the reliability of the perception data source under different lighting environments, thereby enhancing the effective recall rate of the perception data. By equipping a device for multi-sensor spatiotemporal synchronization and traffic-driven scheduling based on a vehicle-road collaborative global network, the traffic light status information can be uploaded to the cloud server through the network interface, and the scheduling information of the node unit with control authority can also be received to realize the scheduling of traffic flow.

Description

Technical Field

[0001] The present invention relates to the field of road traffic technology, and in particular to a multi-sensor spatiotemporal synchronization and traffic scheduling device and method based on vehicle-road collaboration. Background Art

[0002] Time synchronization and network port data aggregation are important guarantees for the full-speed construction of the vehicle-road cooperative full-area network. Figure 1 As shown, the current technical features are as follows: the data streams of the various sensing devices of the on-board unit and the roadside unit are transmitted to the computing unit, and the time when the computing unit captures the data is used as the time reference to realize data fusion; NTP or PTP synchronization and data stream convergence are performed on devices with network ports; the fusion of mechanical lidar and camera data is based on feature extraction to achieve data fusion; the effective recall rate of camera perception data is affected by light conditions and can only be compensated by algorithms; the scheduling of the traffic system is artificially scheduled based on the local intersection status.

[0003] The above technologies have the following disadvantages: data fusion uses the time point when the computing unit obtains the data as the fusion benchmark, lacking the guarantee of data time validity, because data may be delayed due to link transmission delay, or it may be unable to receive data in time because the computing unit is busy. In addition, the computing unit is directly deployed on the roadside unit side, which places high demands on the environmental adaptability and security of the computing unit, and is incompatible with the data streams of perception devices of other interfaces. Therefore, requirements are put forward for the perception device interfaces of the participants in the construction of the vehicle-road cooperative full-area network, resulting in incompatibility with the perception devices of existing vehicle-mounted units and roadside units; there are mature synchronization protocols based on the local area network, but the perception devices are required to be able to access the local area network and support ptp synchronization or ntp synchronization, resulting in the inability of other existing perception devices to access; the hardware frame synchronization of the mechanical lidar and the camera cannot be achieved, and the data set training and compensation need to be performed on the algorithm side, which increases the computing power load and algorithm accuracy requirements; in order to increase the effective recall rate of camera perception data at night, on the one hand, additional computing power is needed for training, and on the other hand, it is necessary to rely on long-term night fill light, which will cause light pollution; it is incompatible with traffic lights in various regions, and puts forward high requirements for the network scheduling of traffic systems participating in the regional boundary lines of the vehicle-road cooperative full-area network. Summary of the Invention

[0004] The present invention addresses the above technical issues and overcomes the shortcomings of the prior art by providing a multi-sensor spatiotemporal synchronization and traffic-driven scheduling method based on vehicle-road collaboration, comprising:

[0005] (1) Obtain NMEA sentences and PPS signals from the GPS receiver to achieve absolute time and frequency synchronization;

[0006] (2) By writing the physical interface baud rate information and physical link length into the register table in the ARM architecture, the data streams of the sensing devices of various interfaces that do not support the PTP algorithm firmware are marked with timestamp information to eliminate link delays, so that the data streams converge on the network side, and the time of the network interface sensing devices that support the PTP algorithm firmware is synchronized with the hardware time register of the network physical layer under the ARM architecture;

[0007] (3) The time axis of the computing unit and the multi-sensory device is constrained to the same time axis as the multi-sensor spatiotemporal synchronization and traffic-driven scheduling device based on the vehicle-road cooperative global network through a time synchronization protocol.

[0008] (4) Taking the motor scanning rotation point in the mechanical laser radar as the polar coordinate origin, the relative physical position of the camera and the mechanical laser radar is obtained, and by locking the mechanical laser radar scanning phase or obtaining the relative rotation angle and rotation speed of the mechanical laser radar, the hard touch control of the camera is realized, thereby achieving frame synchronization between the mechanical laser radar and the camera;

[0009] (5) By acquiring the signal flow of the photosensitive components, the dimming control of the fill light is performed based on the closed-loop control algorithm of the multi-sensor spatiotemporal synchronization and traffic-driven scheduling device under the vehicle-road cooperative global network;

[0010] (6) Under the full-area vehicle-road cooperative network, the dispatching instructions of the node with control authority are obtained, and the state of the traffic lights is controlled by the combination of the multi-sensor spatiotemporal synchronization and traffic-driven dispatching devices and relay modules under the full-area vehicle-road cooperative network. The countdown lights are controlled according to the communication protocol code stream to realize the dynamic dispatching of the traffic system.

[0011] Technical effect: The present invention can converge the data flow constraints of various sensing devices in the distributed system under the same network time scale, accelerate the global networking of vehicle-road collaboration under the existing infrastructure of vehicle-mounted units and roadside units, and realize the data validity and real-time performance of the communication and interaction of each distributed subsystem. It can further achieve frame synchronization between the mechanical lidar and the camera, improve the ability of sensing data fusion, and thus improve the accuracy of the sensing node. It can achieve an effective recall rate of camera sensing device data under different lighting conditions. It can also obtain the driving capability of traffic light scheduling, and build an execution driving node for the vehicle-road collaborative cloud traffic scheduling algorithm node. This reduces the difficulty of the global networking of vehicle-road collaboration, while reducing the requirements for the environmental adaptability of computing nodes in roadside units, and reducing the cost of the global networking of vehicle-road collaboration.

[0012] The technical solution further defined in the present invention is:

[0013] The multi-sensor spatiotemporal synchronization and traffic-driven scheduling method based on vehicle-road collaboration mentioned above, the spatiotemporal synchronization of the distributed system is achieved in the following way: the computing chip based on the ARM architecture completes the synchronization of absolute time and clock frequency through the NMEA sentences and PPS signals in the GPS receiver, and aggregates the data streams of the perception device participants of various types of interfaces and attaches timestamps.

[0014] The multi-sensor spatiotemporal synchronization and traffic-driven scheduling method based on vehicle-road collaboration mentioned above can also achieve time synchronization of the distributed system's data streams of perception devices in each subsystem of the distributed system by requesting time synchronization services from other effective synchronization subsystems in the vehicle-road collaboration global network when valid satellite information cannot be searched.

[0015] In the aforementioned multi-sensor spatiotemporal synchronization and traffic-driven scheduling method based on vehicle-road collaboration, step (5) obtains the algorithm's demand interface and uses the algorithm as the fill light driver.

[0016] The beneficial effects of the present invention are:

[0017] (1) The present invention is compatible with sensing devices of different interfaces, and can converge the data streams of existing vehicle-mounted units and roadside units to the network side without changing the sensing devices of the existing vehicle-mounted units and roadside units. In addition, the baud rates of various interfaces and the physical lengths of the transmission lines can be written into the register table of the synchronous transparent transmission module. The synchronous transparent transmission module can compensate for the data link transmission delay of the sensing device of the corresponding interface based on a unified clock source when receiving the data stream, and then attach a timestamp to the data of the current frame and transmit the data stream of the frame to the network side. The node construction of this type of data communication link reduces the requirements of the roadside unit computing node on the environmental performance, and converges the data stream to the computing node through the switch and optical fiber. In addition, all data connected to the computing unit will be accompanied by a timestamp, and the algorithm can rely on the timestamp to perform data fusion, avoiding data blocking and causing the processed data results to not have real-time characteristics.

[0018] (2) Based on the time synchronization protocol, the present invention unifies the time of the computing unit, the multi-sensor spatiotemporal synchronization and traffic-driven scheduling device based on the vehicle-road cooperative global network, and the sensing device supporting the time synchronization protocol onto one time axis, providing time guarantee for the data interaction of each distributed subsystem and ensuring the reliability of communication and exchange processing under the vehicle-road cooperative global network;

[0019] (3) The present invention establishes the physical positions of the mechanical laser radar and the camera in the same polar coordinate system, inputs the corresponding coordinate parameters into a device for multi-sensor spatiotemporal synchronization and traffic-driven scheduling based on a vehicle-road cooperative global network, obtains the rotation speed or the rotation angle of the mechanical laser radar to control the exposure time of the camera, and thus realizes the frame synchronization of the mechanical laser radar and the camera, provides an accurate data source for data fusion, and ensures that the data time within the FOV of the mechanical laser radar and the camera is consistent;

[0020] (4) The present invention can use environmental conditions as the driving light source, and can also use algorithm requirements as the driving light source;

[0021] (5) While participating in the vehicle-road cooperative global network, the present invention can also accept scheduling instructions from nodes with control authority, thereby realizing traffic light status control based on relay modules to adapt to traffic control systems in different regions. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a diagram of a time synchronization architecture in the background technology;

[0023] Figure 2 Schematic diagram of multi-sensor spatiotemporal synchronization and traffic scheduling in the present invention. DETAILED DESCRIPTION

[0024] Example 1

[0025] This embodiment provides a multi-sensor spatiotemporal synchronization and traffic scheduling method based on vehicle-road collaboration, including:

[0026] (1) Obtain NMEA sentences and PPS signals from the GPS receiver to achieve absolute time and frequency synchronization;

[0027] (2) By writing the physical interface baud rate information and the physical link length into the register table in the ARM architecture, the data streams of the sensing devices of various interfaces that do not support the PTP algorithm firmware are marked with timestamp information to eliminate link delays, so that the data streams converge on the network side, and the time of the network interface sensing devices that support the PTP algorithm firmware is synchronized with the hardware time register of the network physical layer under the ARM architecture, eliminating the time uncertainty of the sensing devices or computing units with different computing capabilities when transmitting the network application layer data packets to the physical layer;

[0028] (3) The time axis of the computing unit and the multi-sensory device is constrained to the same time axis as the multi-sensor spatiotemporal synchronization and traffic-driven scheduling device based on the vehicle-road cooperative global network through a time synchronization protocol, and time drive is used as the basis for algorithm processing to enhance the reliability and real-time performance of the data.

[0029] (4) Taking the motor scanning rotation point in the mechanical laser radar as the polar coordinate origin, the relative physical position of the camera and the mechanical laser radar is obtained, and by locking the mechanical laser radar scanning phase or obtaining the relative rotation angle and rotation speed of the mechanical laser radar, the hard touch control of the camera is realized, ensuring that the perception data time of the mechanical laser radar and the camera is consistent under the same FOV, thereby achieving frame synchronization between the mechanical laser radar and the camera;

[0030] (5) By acquiring the signal flow of the photosensitive components, the fill light is dimmed and controlled based on the closed-loop control algorithm of the multi-sensor spatiotemporal synchronization and traffic-driven scheduling device under the vehicle-road cooperative global network, thus compensating for the deficiency of the camera in extracting the characteristics of the object in a low-light environment;

[0031] (6) Under the full-area vehicle-road cooperative network, the dispatching instructions of the node with control authority are obtained, and the state of the traffic lights is controlled by the combination of the multi-sensor spatiotemporal synchronization and traffic-driven dispatching devices and relay modules under the full-area vehicle-road cooperative network. The countdown lights are controlled according to the communication protocol code stream to realize the dynamic dispatching of the traffic system.

[0032] Example 2

[0033] This embodiment provides a multi-sensor spatiotemporal synchronization and traffic scheduling device based on vehicle-road collaboration, including the following modules:

[0034] Spatiotemporal synchronization of distributed systems is achieved through the following methods: ARM-based computing chips synchronize absolute time and clock frequency using NMEA commands and PPS signals from GPS receivers. They also aggregate and timestamp data streams from sensory devices participating in various interfaces. If valid satellite information cannot be retrieved, the system can also synchronize the data streams from sensor devices within the distributed system by requesting time synchronization services from other validly synchronized subsystems within the V2X network. Within the V2X architecture, vehicle-road, road-road, and vehicle-to-vehicle information interactions are constrained to the same time scale, providing time guarantees for traffic system scheduling.

[0035] Data stream convergence and time synchronization for various sensor devices: An ARM-based computing chip, by writing the baud rate and physical length of each interface device into its register table, timestamps the data streams of existing onboard and roadside unit sensor devices to eliminate link latency. This data is then converged onto the network link via the network port, rapidly enabling existing onboard and roadside unit sensor devices to participate in vehicle-road collaborative networking. A unified hardware time source is also provided over the LAN link for sensor devices with network ports and PTP firmware support, eliminating network link transmission latency through message data exchange.

[0036] Frame synchronization control of mechanical LiDAR and camera: By locking the phase of the mechanical LiDAR, its rotation speed or angle information is obtained, thereby achieving frame synchronization between the mechanical LiDAR and camera.

[0037] Traffic dispatch under the full-area vehicle-road collaboration: Under the full-area vehicle-road collaboration network, dispatch instructions are obtained from the node with control authority, and the status of traffic lights is controlled with the help of relay modules and communication countdown lights.

[0038] Without changing the hardware layout of the existing onboard and roadside units, the subsystem's data aggregation method is changed by installing a device based on the vehicle-road collaborative global network for multi-sensor spatiotemporal synchronization and traffic-driven scheduling, and the distributed subsystems are constrained to the same timeline, providing a prerequisite for data fusion of multiple sensing devices. By installing a device based on the vehicle-road collaborative global network for multi-sensor spatiotemporal synchronization and traffic-driven scheduling, the camera's hard-touch exposure control and the coordinated cooperation of the camera and fill light in different lighting environments are achieved, providing frame synchronization for the perception data fusion of the mechanical lidar and the camera, ensuring the reliability of the perception data source in different lighting environments and enhancing the effective recall rate of perception data. By installing a device based on the vehicle-road collaborative global network for multi-sensor spatiotemporal synchronization and traffic-driven scheduling, traffic light status information can be uploaded to the cloud server through the network interface, and scheduling information from the node unit with control authority can also be received to achieve traffic flow scheduling.

[0039] In addition to the above embodiments, the present invention may also have other implementations. Any technical solution formed by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A multi-sensor spatiotemporal synchronization and traffic scheduling method based on vehicle-road collaboration, characterized by: include: (1) Obtain NMEA sentences and PPS signals from the GPS receiver to achieve absolute time and frequency synchronization; (2) By writing the physical interface baud rate information and physical link length into the register table in the ARM architecture, the data streams of the sensing devices of various interfaces that do not support the PTP algorithm firmware are marked with timestamp information to eliminate link delays, so that the data streams converge on the network side, and the time of the network interface sensing devices that support the PTP algorithm firmware is synchronized with the hardware time register of the network physical layer under the ARM architecture; (3) The time axis of the computing unit and the multi-sensor device is constrained to the same time axis as the multi-sensor spatiotemporal synchronization and traffic-driven scheduling device based on the vehicle-road cooperative global network through a time synchronization protocol; (4) Taking the motor scanning rotation point in the mechanical laser radar as the polar coordinate origin, the relative physical position of the camera and the mechanical laser radar is obtained, and by locking the mechanical laser radar scanning phase or obtaining the relative rotation angle and rotation speed of the mechanical laser radar, the hard touch control of the camera is realized, thereby achieving frame synchronization between the mechanical laser radar and the camera; (5) By acquiring the signal flow of the photosensitive components, the dimming control of the fill light is performed based on the closed-loop control algorithm of the multi-sensor spatiotemporal synchronization and traffic-driven scheduling device under the vehicle-road cooperative global network; (6) Under the full-area vehicle-road cooperative network, the dispatching instructions of the node with control authority are obtained, and the state of the traffic light is controlled by the combination of the multi-sensor spatiotemporal synchronization and traffic-driven dispatching device and relay module under the full-area vehicle-road cooperative network. The countdown light is controlled according to the communication protocol code stream to realize the dynamic dispatching of the traffic system; The time synchronization of the distributed system can also be achieved by requesting the time synchronization services of other valid synchronization subsystems in the vehicle-road cooperative global network when valid satellite information cannot be searched, so as to achieve the time synchronization of the data streams of the perception devices of each subsystem in the distributed system.

2. The multi-sensor spatiotemporal synchronization and traffic scheduling method based on vehicle-road collaboration according to claim 1 is characterized by: The spatiotemporal synchronization of the distributed system is achieved in the following way: the computing chip based on the ARM architecture completes the synchronization of absolute time and clock frequency through the NMEA sentences and PPS signals in the GPS receiver, and aggregates the data streams of the perception device participants of various types of interfaces and attaches timestamps.

3. The multi-sensor spatiotemporal synchronization and traffic scheduling method based on vehicle-road collaboration according to claim 1 is characterized by: The step (5) obtains the algorithm's requirement interface and uses the algorithm as a fill light driver.

4. A multi-sensor spatiotemporal synchronization and traffic dispatching device based on vehicle-road collaboration, characterized by: The software architecture of the device applies any one of claims 1-3.

Images