Multi-source time domain synchronization method and device in vehicle infrastructure collaborative awareness

Abstract

The invention relates to the technical field of automatic driving, in particular to a multi-source time domain synchronization method and device in vehicle infrastructure collaborative awareness. The method comprises the following steps: the RSU periodically broadcasts configuration information of sensing equipment to vehicles in a coverage range; the vehicle receives the configuration information of the RSU after entering the coverage area, and calculates the time offset of the RSU sensing equipment according to the received configuration information; after the vehicle receives the sensing data of the RSU sensing equipment, the sensing data are cached, and the size of the cached data is predefined according to the fusion sensing capability of the vehicle; and the vehicle extracts self-sensing data and sensing data frames of the RSU sensing equipment according to a shortest time delay criterion, and fusion sensing is carried out. According to the method, a multi-source time domain synchronization scheme is provided, the amount of time needed to be shifted when different RSUs reach data is calculated, the amount of time is compensated, then similar data frames in the time domain are extracted for fusion, and the sensing precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a multi-source time domain synchronization method and device in vehicle-road cooperative perception. Background Art

[0002] With the rapid development of intelligent transportation, vehicle-infrastructure cooperative perception has gradually become one of the important technologies to improve traffic safety. In VICS (Vehicle Infrastructure Cooperative Systems), multiple sensing devices are usually installed on both sides of the road, such as cameras, millimeter-wave radars, and lidars, which are used to monitor the traffic environment in real time; at the same time, the vehicle itself is also equipped with on-board sensing devices that can collect data about its surrounding environment.

[0003] When a vehicle is traveling on a road area that supports vehicle-road collaboration, it can communicate with the RSU (Road Side Unit) device through its own communication terminal to obtain real-time information collected by the roadside perception device. By fusing the perception data collected by the vehicle itself with the perception data obtained from the RSU, the vehicle can significantly improve the perception accuracy, expand the perception range, and enhance the robustness of the system.

[0004] For vehicles, it is necessary to fuse perception data from multiple sources. In order to ensure the accuracy of data fusion, multi-source data from different perception devices need to be synchronized as much as possible in time. However, since high-precision global time synchronization is usually not performed between different RSUs, the collected perception data will cause time differences during transmission to the vehicle due to the transmission delay of the communication link from RSU to the vehicle and the movement of the vehicle. This time difference makes it difficult to directly fuse data, which may cause perception errors, thereby affecting the reliability and accuracy of the system.

[0005] Existing global time synchronization solutions usually rely on high-precision time synchronization devices, such as GPS (Global Positioning System) timing devices. Such devices provide a unified time reference through GPS to ensure that the clocks of all vehicles, RSUs, and sensing devices remain highly consistent. Although this global synchronization solution can provide extremely high accuracy, it is expensive and difficult to implement in actual deployment. Therefore, it is necessary to design a new time domain synchronization method.

[0006] Prior art 1: Global synchronization:

[0007] Global synchronization means that all devices, including vehicles, RSUs, and sensing devices, are equipped with high-precision time synchronization devices, such as GPS high-precision timing, to achieve global consistency of time. Through synchronization devices, vehicles, RSUs, and sensing devices can obtain time information under the same time reference, calculate offsets based on absolute time, and extract sensing information at the same time.

[0008] GPS high-precision timing is a technology that uses the time signals provided by GPS satellites to achieve high-precision time synchronization. Each GPS satellite carries a high-precision atomic clock that can provide very accurate timestamp information. Vehicles, RSUs, and sensing devices receive time signals from GPS satellites by installing GPS receivers, and synchronize with the global unified time base through the received time signals, thereby ensuring that their internal clocks are consistent with the external time source.

[0009] Prior art 2: Rough synchronization / asynchronous:

[0010] Coarse synchronization means that all devices, including vehicles, RSUs, and sensing devices, do not perform precise time synchronization. Specifically, the timestamps between sensing devices do not need to be strictly aligned, but a certain time error is allowed. This time error may come from transmission delays and processing delays between different devices. The time error of coarse synchronization is relatively small, but it is still not accurate.

[0011] Asynchronous means that there is no synchronization between all devices, and each device relies on its own local clock to work independently. Since there is no synchronization mechanism, the time error between devices may be very large, resulting in the inability to directly integrate the perception data.

[0012] Global synchronization usually relies on high-precision time synchronization equipment, which is costly, inflexible, and difficult to implement in actual deployment. Coarse synchronization / asynchronous synchronization, on the other hand, has a small joint perception range supported by vehicles and poor perception accuracy performance. Summary of the invention

[0013] In order to solve the technical problems that global synchronization in the prior art usually relies on high-precision time synchronization equipment, which is costly, inflexible, and difficult to implement in actual deployment; and rough synchronization / asynchronous synchronization, the joint perception range supported by vehicles is small, and the perception accuracy performance is poor, the embodiment of the present invention provides a multi-source time domain synchronization method and device in vehicle-road cooperative perception. The technical solution is as follows:

[0014] On the one hand, a multi-source time domain synchronization method in vehicle-road cooperative perception is provided, characterized in that the method includes:

[0015] S1. RSU periodically broadcasts the configuration information of the sensing device to vehicles within the coverage area;

[0016] S2, after entering the coverage area, the vehicle receives the configuration information of the RSU and calculates the time offset of the RSU sensing device according to the received configuration information;

[0017] S3. After receiving the perception data from the RSU perception device, the vehicle caches the perception data. The size of the cached data is predefined according to the vehicle's fusion perception capability.

[0018] S4. The vehicle extracts its own perception data and the perception data frames of the RSU perception device according to the criterion of the shortest delay, and performs fusion perception.

[0019] Optionally, in S1, the RSU periodically broadcasts the configuration information of the sensing device to the vehicles within the coverage area, including:

[0020] Each RSU on both sides of the road periodically broadcasts the configuration information of its sensing equipment to vehicles within the coverage area;

[0021] The configuration information includes: GPS coordinates, connected sensor type, sensor data volume parameters, and data frame duration information.

[0022] Optionally, in S2, calculating the time offset of the RSU sensing the device according to the received configuration information includes:

[0023] The vehicle receives the configuration information and calculates the time offset of the RSU sensing device according to the following formula (1):

[0024] (1)

[0025] Where T is the time domain offset of the RSU sensing device calculated by the vehicle; is the time advance for sending wireless signals, indicating the delay compensation that may occur during data transmission; N is an integer; f is the length of the wireless data frame or time slot; P is the RSU processing time delay, which is broadcast by the RSU or predefined by the system.

[0026] Optionally, in S3, after receiving the perception data from the RSU perception device, the vehicle caches the perception data, and the size of the cached data is predefined according to the fusion perception capability of the vehicle, including:

[0027] The vehicle's cache size is calculated according to the following formula (2):

[0028] (2)

[0029] Where C is the cache capacity of the vehicle; M is the number of sensing devices; is the frame transmission rate of the i-th sensing device; is the time offset of the i-th sensing device; The size of each frame of data transmitted to the i-th sensing device.

[0030] Optionally, in S4, the vehicle extracts its own perception data and the perception data frames of the RSU perception device according to the criterion of the shortest delay, and performs fusion perception, including:

[0031] When the vehicle receives data from multiple sources, it preferentially selects the data frame with the shortest delay, that is, the data frame with the smallest time offset, as the reference data frame;

[0032] The data of other sensing devices are processed synchronously based on the time of the reference data frame;

[0033] The vehicle fuses the data of the baseline data frame with other perception data to generate accurate perception results.

[0034] On the other hand, a multi-source time domain synchronization device in vehicle-road cooperative perception is provided, and the device is applied to a multi-source time domain synchronization method in vehicle-road cooperative perception, and the device includes:

[0035] RSU broadcast module, used for RSU to periodically broadcast the configuration information of the sensing device to vehicles within the coverage area;

[0036] A time offset calculation module is used for receiving the configuration information of the RSU after the vehicle enters the coverage area, and calculating the time offset of the RSU sensing device according to the received configuration information;

[0037] The information cache module is used for caching the perception data after the vehicle receives the perception data from the RSU perception device. The size of the cached data is predefined according to the vehicle's fusion perception capability;

[0038] The fusion perception module is used by the vehicle to extract its own perception data and the perception data frames of the RSU perception device according to the criterion of the shortest delay for fusion perception.

[0039] Optionally, an RSU broadcast module is used for each RSU on both sides of the road to periodically broadcast the configuration information of its sensing device to vehicles within the coverage area;

[0040] The configuration information includes: GPS coordinates, connected sensor type, sensor data volume parameters, and data frame duration information.

[0041] Optionally, a time offset calculation module is used for calculating the time offset of the RSU sensing device according to the following formula (1) based on the configuration information received by the vehicle:

[0042] (1)

[0043] Where T is the time domain offset of the RSU sensing device calculated by the vehicle; is the time advance for sending wireless signals, indicating the delay compensation that may occur during data transmission; N is an integer; f is the length of the wireless data frame or time slot; P is the RSU processing time delay, which is broadcast by the RSU or predefined by the system.

[0044] On the other hand, a multi-source time domain synchronization device for vehicle-road collaborative perception is provided, and the multi-source time domain synchronization device for vehicle-road collaborative perception includes: a processor; a memory, and the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the multi-source time domain synchronization methods for vehicle-road collaborative perception described above is implemented.

[0045] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the multi-source time domain synchronization methods in the above-mentioned vehicle-road cooperative perception.

[0046] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0047] In the embodiment of the present invention, the system architecture includes RSU periodically broadcasting configuration information, the vehicle receiving RSU broadcast information, the vehicle calculating the time domain offset of different RSU sensing devices, and the vehicle fusing its own sensing data with the sensing data obtained from the RSU. This method can be used in all scenarios where the sensing data of different sensing devices need to be synchronized in the time domain. The time offset of different RSU sensing devices is calculated by the offset formula, and the time offset is compensated, and then the data frames of different RSUs that are close in time domain are extracted for fusion, thereby improving the perception accuracy of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 It is a flow chart of a multi-source time domain synchronization method in vehicle-road cooperative perception provided by an embodiment of the present invention;

[0050] Figure 2 It is an implementation scenario diagram provided by an embodiment of the present invention;

[0051] Figure 3 It is a block diagram of a multi-source time domain synchronization device in vehicle-road cooperative perception provided by an embodiment of the present invention;

[0052] Figure 4It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0054] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0055] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0056] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0057] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0058] The embodiment of the present invention provides a multi-source time domain synchronization method in vehicle-road cooperative perception, which can be implemented by a multi-source time domain synchronization device in vehicle-road cooperative perception, and the multi-source time domain synchronization device in vehicle-road cooperative perception can be a terminal or a server. Figure 1 The flowchart of the multi-source time domain synchronization method in vehicle-road cooperative perception shown in the figure may include the following steps:

[0059] S1. RSU periodically broadcasts the configuration information of the sensing device to vehicles within the coverage area;

[0060] In a feasible implementation, in S1, the RSU periodically broadcasts the configuration information of the sensing device to the vehicles within the coverage area, including:

[0061] Each RSU on both sides of the road periodically broadcasts the configuration information of its sensing equipment to vehicles within the coverage area;

[0062] The configuration information includes: GPS coordinates, connected sensor types, sensor data volume parameters, data frame duration information, etc. This information will serve as an important reference parameter for the vehicle's subsequent calculation of time offset.

[0063] In a feasible implementation, GPS coordinates are used to calculate distance, and the signal propagation time delay is calculated based on the distance, (distance / speed of light), which is used to calculate the first term TA in formula 1; the type of connected sensor may affect the amount of data, for example, a monocular camera has half as much data as a binocular camera; the frame data mean and the average effective data volume of a single frame of the perception device.

[0064] In a feasible implementation, vehicle-road cooperative perception refers to the technology of information exchange and perception data sharing between vehicles and RSUs through wireless communication technology. Its purpose is to improve the overall perception of the traffic environment and enhance traffic safety and traffic efficiency by integrating the perception data of vehicles and RSUs. Vehicle-road cooperative perception can realize the fusion of multi-source information, expand the perception range of vehicles, provide more accurate environmental information, and provide a basis for vehicle decision-making and control.

[0065] Time domain synchronization refers to the time alignment of data from different devices or sensors so that their data can be collected and processed on the same time basis. Since the transmission delay and collection time of different devices are different, time domain synchronization can synchronize all data in time by calculating the time offset of the data and performing time compensation to ensure the accuracy of data fusion.

[0066] S2, after entering the coverage area, the vehicle receives the configuration information of the RSU and calculates the time offset of the RSU sensing device according to the received configuration information;

[0067] In a feasible implementation manner, in S2, calculating the time offset of the RSU sensing device according to the received configuration information includes:

[0068] The vehicle receives the configuration information and calculates the time offset of the RSU sensing device according to the following formula (1):

[0069] (1)

[0070] Where T is the time domain offset of the RSU sensing device calculated by the vehicle; It is the time advance of wireless signal transmission, indicating the delay compensation that may occur during data transmission. N is an integer. f is the length of the wireless data frame or time slot. P is the RSU processing time delay, which is broadcast by the RSU or predefined by the vehicle-road cooperative system. There is no specific standard for predefinition here, but only a fixed value is defined, such as 1ms, 2ms, etc.

[0071] In a feasible implementation, the integer N is calculated according to the following formula:

[0072]

[0073] Among them, the average value of each frame data of the sensing device and the effective data volume of a single frame are obtained by RSU broadcast.

[0074] In a feasible implementation, the vehicle can calculate the time offset of the RSU sensing data based on the configuration information provided by each RSU. This time offset reflects the degree of asynchrony of the data of different sensing devices in the time domain. The vehicle can subsequently use this offset to compensate for data from different sources to ensure data alignment during data fusion.

[0075] S3. After receiving the perception data from the RSU perception device, the vehicle caches the perception data. The size of the cached data is predefined according to the vehicle's fusion perception capability.

[0076] In a feasible implementation, the vehicle receives and caches the perception data from different RSU perception devices, and the cache size is predefined according to the vehicle's fusion perception capability. Specifically, a vehicle with a strong fusion perception capability needs to process a large amount of perception data from multiple sensors, and requires a larger cache space. On the other hand, a vehicle with a weaker fusion perception capability has a relatively small cache size because it processes less perception data.

[0077] In a feasible implementation, the buffer size of the vehicle is calculated according to the following formula (2):

[0078] (2)

[0079] Where C is the cache capacity of the vehicle; M is the number of sensing devices; is the frame transmission rate of the i-th sensing device; is the time offset of the i-th sensing device; The size of each frame of data transmitted to the i-th sensing device.

[0080] S4. The vehicle extracts its own perception data and the perception data frames of the RSU perception device according to the criterion of the shortest delay, and performs fusion perception.

[0081] In a feasible implementation, in S4, the vehicle extracts its own perception data and the perception data frame of the RSU perception device according to the criterion of the shortest delay, and performs fusion perception, including:

[0082] When the vehicle receives data from multiple sources, it preferentially selects the data frame with the shortest delay, that is, the data frame with the smallest time offset, as the reference data frame;

[0083] The data of other sensing devices are processed synchronously based on the time of the reference data frame;

[0084] The vehicle fuses the data of the baseline data frame with other perception data to generate accurate perception results.

[0085] In a feasible implementation manner, the other sensing devices are all sensing devices except the reference sensing device with the smallest time domain offset. Figure 2 As shown, it involves a vehicle-road cooperative perception system, in which there are multiple RSUs on the roadside, and each RSU may be equipped with multiple types of perception devices, such as cameras, millimeter-wave radars, lidars, etc. These perception devices can collect surrounding environment data in real time and broadcast the perception data to vehicles in the coverage area through the RSU.

[0086] Similarly, the target vehicle Car1 itself is also equipped with an on-board sensing device to collect environmental information around the vehicle itself. When Car1 is driving in the RSU signal coverage area, the target vehicle Car1 receives the sensing information broadcast by multiple RSUs. In order to more accurately integrate the sensing data collected by Car1 itself and the sensing data obtained from the RSU, the sensing data of multiple RSUs needs to be time synchronized.

[0087] In a feasible implementation, in the prior art, the RSU divides vehicles into a set of sensing vehicles and a set of computing vehicles, and the sensing and computing functions are respectively undertaken by different vehicles. At this time, the sensing data needs to be transmitted between vehicles, which is easily affected by factors such as network delays and signal interference, thereby reducing the response efficiency of the system; however, this patent integrates the sensing and computing functions on the same vehicle, so that the data does not need to be transmitted between vehicles, eliminating the delay caused by data transmission, and the sensing data can be processed instantly in the vehicle to achieve real-time calculation and decision-making. Especially in high-speed driving or emergency situations, the vehicle can quickly process and respond to the sensing information, greatly improving the overall response speed and accuracy of the system, thereby ensuring driving safety.

[0088] The present invention proposes a multi-source time domain synchronization scheme, which calculates the amount of time required for different RSUs to reach data, compensates for the amount of time, and then extracts data frames that are close in time domain for fusion, thereby improving perception accuracy. The system architecture of the present invention includes RSU periodically broadcasting configuration information, vehicles receiving RSU broadcast information, vehicles calculating the time domain offset of different RSU perception devices, and vehicles fusing their own perception data with the perception data obtained from RSU. This method can be used in all scenarios where the perception data of different perception devices need to be synchronized in time domain.

[0089] The present invention calculates the time offset of different RSU sensing devices through an offset formula, compensates for the time offset, and then extracts data frames of different RSUs that are close in time domain for fusion, thereby improving the perception accuracy of the vehicle.

[0090] The present invention can be applied to any scenario with time domain synchronization and perception fusion, such as V2V (Vehicle-to-Vehicle) scenario, where multiple vehicles can ensure that the data between them can be aligned and fused under the same time reference through time domain synchronization to achieve accurate collaborative perception and decision-making.

[0091] Figure 3 1 is a block diagram of a multi-source time domain synchronization device 300 in vehicle-road cooperative perception according to an exemplary embodiment. The device 300 is used in a multi-source time domain synchronization method in vehicle-road cooperative perception. Figure 3 The device includes an RSU broadcast module 310, a time offset calculation module 320, an information cache module 330, and a fusion perception module 340. Among them:

[0092] The RSU broadcast module 310 is used for the RSU to periodically broadcast the configuration information of the sensing device to the vehicles within the coverage area;

[0093] The time offset calculation module 320 is used for receiving the configuration information of the RSU after the vehicle enters the coverage area, and calculating the time offset of the RSU sensing device according to the received configuration information;

[0094] The information cache module 330 is used for caching the perception data after the vehicle receives the perception data from the RSU perception device, and the size of the cached data is predefined according to the fusion perception capability of the vehicle;

[0095] The fusion perception module 340 is used for the vehicle to extract its own perception data and the perception data frame of the RSU perception device according to the criterion of the shortest delay, and perform fusion perception.

[0096] Optionally, the RSU broadcast module 310 is used for each RSU on both sides of the road to periodically broadcast the configuration information of its sensing device to vehicles within the coverage area;

[0097] The configuration information includes: GPS coordinates, connected sensor type, sensor data volume parameters, and data frame duration information.

[0098] Optionally, the time offset calculation module 320 is used for calculating the time offset of the RSU sensing device according to the following formula (1) based on the configuration information received by the vehicle:

[0099] (1)

[0100] Where T is the time domain offset of the RSU sensing device calculated by the vehicle; is the time advance for sending wireless signals, indicating the delay compensation that may occur during data transmission; N is an integer; f is the length of the wireless data frame or time slot; P is the RSU processing time delay, which is broadcast by the RSU or predefined by the system.

[0101] Optionally, the time offset calculation module 320 is used to calculate the buffer size of the vehicle according to the following formula (2):

[0102] (2)

[0103] Where C is the cache capacity of the vehicle; M is the number of sensing devices; is the frame transmission rate of the i-th sensing device; is the time offset of the i-th sensing device; The size of each frame of data transmitted to the i-th sensing device.

[0104] Optionally, the fusion perception module 340 is used for, when the vehicle receives multi-source data, the vehicle preferentially selects a data frame with the shortest delay, that is, a data frame with the smallest time offset, as a reference data frame;

[0105] The data of other sensing devices are processed synchronously based on the time of the reference data frame;

[0106] The vehicle fuses the data of the baseline data frame with other perception data to generate accurate perception results.

[0107] The present invention proposes a multi-source time domain synchronization scheme, which calculates the amount of time required for different RSUs to reach data, compensates for the amount of time, and then extracts data frames that are close in time domain for fusion, thereby improving perception accuracy. The system architecture of the present invention includes RSU periodically broadcasting configuration information, vehicles receiving RSU broadcast information, vehicles calculating the time domain offset of different RSU perception devices, and vehicles fusing their own perception data with the perception data obtained from RSU. This method can be used in all scenarios where the perception data of different perception devices need to be synchronized in time domain.

[0108] The present invention calculates the time offset of different RSU sensing devices through an offset formula, compensates for the time offset, and then extracts data frames of different RSUs that are close in time domain for fusion, thereby improving the perception accuracy of the vehicle.

[0109] The present invention can be applied to any scenario with time domain synchronization and perception fusion, such as V2V (Vehicle-to-Vehicle) scenario, where multiple vehicles can ensure that the data between them can be aligned and fused under the same time reference through time domain synchronization to achieve accurate collaborative perception and decision-making.

[0110] Figure 4 is a structural diagram of a multi-source time domain synchronization device in vehicle-road cooperative perception provided by an embodiment of the present invention, such as Figure 4 As shown, the multi-source time domain synchronization device in vehicle-road cooperative perception may include the above Figure 3 The multi-source time domain synchronization device in vehicle-road cooperative perception shown. Optionally, the multi-source time domain synchronization device 410 in vehicle-road cooperative perception may include a first processor 2001.

[0111] Optionally, the multi-source time domain synchronization device 410 in vehicle-road cooperative perception may also include a memory 2002 and a transceiver 2003 .

[0112] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0113] Combine the following Figure 4 The components of the multi-source time domain synchronization device 410 in vehicle-road cooperative perception are specifically introduced:

[0114] The first processor 2001 is the control center of the multi-source time domain synchronization device 410 in the vehicle-road cooperative perception, and can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0115] Optionally, the first processor 2001 can perform various functions of the multi-source time domain synchronization device 410 in vehicle-road cooperative perception by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0116] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 are shown in FIG.

[0117] In a specific implementation, as an embodiment, the multi-source time domain synchronization device 410 in vehicle-road cooperative perception may also include multiple processors, such as Figure 4 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0118] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0119] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be accessed through the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0120] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0121] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 4 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0122] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and may be connected to the first processor 2001 through the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0123] It should be noted that Figure 4 The structure of the multi-source time domain synchronization device 410 in the vehicle-road cooperative perception shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0124] In addition, the technical effects of the multi-source time domain synchronization device 410 in vehicle-road cooperative perception can refer to the technical effects of the multi-source time domain synchronization method in vehicle-road cooperative perception described in the above method embodiment, and will not be repeated here.

[0125] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0126] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0127] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0128] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0129] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0130] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0133] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.

[0134] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A multi-source time domain synchronization method in vehicle-road cooperative perception, characterized in that: The method comprises: S1. RSU periodically broadcasts the configuration information of the sensing device to vehicles within the coverage area; S2, the vehicle receives the configuration information of the RSU after entering the coverage area, and calculates the time offset of the RSU sensing device according to the received configuration information; S3. After receiving the perception data from the RSU perception device, the vehicle caches the perception data, and the size of the cached data is predefined according to the fusion perception capability of the vehicle; S4. The vehicle extracts its own perception data and the perception data frames of the RSU perception device according to the criterion of the shortest delay, and performs fusion perception.

2. The multi-source time domain synchronization method in vehicle-road cooperative perception according to claim 1 is characterized in that: In S1, the RSU periodically broadcasts the configuration information of the sensing device to the vehicles within the coverage area, including: Each RSU on both sides of the road periodically broadcasts the configuration information of its sensing equipment to vehicles within the coverage area; The configuration information includes: GPS coordinates, connected sensor type, sensor data volume parameters and data frame duration information.

3. The multi-source time domain synchronization method in surface-vehicle-road cooperative perception according to claim 2 is characterized in that: In S2, the time offset of the RSU sensing device is calculated according to the received configuration information, including: The configuration information received by the vehicle is used to calculate the time offset of the RSU sensing device according to the following formula (1): （1） Where T is the time domain offset of the RSU sensing device calculated by the vehicle; is the time advance for sending wireless signals, indicating the delay compensation that may occur during data transmission; N is an integer; f is the length of the wireless data frame or time slot; P is the RSU processing time delay, which is broadcast by the RSU or predefined by the system.

4. The multi-source time domain synchronization method in vehicle-road cooperative perception according to claim 3 is characterized in that: In S3, after receiving the perception data from the RSU perception device, the vehicle caches the perception data, and the size of the cached data is predefined according to the fusion perception capability of the vehicle, including: The buffer size of the vehicle is calculated according to the following formula (2): （2） Where C is the cache capacity of the vehicle; M is the number of sensing devices; is the frame transmission rate of the i-th sensing device; is the time offset of the i-th sensing device; The size of each frame of data transmitted to the i-th sensing device.

5. The multi-source time domain synchronization method in vehicle-road cooperative perception according to claim 4 is characterized in that: In S4, the vehicle extracts its own perception data and the perception data frame of the RSU perception device according to the criterion of the shortest delay, and performs fusion perception, including: When the vehicle receives data from multiple sources, the vehicle preferentially selects the data frame with the shortest delay, that is, the data frame with the smallest time offset, as the reference data frame; The data of other sensing devices are processed synchronously based on the time of the reference data frame; The vehicle fuses the data of the reference data frame with other perception data to generate accurate perception results.

6. A multi-source time domain synchronization device in vehicle-road cooperative perception, the multi-source time domain synchronization device in vehicle-road cooperative perception is used to implement the multi-source time domain synchronization method in vehicle-road cooperative perception as claimed in any one of claims 1 to 5, characterized in that: The device comprises: RSU broadcast module, used for RSU to periodically broadcast the configuration information of the sensing device to vehicles within the coverage area; A time offset calculation module, used for the vehicle to receive the configuration information of the RSU after entering the coverage area, and calculate the time offset of the RSU sensing device according to the received configuration information; An information cache module, used for the vehicle to cache the perception data after receiving the perception data from the RSU perception device, and the size of the cached data is predefined according to the fusion perception capability of the vehicle; The fusion perception module is used by the vehicle to extract its own perception data and the perception data frames of the RSU perception device according to the criterion of the shortest delay for fusion perception.

7. The multi-source time domain synchronization device in vehicle-road cooperative perception according to claim 6 is characterized in that: The RSU broadcast module is used for each RSU on both sides of the road to periodically broadcast the configuration information of its sensing device to vehicles within the coverage area; The configuration information includes: GPS coordinates, connected sensor type, sensor data volume parameters and data frame duration information.

8. The multi-source time domain synchronization device in vehicle-road cooperative perception according to claim 7 is characterized in that: The time offset calculation module is used for the configuration information received by the vehicle to calculate the time offset of the RSU sensing device according to the following formula (1): （1） Where T is the time domain offset of the RSU sensing device calculated by the vehicle; is the time advance for sending wireless signals, indicating the delay compensation that may occur during data transmission; N is an integer; f is the length of the wireless data frame or time slot; P is the RSU processing time delay, which is broadcast by the RSU or predefined by the system.

9. A multi-source time domain synchronization device in vehicle-road cooperative perception, characterized in that: The multi-source time domain synchronization device in the vehicle-road cooperative perception includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 5.

Images