Vehicle-road collaborative data fusion method, device, electronic device and system

Abstract

The present invention relates to the field of intelligent transportation technology, and particularly relates to a data fusion method, device, electronic device and system based on vehicle-road cooperation. The method includes receiving target information of a preset area sent by a road-side device, where the target information includes the position and speed of a target; acquiring first point cloud data of the target detected by the vehicle itself in the current detection area; determining a fusion strategy for the target information and the first point cloud data based on the position relationship between the vehicle itself and the preset area; and fusing the target information and the first point cloud data according to the fusion strategy to determine target detection information in the current detection area. By fusing the target information of the preset area determined by the road-side device with the first point cloud data detected by the vehicle itself in the current detection area, and adopting corresponding fusion strategies in different scenarios during fusion, the computational complexity of data fusion is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent driving, and particularly relates to a data fusion method, device, electronic device, and system based on vehicle-road cooperation. Background Art

[0002] In intelligent driving technology, the perception fusion system, as the "senses" of a driverless vehicle, plays a crucial role. By fusing information from multiple sensors, it can provide a vehicle with a wider detection range, higher accuracy, better stability, and more effective surrounding environment information. Multi-sensor fusion has extensive applications in the field of intelligent driving. For example, it can be used for the detection and tracking of dynamic and static targets, the detection of lane lines, the recognition of traffic signals and traffic signs, etc.

[0003] With the increasing maturity of 5G technology and road infrastructure, it has accelerated the development of technologies such as vehicle-to-everything (V2X) and vehicle-road cooperation in the vehicle network. Perception fusion is no longer limited to a single vehicle. Through cellular network or WIFI technology, communication and interaction between vehicles and between vehicles and roads can be realized, greatly expanding the scope of environmental perception and achieving the maximum sharing of information. However, the numerous and redundant multi-source information brings challenges such as a sharp increase in the demand for hardware computing resources and a complex and time-consuming fusion process. Therefore, how to effectively fuse road-side information and vehicle-side information is an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a data fusion method, device, electronic device, and system based on vehicle-road cooperation to solve the complexity problem of data fusion in vehicle-road cooperation.

[0005] According to a first aspect, an embodiment of the present invention provides a data fusion method based on vehicle-road cooperation, including:

[0006] Receiving target information of a preset area sent by a road-side device, where the target information includes the position and speed of a target;

[0007] Obtaining first point cloud data of a target detected by the vehicle itself in the current detection area;

[0008] Based on the positional relationship between the position of the vehicle itself and the position of the preset area, determining a fusion strategy for the target information and the first point cloud data;

[0009] Fusing the target information and the first point cloud data according to the fusion strategy to determine target detection information in the current detection area.

[0010] The data fusion method based on vehicle-road cooperation provided by the embodiment of the present invention fuses the target information of a preset area determined in a roadside device with the first point cloud data detected by the vehicle itself in the current detection area, and adopts a corresponding fusion strategy in combination with the positional relationship between the vehicle itself and the preset area during fusion, realizing the adoption of corresponding fusion strategies in different scenarios and reducing the computational complexity of data fusion. At the same time, on the one hand, this fusion method can achieve the full fusion of data in the roadside device and the vehicle device. On the other hand, the data collected by the roadside device is processed on the roadside device side to obtain target information and then sent to the vehicle device for fusion, which can reduce the demand for computing power of the vehicle device, improve the reuse rate of the roadside device, and enhance the stability of the vehicle device.

[0011] In some optional embodiments, fusing the target information and the first point cloud data according to the fusion strategy to determine the target detection information in the current detection area includes:

[0012] Obtain the position of the target in the target information;

[0013] Based on the positional relationship between the position of the target and the first point cloud data, determine the corresponding relationship between the target and the detection target;

[0014] Based on the fusion strategy and the corresponding relationship, fuse the target information and the first point cloud data to determine the target detection information in the current detection area.

[0015] When the data fusion method based on vehicle-road cooperation provided by the embodiment of the present invention fuses the target information of the roadside device and the target detection information of the vehicle device, it first determines the corresponding relationship between the target and the detection target by using the positional relationship, selects the corresponding fusion strategy based on this corresponding relationship and then performs data fusion, which can ensure the accuracy requirements for the fused data in different scenarios.

[0016] In some optional embodiments, when the position of the vehicle itself is within the preset area, fusing the target information and the first point cloud data according to the fusion strategy to determine the target detection information in the current detection area includes:

[0017] Use the corresponding relationship to determine the targets simultaneously detected by the roadside device and the vehicle itself to obtain the first target point cloud data corresponding to the simultaneously detected targets, and the position accuracy of the first target point cloud data is higher than the position accuracy in the target information;

[0018] Use the point cloud coordinates in the first target point cloud data to calculate the target position of the simultaneously detected targets;

[0019] Based on the target position and the speed of the simultaneously detected target, determine the target detection information within the current detection area, where the speed accuracy in the target information is higher than that in the first point cloud data.

[0020] The vehicle-road collaborative data fusion method provided by the embodiments of the present invention always trusts the data with higher accuracy and uses it as the fused data when performing data fusion. That is, for the targets simultaneously detected by the roadside device and the vehicle itself, a tight fusion strategy is adopted, and the data with higher accuracy is used for data update during data fusion, which not only ensures low computing power but also high accuracy.

[0021] In some alternative embodiments, when the position of the vehicle is outside the preset area, the fusing the target information and the detection information according to the fusion strategy to determine the target detection information of the detected target within the current detection area includes:

[0022] Use the corresponding relationship to determine the missed detection targets detected by the roadside device but not by the vehicle itself to obtain the target information of the missed detection targets;

[0023] Based on the target information of the missed detection targets and the first point cloud data of the detected target, determine the target detection information within the current detection area.

[0024] For the missed detection targets detected by the roadside device but not by the vehicle itself, when the position of the vehicle is outside the preset area, the vehicle-road collaborative data fusion method provided by the embodiments of the present invention adopts a loose fusion strategy, and only needs to inform the vehicle of the target information of the missed detection targets, and uses the target information given by the roadside device for blind area compensation and reminder, reducing the amount of data processing.

[0025] In some alternative embodiments, the method for determining the target information includes:

[0026] Obtain the image detection data and the second point cloud data of the preset area;

[0027] Perform target recognition on the image detection data to determine the region of interest corresponding to the target within the preset area;

[0028] Based on the positional relationship between the region of interest corresponding to the target and the point cloud in the second point cloud data, determine the second target point cloud data corresponding to the target;

[0029] Based on the second target point cloud data, determine the target information.

[0030] In the data fusion method based on vehicle-road cooperation provided by the embodiments of the present invention, since the accuracy of image detection data for target determination is higher than that of point cloud data, the region of interest corresponding to the target within the preset region is determined using the target recognition result of the image detection data, ensuring the reliability of the determination of the region of interest. Then, the second point cloud data is fused with the region of interest, that is, data fusion is achieved at the roadside device to obtain target information, which can reduce the data processing volume of the vehicle device and lower the computing power requirement of the vehicle device.

[0031] In some alternative embodiments, determining the target information based on the second target point cloud data includes:

[0032] Screening out the target point cloud closest to the target from the second target point cloud data;

[0033] Determining the position and speed of the target point cloud as the position and speed of the target.

[0034] In the data fusion method based on vehicle-road cooperation provided by the embodiments of the present invention, due to the characteristics of sparsity and randomness of point cloud reflection points, and there is no one-to-one mapping between the second point cloud data and the region of interest, selecting the coordinates and speed of the closest point as the coordinates and speed of the target can ensure the reliability of the target information.

[0035] In some alternative embodiments, determining the target information based on the second target point cloud data includes:

[0036] Performing clustering processing on the second target point cloud data to determine the target point cloud;

[0037] Determining the position and speed of the target point cloud as the position and speed of the target.

[0038] In the data fusion method based on vehicle-road cooperation provided by the embodiments of the present invention, the target point cloud is screened out by clustering, for example, density-based clustering or graph theory-based clustering methods, etc., which can also ensure the reliability of the target information.

[0039] According to a second aspect, the embodiments of the present invention further provide a data fusion device based on vehicle-road cooperation, including:

[0040] A receiving module, configured to receive the target information of the preset region sent by the roadside device, where the target information includes the position and speed of the target;

[0041] An obtaining module, configured to obtain the first point cloud data of the detected target of the vehicle in the current detection region;

[0042] A first determination module, configured to determine a fusion strategy for the target information and the first point cloud data based on the positional relationship between the position of the vehicle itself and the position of the preset area;

[0043] A second determination module, configured to fuse the target information and the first point cloud data according to the fusion strategy to determine target detection information within the current detection area.

[0044] According to a third aspect, an embodiment of the present invention provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute the vehicle-road collaborative data fusion method described in the first aspect or any one of the implementation manners of the first aspect.

[0045] According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the vehicle-road collaborative data fusion method described in the first aspect or any one of the implementation manners of the first aspect.

[0046] According to a fifth aspect, an embodiment of the present invention provides a vehicle-road collaborative system, including:

[0047] A roadside device, including an image acquisition device, a first point cloud acquisition device, and a first computing platform. The first computing platform is configured to perform data fusion based on the image detection data acquired by the image acquisition device and the second point cloud data acquired by the first point cloud acquisition device to determine target information, where the target information includes the position and speed of a target within a preset area;

[0048] A vehicle-side device, including a second point cloud acquisition device and a second computing platform. The second point cloud acquisition device is configured to acquire first point cloud data of a target detected by the vehicle itself within the current detection area. The second computing platform is connected to the first computing platform and is configured to execute the vehicle-road collaborative data fusion method described in the first aspect or any one of the implementation manners of the first aspect of the present invention.

[0049] It should be noted that for the corresponding beneficial effects of the vehicle-road collaborative data fusion device, electronic device, computer-readable storage medium, and vehicle-road collaborative system provided by the embodiments of the present invention, please refer to the description of the corresponding beneficial effects of the vehicle-road collaborative data fusion method above, and will not be elaborated here. Description of the Drawings

[0050] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0051] Figure 1 is a schematic diagram of a vehicle-road collaborative system according to an embodiment of the present invention;

[0052] Figure 2 is a schematic diagram of data fusion based on vehicle-road collaboration according to an embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of the conversion from the lidar coordinate system to the map coordinate system according to an embodiment of the present invention;

[0054] Figure 4 is a flowchart of a data fusion method based on vehicle-road collaboration according to an embodiment of the present invention;

[0055] Figure 5 is a flowchart of a data fusion method based on vehicle-road collaboration according to an embodiment of the present invention;

[0056] Figure 6 is a structural block diagram of a data fusion device based on vehicle-road collaboration according to an embodiment of the present invention;

[0057] Figure 7 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. Specific Embodiments

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0059] An embodiment of the present invention provides a vehicle-road collaborative system, including roadside devices and vehicle-mounted devices. Among them, the roadside devices are used to be deployed on roads, including standardized roads and non-standardized roads in some special scenarios, such as intersections, complex sections, vehicle driving tracks without clear boundary lines, etc.; the vehicle-mounted devices are used to be deployed on vehicles, and the vehicles here include passenger cars, commercial vehicles, construction machinery, etc. The specific deployment location of the roadside devices is set according to actual needs, and no restrictions are imposed on it here. Taking the same intersection as an example, in order to achieve full-round and dead-angle-free data collection of the intersection, multiple sets of roadside devices can be deployed, and these roadside devices can share a computing platform.

[0060] Specifically, the roadside device includes an image acquisition device, a first point cloud acquisition device, and a first computing platform. Among them, the image acquisition device is used to acquire image detection data, the first point cloud acquisition device is used to acquire second point cloud data, and the first computing platform is used to fuse the image acquisition data and the second point cloud data to obtain target information. Among them, the target information includes the position and speed of the target within a preset area. Continuing with the above example, multiple image acquisition devices, multiple first point cloud acquisition devices, and multiple first computing platforms are deployed within the preset area. Among them, the number of image acquisition devices, first point cloud acquisition devices, and first computing platforms is set according to actual needs, and no restrictions are imposed on it here. For example, multiple image acquisition devices and multiple first point cloud acquisition devices share a first computing platform, that is, the first computing platform is connected to multiple image acquisition devices and multiple first point cloud acquisition devices. Multiple image acquisition devices send the acquired image detection data and multiple first point cloud acquisition devices send the acquired second point cloud data to the first computing platform for data fusion. At the same time, since multiple image acquisition devices and multiple first point cloud acquisition devices may have the situation that the same target is repeatedly acquired when deployed within the preset area, therefore, duplicate removal processing is also required when performing data fusion in the first computing platform.

[0061] Of course, in some alternative embodiments, for the computing platform of the vehicle-mounted device, the data fusion task can also be requested from a remote cloud server to execute. After obtaining the fusion result in the cloud server, it is then sent to the second computing platform of the vehicle-mounted device.

[0062] In some alternative embodiments, the image acquisition device is a camera, and the first point cloud acquisition device is a millimeter-wave radar, etc.

[0063] The vehicle-end device includes a second point cloud acquisition device and a second computing platform. The second point cloud acquisition device is used to obtain the first point cloud data of the detection target of the vehicle in the current detection area. The second computing platform is connected to the first computing platform and is used to fuse the target information of the first computing platform with the first point cloud data collected by the second point cloud acquisition device to obtain the target detection information in the current detection area. Among them, the second computing platform can be an intelligent driving domain controller, an intelligent cockpit domain controller or a central controller in the vehicle, and the second point cloud acquisition device is a lidar, etc.

[0064] Figure 1 Fig. shows an optional schematic diagram of the vehicle-road collaborative system. Road-side devices 10 are deployed at intersections, and vehicle-end devices 20 are deployed for vehicles. Among them, Figure 1 Only one set of road-side devices 10 is shown in the figure, but the protection scope of the present invention is not limited thereto, and it is specifically set according to actual needs. For example, road-side devices are installed at intersection positions, including a monocular camera, a millimeter-wave radar, a first computing platform and a first communication device. The first communication device here includes but is not limited to 5G modules, Bluetooth modules or WIFI modules, etc. Only the installation positions of a monocular camera and a millimeter-wave radar are schematically shown in the figure. Of course, the specific quantity depends on whether the entire intersection area can be covered. The vehicle-end device includes a 360-degree surround-view lidar, a second computing platform and a second communication device. Only the installation position of the lidar is schematically shown in the figure. Among them, the second communication device includes but is not limited to 5G modules, Bluetooth modules or WIFI modules, etc.

[0065] The working principle of this vehicle-road collaborative system is as follows: The image acquisition device and the first point cloud data in the road-side device 10 respectively collect the image detection data and the second point cloud data in the preset area. The first computing platform performs data fusion on the image detection data and the second point cloud data to determine the target information of the target in the preset area. The first computing platform sends the determined target information regularly or in real time. For vehicles within the communication range of the road-side device 10, they can receive this target information. Among them, the second point cloud acquisition device in the vehicle-end device 20 deployed on the vehicle collects the first point cloud data in the current detection area, and performs data fusion on the first point cloud data and the target information to obtain the target detection information in the current detection area. Further, after the vehicle obtains the target detection information, it can use this target detection information to control the vehicle, so as to achieve intelligent control or driverless control.

[0066] For an intersection, multiple sets of cameras and lidars are installed to cover the entire area. If there is an overlapping field of view area, a target may be repeatedly detected by multiple sets of combinations. Therefore, duplicate removal processing is required later. After duplicate removal processing, all sensors at the roadside will output a target information, and each target is a target information containing coordinate and speed information. It should be noted that the coordinate values of this target information are in the Cartesian coordinate system based on the millimeter-wave radar. In order to fuse with the sensor information at the vehicle end, the coordinate values of this target information need to be converted to the map coordinate system, where the map coordinate system is a Cartesian coordinate system with a certain point in the park as the origin. Finally, the target information after coordinate conversion is transmitted to the vehicle end through 5G communication.

[0067] To reduce the sensor configuration and computing unit at the vehicle end, only one surround-view lidar is equipped at the vehicle end. The lidar will provide the 3D bounding box and point cloud of the target in the map coordinate system, and then fuse with the 3D bounding box provided by the roadside.

[0068] Figure 2 The schematic diagram of data fusion based on vehicle-road collaboration is shown. For roadside devices, first fuse the data of a set of radar and image acquisition device camera, and then fuse and remove duplicates from the data of all roadside devices to obtain the target information of roadside devices. Since the position information in the target information is the coordinate in the radar coordinate system, in order to fuse with the data in the vehicle-end device later, coordinate transformation is required, that is, project the coordinate in the radar coordinate system onto the map coordinate system. Among them, the specific coordinate transformation is as Figure 3 shown. For vehicle-end devices, process the acquisition data of the vehicle-end lidar, and then project the target into the map coordinate system through coordinate mapping. Finally, fuse the vehicle-end target and the roadside target to obtain the target detection information of the current detection area of the vehicle.

[0069] For example, the formula for converting the vehicle-end lidar coordinate system to the map coordinate system is as follows:

[0070] x g = x v + x s sinΨ + y s cosΨ

[0071] y g = y v - x s cosΨ + y s sinΨ

[0072] Among them, x s 、y sis the position of the target in the vehicle-mounted lidar coordinate system; x g and y g are the positions of the target in the map coordinate system; x v and y v are the positions of the vehicle itself in the map coordinate system; Ψ is the current heading angle of the vehicle itself.

[0073] During the fusion process, always fully trust the most reliable information given by a certain type of sensor, and do not use the less reliable information of the same type provided by other sensors. For example, both millimeter-wave radar and lidar can provide the speed information of the target. The former can give a relatively accurate target speed using the Doppler principle, while the latter estimates the speed using the time difference method, but with lower accuracy and larger fluctuations. Therefore, during the fusion of speed, only the speed information of the millimeter-wave radar is adopted, and the speed information provided by the lidar is not adopted.

[0074] Following this principle, the cameras and millimeter-wave radars at the roadside first perform feature-level fusion. Since the cameras have a lower miss detection rate and false detection rate for targets, and the millimeter-wave radar has the advantages of high accuracy and high resolution in speed estimation, when fusing the cameras and millimeter-wave radars at the roadside, fully trust the detection of the cameras for targets, that is, only use the 2D target boxes output by the cameras to fuse with the detection results of the millimeter-wave radar (including information such as coordinates and speed), and then output a more accurate 3D target box including coordinates and speed.

[0075] Since most multi-sensor target fusion solutions rely only on vehicle intelligence, it is necessary to equip more and more types of sensors, and at the same time, it is necessary to deploy computing units with better performance and higher power consumption, thus increasing the cost and system complexity of the vehicle; at the same time, it reduces the reuse rate of the equipment and the stability of the system. In addition, the detection range of the perception system based on the vehicle is limited, and there are detection blind spots. To solve this problem, the vehicle-road collaborative system provided by the embodiments of the present invention fuses the target information of the roadside equipment with the first point cloud data of the vehicle-mounted equipment, improves the reuse rate of the roadside equipment and the stability of the vehicle-mounted system, and can effectively fill in the blind spots.

[0076] According to the embodiments of the present invention, an embodiment of a data fusion method based on vehicle-road collaboration is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0077] In this embodiment, a data fusion method based on vehicle-road collaboration is provided, which can be used in electronic devices, such as the second computing platform of a vehicle, a vehicle, etc. Figure 4is a flowchart of a data fusion method based on vehicle-road cooperation according to an embodiment of the present invention, as Figure 4 shown. The process includes the following steps:

[0078] S11, receiving target information of a preset area sent by a roadside device.

[0079] Among them, the target information includes the position and speed of the target.

[0080] The roadside device is deployed in the preset area. The target information of the preset area is obtained by the first computing platform in the roadside device after fusing data from an image acquisition device and a first point cloud acquisition device. The target information includes the targets detected by the roadside device in the preset area, as well as the positions and speeds of each target. In some embodiments, the targets in the preset area are obtained by performing target recognition on the image detection data collected by the image acquisition device. The positions and speeds of each target are obtained by analyzing the positional relationship between the second point cloud data collected by the first point cloud acquisition device and the target, and using the coordinates and speeds of the point clouds in the second point cloud data. No specific limitation is imposed on the specific determination method of the target information here, and it can be set according to actual needs.

[0081] As described above, after determining the target information of the preset area, the roadside device sends the target information regularly or in real time. Vehicles within the communication range of the roadside device will receive this target information. Correspondingly, for the vehicle-mounted device, it can receive the target information of the preset area.

[0082] S12, obtaining first point cloud data of the targets detected by the vehicle itself in the current detection area.

[0083] It should be noted that the targets in the embodiments of the present invention are the targets detected by the roadside device; the detected targets are the targets collected by the vehicle itself, that is, the targets detected by the vehicle-mounted device.

[0084] For the vehicle itself, since its location may not necessarily be within the preset area, the first point cloud data collected by the vehicle itself is the point cloud data of the targets detected in the current detection area. Among them, the detected targets in the current detection area may have the same targets as those in the preset area described above, or may have different targets. For example, the targets in the preset area include target 1, target 2, and target 3, and the detected targets in the current detection area include detected target 1, detected target 2, and detected target 3.

[0085] The second point cloud acquisition device in the vehicle-end device deployed on the vehicle acquires point cloud data. By analyzing the point cloud data, the first point cloud data of each detection target can be determined. For example, the second point cloud acquisition device is a lidar. By processing the point cloud information of the lidar, detection targets are obtained. Each detection target includes a three-dimensional bounding box, that is, a 3D bounding box (including the coordinates of the center point, speed, and the length, width, and height of the bounding box) and the point cloud included in the 3D bounding box (for example, using the coordinate values of the point cloud), etc. Since the center point coordinates of the obtained 3D bounding box are based on the lidar coordinate system, if it is to be fused with the target at the road end, the center point coordinates need to be converted from the lidar coordinate system to the map coordinate system. Among them, for the specific coordinate conversion, please refer to the above description.

[0086] S13. Based on the positional relationship between the vehicle's position and the preset area, determine the fusion strategy for the target information and the first point cloud data.

[0087] The vehicle's position can be determined by the positioning device deployed on the vehicle, and the position of the preset area can be sent by the road-end device to the vehicle. By comparing the vehicle's position with the position of the preset area, it is determined whether the vehicle is within the range of the preset area. For the vehicle being within the range of the preset area or outside the range of the preset area, corresponding fusion strategies are respectively adopted to fuse the target information and the first point cloud data.

[0088] For example, the preset area is an intersection area. For the vehicle, it is necessary to obtain more accurate target positions and speeds in order to avoid in time; for non-intersection areas, for the vehicle, it only needs to know whether there are targets around. Therefore, for the above two situations, corresponding fusion strategies need to be respectively adopted.

[0089] It should be noted that the definition of the preset area is set according to the actual scenario requirements and is not limited to the intersection described above.

[0090] Of course, for the road-end device in the non-preset area, it will also output the processed target information, so that the vehicle receiving the target information can perform data fusion between the road-end device and the vehicle-end device based on the target information, and further achieve accurate control of the vehicle.

[0091] S14. According to the fusion strategy, fuse the target information and the first point cloud data to determine the target detection information in the current detection area.

[0092] After determining the fusion strategy in S13 above, the electronic device fuses the target information and the first point cloud data based on the fusion strategy. Since there may be the same or different targets in the target information and the first point cloud data, for the same target, it is necessary to determine whether to use the position and speed in the target information or the first point cloud data; for different targets, blind area filling processing and so on are required. These processing methods all depend on their corresponding fusion strategies, and the fusion strategy is set according to actual needs, and no restrictions are imposed on it here.

[0093] The data fusion method based on vehicle-road collaboration provided in this embodiment fuses the target information of the preset area determined in the roadside device with the first point cloud data detected by the vehicle itself in the current detection area, and adopts the corresponding fusion strategy in combination with the position relationship between the vehicle's own position and the preset area during fusion, realizing the adoption of corresponding fusion strategies in different scenarios and reducing the computational complexity of data fusion. At the same time, on the one hand, this fusion method can achieve the full fusion of data in the roadside device and the vehicle device. On the other hand, the data collected by the roadside device is processed on the roadside device side to obtain the target information and then sent to the vehicle device for fusion, which can reduce the demand for computing power of the vehicle device and improve the reuse rate of the roadside device and the stability of the vehicle device.

[0094] In some embodiments, the method for determining the target information includes:

[0095] (1) Obtain the image detection data and the second point cloud data of the preset area.

[0096] (2) Perform target recognition on the image detection data to determine the region of interest corresponding to the target in the preset area.

[0097] (3) Based on the position relationship between the region of interest corresponding to the target and the point cloud in the second point cloud data, determine the second target point cloud data corresponding to the target.

[0098] (4) Determine the target information based on the second target point cloud data.

[0099] The image detection data of the preset area is collected by the image acquisition device in the roadside device, and the second point cloud data is collected by the first point cloud acquisition device in the roadside device. For the image detection data, image analysis can be performed on it to determine the region of interest corresponding to each target in the preset area; or, using a target recognition model, input the image detection data into the target recognition model to output the region of interest corresponding to each target, and so on. No restrictions are imposed on the method for determining the region of interest of each target here.

[0100] The region of interest is used to represent the regions corresponding to each target. The second point cloud data includes the point cloud data of all targets. By comparing the positions of the points in the second point cloud data with the region of interest, the points that belong to the region of interest are determined, thereby determining the second target point cloud data corresponding to the target. Since the positions and velocities of each point cloud can be obtained when collecting the second point cloud data, correspondingly, the second target point cloud data includes the positions and velocities of each point cloud. Based on this, by analyzing the second target point cloud data corresponding to each region of interest, the positions and velocities of the targets corresponding to the regions of interest can be determined.

[0101] Since the accuracy of target determination by image detection data is higher than that of point cloud data, therefore, the region of interest corresponding to the target within the preset region is determined using the target recognition result of the image detection data, ensuring the reliability of the determination of the region of interest. Then, the second point cloud data is fused with the region of interest, that is, data fusion is achieved at the roadside device to obtain target information, which can reduce the data processing volume of the vehicle-side device and reduce the computing power requirement of the vehicle-side device.

[0102] In some embodiments, step (3) above includes:

[0103] 3.1) Screen out the target point cloud closest to the target from the second target point cloud data.

[0104] 3.2) Determine the position and velocity of the target as the position and velocity of the target point cloud.

[0105] Since the point cloud reflection points have the characteristics of sparsity and randomness, and there is no one-to-one mapping between the second point cloud data and the region of interest, therefore, selecting the coordinates and velocity of the closest point as the coordinates and velocity of the target can ensure the reliability of the target information.

[0106] In other embodiments, step (3) above includes:

[0107] 3.1) Perform clustering processing on the second target point cloud data to determine the target point cloud.

[0108] 3.2) Determine the position and velocity of the target as the position and velocity of the target point cloud.

[0109] A clustering method is used to find a target point cloud from the second target point cloud data, and this target point cloud is used to represent the target, thereby obtaining the position and velocity of the target. Using a clustering method to screen out the target point cloud, for example, density-based clustering or graph theory-based clustering methods, etc., can also ensure the reliability of the target information.

[0110] As a specific application example of this embodiment, the image acquisition device in the roadside device is a camera, and the first point cloud acquisition device is a millimeter-wave radar. Based on this, the method for the roadside device to determine the target information includes:

[0111] (1) Process the image detection data collected by the camera to obtain a preliminary detection. The detection result includes the center point of the target on the image, the size, depth, and classification information of the two-dimensional bounding box (i.e., 2D bounding box).

[0112] (2) Traverse each 2D bounding box obtained in step (1) to obtain the size, depth, and classification information of the target box.

[0113] (3) For each target box, map it into a 3D region of interest (ROI) in the form of a frustum.

[0114] (4) Traverse all the millimeter-wave radar point clouds. For each millimeter-wave radar detection point, determine whether the point is within the above-mentioned ROI region. If it is true, put it into the point cloud buffer. It should be noted that a unified coordinate system has been selected for a set of camera and millimeter-wave radar combinations during installation at the roadside, which is called the roadside coordinate system, that is, there is no coordinate system extrinsic parameter conversion between the millimeter-wave radar and the camera. Therefore, no coordinate system conversion is required before determining whether the millimeter-wave radar point cloud is within the ROI region.

[0115] (5) After traversing the millimeter-wave radar point clouds, find the point with the closest distance in the point cloud buffer in step (4), and use the coordinates and speed of this point as the coordinates and speed of the target.

[0116] (6) After traversing all the 2D bounding boxes, an object information will be output as the fusion output result of this set of camera and millimeter-wave radar combinations. This object list is based on the camera detection and completely adopts the coordinate values and speed values provided by the millimeter-wave radar.

[0117] (7) The object information output in step (6) is based on the roadside coordinate system. If this object list needs to be fused with the vehicle-side sensor information, the coordinates of the target in this object information need to be converted from the roadside coordinate system to the map coordinate system.

[0118] In this embodiment, a data fusion method based on vehicle-road cooperation is provided, which can be used in electronic devices, such as the second computing platform of a vehicle, a vehicle, etc. Figure 5 It is a flowchart of the data fusion method based on vehicle-road cooperation according to the embodiment of the present invention, as Figure 5 shown, and this process includes the following steps:

[0119] S21, Receive the target information of the preset area sent by the roadside device.

[0120] Among them, the target information includes the position and speed of the target.

[0121] For details, please refer to Figure 4 S11 of the illustrated embodiment, which will not be elaborated here.

[0122] S22, Obtain the first point cloud data of the detected target of the vehicle in the current detection area.

[0123] For details, please refer to Figure 4 S12 of the illustrated embodiment, which will not be elaborated here.

[0124] S23, Based on the positional relationship between the position of the vehicle and the position of the preset area, determine the fusion strategy for the target information and the first point cloud data.

[0125] For details, please refer to Figure 4 S12 of the illustrated embodiment, which will not be elaborated here.

[0126] S24, According to the fusion strategy, fuse the target information and the first point cloud data to determine the target detection information in the current detection area.

[0127] Specifically, the above S24 includes:

[0128] S241, Obtain the position of the target in the target information.

[0129] The target information is sent by the roadside device and received by the vehicle. By analyzing the received target information, the position of the target in the target information can be obtained.

[0130] S242, Based on the positional relationship between the position of the target and the position represented by the first point cloud data, determine the correspondence between the target and the detected target.

[0131] The first point cloud data corresponds to the detected target in the current detection area, and the target is the target in the preset area. In order to determine the correspondence between the target and the detected target, that is, which targets correspond to the detected target, it is necessary to analyze the positional relationship between the position of the target and the position represented by the first point cloud data to determine the correspondence between the target and the detected target.

[0132] Specifically, the correspondence includes the simultaneous detection targets that are detected by both the roadside device and the vehicle device, and the missed detection targets that are only detected by the roadside device and not detected by the vehicle device, etc. As mentioned above, for data fusion, the data with higher accuracy is always trusted. Therefore, for the simultaneous detection targets, it is necessary to select the data with higher accuracy as the finally determined data.

[0133] S243. Based on the fusion strategy and the corresponding relationship, fuse the target information with the first point cloud data to determine the target detection information within the current detection area.

[0134] When the position of the vehicle is within the preset area, the above S243 includes:

[0135] (1) Use the corresponding relationship to determine the targets simultaneously detected by the roadside device and the vehicle itself, so as to obtain the first target point cloud data corresponding to the simultaneously detected targets, and the position accuracy of the first target point cloud data is higher than the position accuracy in the target information.

[0136] (2) Use the point cloud coordinates in the first target point cloud data to calculate the target position of the simultaneously detected targets.

[0137] (3) Based on the target position and the speed of the simultaneously detected targets, determine the target detection information within the current detection area, and the speed accuracy in the target information is higher than the speed accuracy in the first point cloud data.

[0138] Since the position accuracy of the first target point cloud data is higher than the position accuracy in the target information, it is necessary to calculate the target position using the first target point cloud data and update the position in the target information using this target position. Specifically, for the simultaneously detected targets, extract the first target point cloud data of the simultaneously detected targets from the first point cloud data. For each point cloud in the first target point cloud data, it has position information, and the average value can be calculated to determine the target position of the simultaneously detected targets; or other methods can be used for calculation, for example, taking the center point position of the first target point cloud data as the target position of the simultaneously detected targets, etc.

[0139] Since the speed accuracy in the target information is higher than the speed accuracy in the first point cloud data, the speed in the target information is used as the speed of the simultaneously detected targets. Thus, the target position and speed of the simultaneously detected targets can be obtained. Since the simultaneously detected targets are also among the detected targets, the information of the simultaneously detected targets in the detected targets is updated to the target position and the determined speed. For other detected targets that do not belong to the simultaneously detected targets, there is no need to adjust their positions and speeds. After the above processing, the target detection information within the current detection area can be obtained.

[0140] When performing data fusion, always trust the data with higher accuracy and use it as the fused data. That is, for the targets simultaneously detected by the roadside device and the vehicle itself, a tight fusion strategy is adopted, and the data with higher accuracy is used for data update during data fusion, which not only ensures low computing power but also ensures high accuracy.

[0141] When the position of the vehicle is outside the preset area, since the communication range of the roadside device is generally larger than the preset area, there will be a situation where the vehicle can receive target information, but the position of the vehicle is outside the preset area. Based on this, the above S243 includes:

[0142] (1) Using the corresponding relationship to determine the undetected targets detected by the roadside device but not by the vehicle itself, so as to obtain the target information of the undetected targets.

[0143] (2) Based on the target information of the undetected targets and the first point cloud data of the detected targets, determine the target detection information in the current detection area.

[0144] The targets detected by the roadside device but not by the vehicle itself are called undetected targets. Accordingly, the target information of the undetected targets can be obtained. By splicing the target information of the undetected targets with the first point cloud data of the detected targets, the target detection information in the current detection area can be obtained.

[0145] When the position of the vehicle is outside the preset area, for the undetected targets detected by the roadside device but not by the vehicle itself, adopting a loose fusion strategy only needs to inform the vehicle of the target information of the undetected targets, and uses the target information given by the roadside device for blind area filling and reminder, reducing the amount of data processing.

[0146] The data fusion method based on vehicle-road collaboration provided in this embodiment, when fusing the target information of the roadside device with the target detection information of the vehicle device, first uses the position relationship to determine the corresponding relationship between the target and the detected target, and then performs data fusion based on this corresponding relationship, which can ensure the accuracy requirements of the fused data in different scenarios.

[0147] As a specific application example of this embodiment, the second point cloud acquisition device in the vehicle device is a lidar. Based on this, the above data fusion method based on vehicle-road collaboration includes:

[0148] (1) Process the point cloud information of the lidar to obtain a list of detected targets, and each target includes a three-dimensional bounding box, that is, a 3D bounding box (including the coordinates of the center point, speed, and the length, width, and height of the bounding box) and the point cloud included in the 3D bounding box, etc.

[0149] (2) The center point coordinates of the 3D bounding box obtained in step (1) are based on the lidar coordinate system. If it is to be fused with the target list of the roadside, the center point coordinates need to be converted from the lidar coordinate system to the map coordinate system.

[0150] (3) In this step, the concepts of electronic fence and scenario switching are introduced. The electronic fence refers to the intersection area range sent from the roadside unit. When the vehicle is not within the intersection area, that is, not within the electronic fence area, at this time, the scenario switching module selects an appropriate fusion strategy (loose fusion), that is, enters step (7), because when not within the intersection area, it is not necessary to know the accurate position of the target in the blind area, only need to tell the vehicle that there is a target in the front blind area to remind the vehicle and select an appropriate decision-making strategy; when the vehicle drives into the intersection area, it means entering the electronic fence. At this time, the scenario switching module selects an appropriate fusion strategy (tight fusion), that is, enters steps (4)-(6). After entering this area, it is necessary to obtain the accurate position of the target in the blind area through the deep fusion of the vehicle side and the roadside unit to help the vehicle avoid the target. By judging whether it is within the electronic fence area and dynamically changing the fusion strategy through the scenario switching module, rather than uniformly adopting the fusion strategy represented by steps (4)-(6), the computational complexity can be effectively reduced.

[0151] (4) As mentioned above, since lidar has obvious advantages over the other two sensors in ranging accuracy, the coordinate values of lidar point clouds will be used in the fusion. If we want to fuse the lidar point cloud and the roadside target, it is necessary to unify the coordinate system, that is, to convert the lidar point cloud from the lidar coordinate system to the map coordinate system.

[0152] (5) Traverse each roadside target, and for each roadside target, traverse each point cloud collected by the lidar. If the point cloud is within the target area, it is put into the point cloud buffer. It should be noted that the size of this target area will select an appropriate range value according to the classification information given by the camera. Considering the relatively dense characteristics of the lidar point cloud, the average value of all point cloud coordinates is used to represent the coordinate value of this target.

[0153] (6) As mentioned above, when adopting the "tight coupling strategy", we fully trust the detection of the camera in target detection, fully trust the speed information provided by the millimeter-wave radar in terms of speed, and fully trust the lidar point cloud information in terms of ranging. Therefore, only the coordinate values in step (5) are used to update the coordinate value of this target.

[0154] (7) In step (3), if the judgment is negative, it directly enters this step, that is, the "loose fusion" strategy, which means simply using the fusion result given by the roadside unit for blind area compensation and reminder.

[0155] This fusion method follows the principles of reducing computational amount, improving accuracy, and effectively compensating for blind areas in design. In addition, it also effectively improves the reuse rate of equipment and the stability of the system.

[0156] In some embodiments, the method further includes: controlling the host vehicle based on the target detection information. Controlling the host vehicle by using the fusion result of the roadside device and the vehicle-side device, that is, the target detection information, can improve the control accuracy and ensure the safety of intelligent driving.

[0157] In this embodiment, a data fusion device based on vehicle-road collaboration is also provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and / or hardware that realizes a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0158] This embodiment provides a data fusion device based on vehicle-road collaboration, as Figure 6 shown, including:

[0159] A receiving module 41, configured to receive target information of a preset area sent by a roadside device, where the target information includes the position and speed of a target;

[0160] An obtaining module 42, configured to obtain first point cloud data of a target detected by the host vehicle within a current detection area;

[0161] A first determination module 43, configured to determine a fusion strategy for the target information and the first point cloud data based on the positional relationship between the position of the host vehicle and the position of the preset area;

[0162] A second determination module 44, configured to fuse the target information and the first point cloud data according to the fusion strategy to determine target detection information within the current detection area.

[0163] In some embodiments, the second determination module 44 includes:

[0164] A first obtaining unit, configured to obtain the position of the target in the target information;

[0165] A first determination unit, configured to determine the correspondence between the target and the detected target based on the positional relationship between the position of the target and the position represented by the first point cloud data;

[0166] A fusion unit, configured to fuse the target information and the first point cloud data based on the fusion strategy and the correspondence to determine target detection information within the current detection area.

[0167] In some embodiments, when the position of the host vehicle is within the preset area, the fusion unit includes:

[0168] A first determination subunit, configured to determine, by using the corresponding relationship, a target that is detected by both the roadside device and the vehicle itself, so as to obtain first target point cloud data corresponding to the simultaneously detected target, where the position accuracy of the first target point cloud data is higher than the position accuracy in the target information;

[0169] A calculation subunit, configured to calculate the target position of the simultaneously detected target by using the point cloud coordinates in the first target point cloud data;

[0170] A second determination subunit, configured to determine, based on the target position and the speed of the simultaneously detected target, target detection information in the current detection area, where the speed accuracy in the target information is higher than the speed accuracy in the first point cloud data.

[0171] In some embodiments, when the position of the vehicle itself is outside the preset area, the fusion unit includes:

[0172] A third determination subunit, configured to determine, by using the corresponding relationship, a missed detection target that is detected by the roadside device and not detected by the vehicle itself, so as to obtain the target information of the missed detection target;

[0173] A fourth determination subunit, configured to determine, based on the target information of the missed detection target and the first point cloud data of the detected target, target detection information in the current detection area.

[0174] In some embodiments, the determination module of the target information includes:

[0175] A second acquisition unit, configured to acquire image detection data and second point cloud data of the preset area;

[0176] An identification unit, configured to perform target identification on the image detection data to determine an interested area corresponding to the target in the preset area;

[0177] A second determination unit, configured to determine, based on the positional relationship between the interested area corresponding to the target and the point cloud in the second point cloud data, second target point cloud data corresponding to the target;

[0178] A third determination unit, configured to determine the target information based on the second target point cloud data.

[0179] In some embodiments, the third determination unit includes:

[0180] A screening subunit, configured to screen out the target point cloud closest to the target from the second target point cloud data;

[0181] A fifth determination subunit, configured to determine the position and speed of the target point cloud as the position and speed of the target;

[0182] In some embodiments, the third determination unit includes:

[0183] A clustering subunit, configured to perform clustering processing on the second target point cloud data to determine a target point cloud;

[0184] A sixth determination subunit, configured to determine the position and speed of the target point cloud as the position and speed of the target.

[0185] The data fusion device based on vehicle-road cooperation in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0186] The further function descriptions of the above-mentioned respective modules are the same as those in the corresponding embodiments above, and will not be elaborated here.

[0187] An embodiment of the present invention further provides an electronic device having the above-mentioned Figure 6 shown data fusion device based on vehicle-road cooperation.

[0188] Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of an electronic device provided by an optional embodiment of the present invention. As Figure 7 shown, the electronic device may include: at least one processor 51, such as a CPU (Central Processing Unit, central processor), at least one communication interface 53, a memory 54, and at least one communication bus 52. Among them, the communication bus 52 is used to realize the connection and communication between these components. Among them, the communication interface 53 may include a display screen (Display), a keyboard (Keyboard), and optionally the communication interface 53 may further include a standard wired interface and a wireless interface. The memory 54 may be a high-speed RAM memory (Random Access Memory, volatile random access memory), or a non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory 54 may further be at least one storage device located far from the aforementioned processor 51. Among them, the processor 51 may be combined with Figure 6 the described device, the memory 54 stores an application program, and the processor 51 calls the program code stored in the memory 54 to be used for executing any of the above method steps.

[0189] Among them, the communication bus 52 can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus 52 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 7 it is only represented by a thick line in Figure 7 , but it does not mean that there is only one bus or one type of bus.

[0190] Among them, the memory 54 can include volatile memory, such as random-access memory (RAM); the memory can also include non-volatile memory, such as flash memory, hard disk drive (HDD), or solid-state drive (SSD); the memory 54 can also include a combination of the above types of memories.

[0191] Among them, the processor 51 can be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP.

[0192] Among them, the processor 51 can further include a hardware chip. The above hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0193] Optionally, the memory 54 is further configured to store program instructions. The processor 51 can call the program instructions to implement the data fusion method based on vehicle-road collaboration as shown in any embodiment of the present application.

[0194] An embodiment of the present invention also provides a non-transitory computer storage medium. The computer storage medium stores computer-executable instructions, and these computer-executable instructions can execute the data fusion method based on vehicle-road collaboration in any of the above method embodiments. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memories.

[0195] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A data fusion method based on vehicle-road cooperation, characterized in that Including: Receiving target information of a preset area sent by a roadside device, where the target information includes the position and speed of a target; Obtaining first point cloud data of a detected target of the vehicle itself within a current detection area; Based on the positional relationship between the position of the vehicle itself and the position of the preset area, determining a fusion strategy for the target information and the first point cloud data; Fusing the target information and the first point cloud data according to the fusion strategy to determine target detection information within the current detection area; The fusing the target information and the first point cloud data according to the fusion strategy to determine target detection information within the current detection area includes: Obtaining the position of the target in the target information; Based on the positional relationship between the position of the target and the position represented by the first point cloud data, determining the corresponding relationship between the target and the detected target; Based on the fusion strategy and the corresponding relationship, fusing the target information and the first point cloud data to determine target detection information within the current detection area; When the position of the vehicle itself is within the preset area, the fusing the target information and the first point cloud data according to the fusion strategy and the corresponding relationship to determine target detection information within the current detection area includes: Using the corresponding relationship to determine targets simultaneously detected by the roadside device and the vehicle itself to obtain first target point cloud data corresponding to the simultaneously detected targets, where the position accuracy of the first target point cloud data is higher than the position accuracy in the target information; Calculating the target position of the simultaneously detected target using the point cloud coordinates in the first target point cloud data; Based on the target position and the speed of the simultaneously detected target, determining target detection information within the current detection area, where the speed accuracy in the target information is higher than the speed accuracy in the first point cloud data; When the position of the vehicle itself is outside the preset area, the fusing the target information and the first point cloud data according to the fusion strategy and the corresponding relationship to determine target detection information within the current detection area includes: Using the corresponding relationship to determine undetected targets detected by the roadside device but not by the vehicle itself to obtain target information of the undetected targets; Based on the target information of the undetected targets and the first point cloud data of the detected targets, determining target detection information within the current detection area.

2. The method according to claim 1, characterized in that, The determination method of the target information includes: Obtaining image detection data and second point cloud data of the preset area; Performing target recognition on the image detection data to determine an interested area corresponding to the target within the preset area; Based on the positional relationship between the interested area corresponding to the target and the point clouds in the second point cloud data, determining second target point cloud data corresponding to the target; Based on the second target point cloud data, determining the target information.

3. The method according to claim 2, wherein The determining the target information based on the second target point cloud data includes: Screening out the target point cloud closest to the target from the second target point cloud data; Determining the position and speed of the target point cloud as the position and speed of the target.

4. The method according to claim 2, wherein Determining the target information based on the second target point cloud data includes: Performing clustering processing on the second target point cloud data to determine the target point cloud; Determining the position and speed of the target point cloud as the position and speed of the target.

5. A data fusion device based on vehicle-road cooperation, characterized in that, Including: A receiving module, configured to receive target information of a preset area sent by a roadside device, where the target information includes the position and speed of a target; An obtaining module, configured to obtain first point cloud data of a target detected by the vehicle itself in a current detection area; A first determination module, configured to determine a fusion strategy for the target information and the first point cloud data based on the positional relationship between the position of the vehicle itself and the position of the preset area; A second determination module, configured to fuse the target information and the first point cloud data according to the fusion strategy to determine target detection information in the current detection area; The fusing the target information and the first point cloud data according to the fusion strategy to determine target detection information in the current detection area includes: obtaining the position of the target in the target information; determining the correspondence between the target and the detected target based on the positional relationship between the position of the target and the position characterized by the first point cloud data; and fusing the target information and the first point cloud data based on the fusion strategy and the correspondence to determine target detection information in the current detection area; When the position of the vehicle itself is within the preset area, the fusing the target information and the first point cloud data based on the fusion strategy and the correspondence to determine target detection information in the current detection area includes: using the correspondence to determine a target simultaneously detected by the roadside device and the vehicle itself to obtain first target point cloud data corresponding to the simultaneously detected target, where the position accuracy of the first target point cloud data is higher than the position accuracy in the target information; calculating the target position of the simultaneously detected target using the point cloud coordinates in the first target point cloud data; and determining target detection information in the current detection area based on the target position and the speed of the simultaneously detected target, where the speed accuracy in the target information is higher than the speed accuracy in the first point cloud data; When the position of the vehicle itself is outside the preset area, the fusing the target information and the first point cloud data based on the fusion strategy and the correspondence to determine target detection information in the current detection area includes: using the correspondence to determine a missed detection target detected by the roadside device but not detected by the vehicle itself to obtain target information of the missed detection target; and determining target detection information in the current detection area based on the target information of the missed detection target and the first point cloud data of the detected target.

6. An electronic device, characterized in that, Including: A memory and a processor, where the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the vehicle-road collaborative data fusion method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to execute the vehicle-road collaborative data fusion method according to any one of claims 1-4.

8. A vehicle-road collaborative system, characterized in that, Including: A roadside device, including an image acquisition device, a first point cloud acquisition device, and a first computing platform. The first computing platform is used to perform data fusion based on the image detection data collected by the image acquisition device and the second point cloud data collected by the first point cloud acquisition device, and determine target information, where the target information includes the position and speed of a target within a preset area; A vehicle-side device, including a second point cloud acquisition device and a second computing platform. The second point cloud acquisition device is used to obtain the first point cloud data of a detection target of the vehicle itself within the current detection area. The second computing platform is connected to the first computing platform and is used to execute the vehicle-road collaborative data fusion method according to any one of claims 1-4.

Images