Lidar-based multi-vehicle platooning perception method and device for intelligent connected public transportation
By analyzing the lidar data and environmental data of intelligent connected buses, using pseudo-image processing and 2D target detection network to determine the three-dimensional coordinates of the front vehicle target, the problem of inaccurate perception of target points during the docking process of intelligent connected buses is solved. Reliable perception and high-precision detection in complex scenarios are achieved.
Patent Information
- Application Number
- CN202410433936.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-11
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-04-11
AI Technical Summary
During the intelligent connected bus docking process, the target on the front vehicle may be blocked by other vehicles or obstacles, resulting in inaccurate perception of the target position or failure, and incorrect target detection or tracking may occur, resulting in the occurrence of the docking process. Problem or failure.
By obtaining the lidar data of the multi-vehicle formation of intelligent connected buses, the environmental data around the vehicle is analyzed, the three-dimensional coordinates and heading angles of the target point in front of the vehicle are determined, and the backbone and detection head of the network are used for pseudo-image processing and 2D target detection network Perform detection to generate multi-vehicle formation perception results for intelligent connected buses.
Provide reliable results for multi-vehicle docking scenarios in complex scenarios, with fast computing speed, enhances the robustness of the system, improves the accuracy of detection, and solves the problem of inaccurate or failure of target position perception.
Smart Images

Figure CN118330666B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving technology, and in particular to a laser radar-based method and device for sensing a multi-vehicle formation of intelligent networked buses. Background Art
[0002] With the continuous development of urban transportation and the improvement of its intelligence level, public transportation systems are becoming increasingly important in solving problems such as urban traffic congestion and environmental pollution. Among them, intelligent connected buses, as a new type of transportation, have received more and more attention and favor from cities.
[0003] Among the related technologies, lidar sensing technology is mainly used to obtain environmental information around the vehicle in real time. Intelligent connected buses can realize accurate monitoring and control of the movement status of vehicle formations, thereby effectively improving the safety and stability of the fleet's driving.
[0004] However, in the related technology, during the docking process of the intelligent connected bus, the target on the front vehicle may be blocked by other vehicles or obstacles, resulting in inaccurate or failed perception of the target point position, and erroneous target detection or tracking may occur, causing problems or failures in the docking process, which urgently needs to be improved. Summary of the invention
[0005] The present application provides a method and device for sensing a multi-vehicle formation of an intelligent connected bus based on laser radar, in order to solve the problem in the related art that during the docking process of the intelligent connected bus, the target on the front vehicle may be blocked by other vehicles or obstacles, resulting in inaccurate or failed perception of the target point position, and erroneous target detection or tracking may occur, causing problems or failures in the docking process.
[0006] The first aspect of the present application provides a method for perceiving a multi-vehicle platoon of an intelligent connected bus based on a laser radar, comprising the following steps: acquiring laser radar data of a multi-vehicle platoon of an intelligent connected bus; obtaining environmental data around vehicles in the multi-vehicle platoon based on the laser radar data analysis; determining the three-dimensional coordinates and heading angle of a target point on a leading vehicle based on the environmental data around the vehicles, and generating a multi-vehicle platoon perception result of the intelligent connected bus.
[0007] Optionally, in one embodiment of the present application, the environmental data around the vehicles of the multi-vehicle formation is obtained based on the laser radar data analysis, including: time synchronization of the laser radar data to obtain images, point clouds and RTK (Real Time Kinematic) information of the front and rear vehicles at the same time; data cleaning and filtering of the images, point clouds and RTK positioning information of the front and rear vehicles at the same time to obtain filtered data that meets preset conditions; establishing a correspondence between the image and the point cloud based on the filtered data, and obtaining the true value label of the data based on the RTK positioning information of the front and rear vehicles to generate the environmental data.
[0008] Optionally, in one embodiment of the present application, determining the three-dimensional coordinates and heading angle of the target point on the preceding vehicle according to the environmental data around the vehicle to generate the multi-vehicle platoon perception result of the intelligent connected bus includes: performing pseudo-image processing based on the environmental data to obtain a pseudo-image; detecting the pseudo-image using a preset 2D target detection network backbone and a detection head to obtain pseudo-image features, and generating the three-dimensional coordinates and heading angle of the target point according to the pseudo-image features.
[0009] Optionally, in one embodiment of the present application, the pseudo image processing is performed based on the environmental data to obtain the pseudo image, including: voxelizing the front area detected by the lidar to classify the points in the point cloud into corresponding voxels to obtain multiple points; expanding the data dimension of each of the multiple points to obtain expanded voxels; and extracting voxel features of the expanded voxels to form the pseudo image.
[0010] Optionally, in one embodiment of the present application, the pseudo image is detected using a preset 2D target detection network backbone and a detection head to obtain pseudo image features, and the three-dimensional coordinates and heading angle of the target point are generated according to the pseudo image features, including: based on a preset pyramid network, extracting the features of the pseudo image layer by layer from top to bottom to obtain feature maps of multiple scales in the pseudo image features; decoding the feature maps of each scale of the multiple scales and splicing them at the channel scale to generate a result of the target dimension.
[0011] Optionally, in one embodiment of the present application, the pseudo image is detected using a preset 2D target detection network backbone and a detection head to obtain pseudo image features, and the three-dimensional coordinates and heading angle of the target point are generated based on the pseudo image features, and further includes: based on the results, generating a corresponding detection head for each voxel coordinate; and obtaining the three-dimensional coordinates and heading angle of the target point according to the detection head.
[0012] The second aspect of the present application provides a laser radar-based intelligent connected bus multi-vehicle formation perception device, including: an acquisition module, used to acquire laser radar data of a multi-vehicle formation of an intelligent connected bus; an analysis module, used to obtain environmental data around vehicles in the multi-vehicle formation based on the laser radar data analysis; a generation module, used to determine the three-dimensional coordinates and heading angle of a target point on a leading vehicle based on the environmental data around the vehicle, and generate a multi-vehicle formation perception result of the intelligent connected bus.
[0013] Optionally, in one embodiment of the present application, the analysis module includes: a first acquisition unit, used to time synchronize the lidar data to obtain images, point clouds and real-time differential positioning RTK information of the front and rear vehicles at the same moment; a screening unit, used to clean and screen the images, point clouds and RTK positioning information of the front and rear vehicles at the same moment to obtain filtered data that meets preset conditions; a first generation unit, used to establish a correspondence between the image and the point cloud based on the filtered data, and obtain the true value label of the data based on the RTK positioning information of the front and rear vehicles to generate the environmental data.
[0014] Optionally, in one embodiment of the present application, the generation module includes: a processing unit, used to perform pseudo-image processing based on the environmental data to obtain a pseudo-image; a second generation unit, used to detect the pseudo-image using a preset 2D target detection network backbone and a detection head to obtain pseudo-image features, and generate the three-dimensional coordinates and heading angle of the target point based on the pseudo-image features.
[0015] Optionally, in one embodiment of the present application, the processing unit includes: a processing subunit, used to perform voxelization on the front area detected by the laser radar, so as to classify the points in the point cloud into corresponding voxels to obtain multiple points; an expansion subunit, used to expand the data dimension of each point of the multiple points to obtain expanded voxels; a first extraction subunit, used to extract voxel features of the expanded voxels to form the pseudo image.
[0016] Optionally, in one embodiment of the present application, the second generation unit includes: a second extraction subunit, used to extract the features of the pseudo image layer by layer from top to bottom based on a preset pyramid network to obtain feature maps of multiple scales in the pseudo image features; a generation subunit, used to decode and splice the feature maps of each scale of the feature maps of the multiple scales at the channel scale to generate a result of the target dimension.
[0017] Optionally, in one embodiment of the present application, the second generation unit further includes: a third generation unit, used to generate a corresponding detection head for each voxel coordinate based on the result; and a second acquisition unit, used to acquire the three-dimensional coordinates and heading angle of the target point according to the detection head.
[0018] The third aspect of the present application provides a vehicle, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the laser radar-based intelligent connected bus multi-vehicle formation perception method as described in the above embodiment.
[0019] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned laser radar-based intelligent connected bus multi-vehicle formation perception method.
[0020] The fifth aspect of the present application provides a computer program product, which, when executed, is used to implement the above-mentioned laser radar-based intelligent connected bus multi-vehicle formation perception method.
[0021] The embodiment of the present application can obtain the laser radar data of a multi-vehicle formation of an intelligent networked bus, and obtain the environmental data around the vehicles in the multi-vehicle formation based on the laser radar data analysis, so as to determine the three-dimensional coordinates and heading angle of the target point on the front vehicle, generate the multi-vehicle formation perception result of the intelligent networked bus, and provide reliable results for multi-vehicle docking scenarios in complex scenarios. It has a faster computing speed, enhances the robustness of the system, and improves the accuracy of detection. Therefore, it solves the problem in the related art that during the docking process of the intelligent networked bus, the target on the front vehicle may be blocked by other vehicles or obstacles, resulting in inaccurate or failed perception of the target point position, and may cause erroneous target detection or tracking, resulting in problems or failures in the docking process.
[0022] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0024] Figure 1 A flowchart of a method for sensing a multi-vehicle formation of an intelligent networked bus based on a laser radar according to an embodiment of the present application;
[0025] Figure 2This is a schematic diagram of an environment for the practical application of a laser radar-based intelligent networked bus multi-vehicle platoon perception method according to an embodiment of the present application;
[0026] Figure 3 A flowchart for data production according to an embodiment of the present application;
[0027] Figure 4 is a schematic diagram of a point cloud according to an embodiment of the present application;
[0028] Figure 5 A schematic diagram of a 3D frame visualization effect according to an embodiment of the present application;
[0029] Figure 6 A flowchart of a 2D object detection network backbone and a detection head according to an embodiment of the present application;
[0030] Figure 7 This is a flowchart of a method for sensing a multi-vehicle formation of an intelligent connected bus based on a laser radar according to an embodiment of the present application;
[0031] Figure 8 It is a structural schematic diagram of a laser radar-based intelligent networked bus multi-vehicle formation perception device according to an embodiment of the present application;
[0032] Fig. 9 It is a schematic diagram of the structure of a vehicle provided according to an embodiment of the present application. DETAILED DESCRIPTION
[0033] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0034] The following describes the method and device for sensing multi-vehicle formation of intelligent connected buses based on laser radar according to the embodiment of the present application with reference to the accompanying drawings. In view of the related technologies mentioned in the background technology center above, since the target on the front vehicle may be blocked by other vehicles or obstacles during the docking process of the intelligent connected bus, resulting in inaccurate or failed perception of the target point position, erroneous target detection or tracking may occur, causing problems or failures in the docking process. The present application provides a method for sensing multi-vehicle formation of intelligent connected buses based on laser radar, in which the laser radar data of the multi-vehicle formation of the intelligent connected bus can be obtained, and the environmental data around the vehicles in the multi-vehicle formation can be obtained according to the laser radar data analysis, so that the three-dimensional coordinates and heading angle of the target point on the front vehicle can be determined, and the multi-vehicle formation sensing result of the intelligent connected bus can be generated, which can provide reliable results for the multi-vehicle docking scene in complex scenes, has a faster computing speed, enhances the robustness of the system, and improves the accuracy of detection. Thereby, a problem in the related technology is solved that during the docking process of the intelligent connected bus, the target on the front vehicle may be blocked by other vehicles or obstacles, resulting in inaccurate or failed perception of the target point position, erroneous target detection or tracking, and causing problems or failures in the docking process.
[0035] Specifically, Figure 1 A schematic flow chart of a laser radar-based intelligent connected bus multi-vehicle platoon perception method provided in an embodiment of the present application.
[0036] like Figure 1 As shown, the laser radar-based intelligent networked bus multi-vehicle formation perception method includes the following steps:
[0037] In step S101, laser radar data of a multi-vehicle formation of an intelligent connected bus is obtained.
[0038] It can be understood that lidar data refers to the lidar measuring the distance and shape of the surrounding environment by emitting laser beams and receiving reflected light signals, and generating corresponding data.
[0039] Specifically, combined Figure 2 and Figure 3 As shown, the embodiment of the present application can obtain the laser radar data of a multi-vehicle formation of an intelligent connected bus through a laser radar, including distance information, intensity information, scanning frequency, etc., and as shown in step S301, the embodiment of the present application can store the collected data in a bag format, wherein the data record stores message data published on different topics.
[0040] The embodiment of the present application obtains the lidar data of a multi-vehicle formation of an intelligent connected bus, so as to scan the surrounding environment in real time, detect the position, shape and distance of obstacles, help vehicles avoid obstacles, and improve the driving efficiency and safety of the fleet.
[0041] In step S102, environmental data around vehicles in a multi-vehicle convoy is obtained based on laser radar data analysis.
[0042] It can be understood that the environmental data around the vehicle refers to various information about the vehicle's surrounding environment obtained through lidar, including key data such as the distance, speed, and direction between vehicles.
[0043] Through real-time analysis of lidar data, the multi-vehicle platooning system of the embodiment of the present application can obtain various data about the vehicle's surrounding environment, and can combine the distance, speed, direction and other information in the lidar data to accurately obtain the distance, speed, direction and other data between vehicles, thereby providing a basis for the subsequent fleet formation of the intelligent vehicle system, helping to achieve safe and efficient platoon driving, and improving the performance and reliability of the intelligent transportation system.
[0044] Optionally, in one embodiment of the present application, environmental data around vehicles in a multi-vehicle formation is obtained based on lidar data analysis, including: time synchronization of lidar data to obtain images, point clouds, and real-time differential positioning RTK information of leading and trailing vehicles at the same moment; data cleaning and screening of images, point clouds, and leading and trailing vehicles RTK positioning information at the same moment to obtain screened data that meets preset conditions; establishing a correspondence between images and point clouds based on the screened data, and obtaining the true value label of the data based on the leading and trailing vehicles RTK positioning information to generate environmental data.
[0045] It can be understood that preset conditions refer to the conditions that need to be set during the data cleaning and screening process to ensure that the final screened data meets the expected standards. For example, setting a timestamp deviation threshold determines the maximum difference allowed between timestamps of two different data sources, that is, the acceptable timestamp error range.
[0046] Specifically, combined Figure 3 and Figure 4As shown, in order to ensure the accuracy of the data, the embodiment of the present application can, as shown in step S302, time synchronize the original information to obtain the image, point cloud, and RTK positioning information of the front and rear vehicles at the same time, and as shown in step S303, the embodiment of the present application can perform data cleaning to remove the data that is in the opposite direction of the front vehicle or is completely occluded. For example, through timestamp synchronization, it is ensured that the image, point cloud, and RTK positioning information of the front and rear vehicles are obtained at the same time, and then by analyzing the vehicle in front of the vehicle, when it is determined that the movement direction of the front vehicle is opposite to that of the vehicle, the data of the front vehicle moving in the opposite direction can be filtered out according to the speed and position prediction of the front vehicle; or when there are traces of occlusion in the point cloud data and the RTK positioning information, the occlusion area can be detected through the point cloud data, and the data within the occlusion area can be marked and eliminated to ensure accurate environmental perception and positioning information.
[0047] Furthermore, the embodiment of the present application can number the screened samples to establish a one-to-one correspondence between the image and the point cloud data, and as shown in step S304, can utilize the corresponding RTK positioning information of the front and rear vehicles, combined with the radar RTK joint calibration matrix and the digital-analog parameters of the minibus, so that, as shown in step S305, through the conversion of the point position and the coordinate system, the 3D box information of the front vehicle in the rear vehicle lidar coordinate system can be directly calculated and used as the true value label of the data.
[0048] The embodiment of the present application can establish a correspondence between images and point clouds based on the filtered data, and obtain the true value label of the data based on the RTK positioning information of the front and rear vehicles to generate environmental data, which is conducive to improving the accuracy and comprehensiveness of the data and providing an information basis for subsequent processing.
[0049] In step S103, the three-dimensional coordinates and heading angle of the target point on the preceding vehicle are determined according to the environmental data surrounding the vehicle, and a multi-vehicle platoon perception result of the intelligent connected bus is generated.
[0050] Specifically, the embodiments of the present application can determine the spatial position of the target point in the vehicle coordinate system and the heading angle of the preceding vehicle based on various information about the vehicle's surrounding environment acquired by the laser radar, including key data such as the distance, speed, and direction between vehicles, thereby generating multi-vehicle formation perception results for intelligent connected buses, including the accurate relative position relationship and heading angle between each vehicle.
[0051] The embodiment of the present application determines the three-dimensional coordinates and heading angle of the target point on the leading vehicle based on the environmental data around the vehicle, and generates a multi-vehicle platoon perception result of the intelligent connected bus, which is conducive to improving the accuracy of the perception result, providing an accurate data basis for multi-vehicle platoon control, improving the safety of the platoon driving, and reducing potential collision risks.
[0052] Optionally, in one embodiment of the present application, the three-dimensional coordinates and heading angle of the target point on the leading vehicle are determined based on the environmental data around the vehicle, and the multi-vehicle formation perception result of the intelligent connected bus is generated, including: performing pseudo-image processing based on the environmental data to obtain a pseudo-image; using a preset 2D target detection network backbone and detection head to detect the pseudo-image to obtain pseudo-image features, and generating the three-dimensional coordinates and heading angle of the target point based on the pseudo-image features.
[0053] It can be understood that pseudo-image features refer to information describing environmental features extracted after converting and processing non-image environmental data, including the location, shape, size, color, etc. of the target.
[0054] The embodiments of the present application use pseudo-image processing to help intuitively understand the vehicle's surroundings and enhance environmental perception capabilities. By utilizing a 2D target detection network, the efficiency of target detection can be improved. By calculating the three-dimensional coordinates and heading angle of the target point, the target can be accurately positioned in three-dimensional space, thereby improving the recognition accuracy of pseudo-image features and enhancing detection performance.
[0055] Optionally, in one embodiment of the present application, pseudo-image processing is performed based on environmental data to obtain a pseudo-image, including: voxelizing the front area detected by the lidar to classify the points in the point cloud into corresponding voxels to obtain multiple points; expanding the data dimension of each of the multiple points to obtain expanded voxels; and extracting voxel features of the expanded voxels to form a pseudo-image.
[0056] Specifically, the embodiment of the present application can voxelize the front area detected by the laser radar, dividing the space into voxels of uniform size, wherein the top view of each voxel is a square with a side length of 16 cm, and the height is regarded as infinite. Then, the points in the 3D point cloud can be classified into corresponding voxels according to their positions, and since each voxel may contain a different number of 3D points, each voxel can be limited to 32 points, and the excess points are sampled, and the insufficient points are zero-filled. In order to ensure the diversity of the results, a random voxelization method can be used for processing. At each voxelization, some points will be randomly eliminated, so the results of each voxelization may be slightly different.
[0057] Furthermore, the embodiment of the present application can expand the data dimension of each point to 10-dimensional data, including (x, y, z, i, x_c, y_c, z_c, x_p, y_p, z_p), where (x, y, z, i) are the 3D coordinates and intensity of the point, (x_c, y_c, z_c) are the coordinate difference between the point and the average point cloud data in the voxel where it is located, and (x_p, y_p, z_p) are the difference between the point and the center coordinates of the voxel where it is located. Thus, all voxels can be uniformly feature extracted and re-stretched into the shape of (C, H, W) to form a pseudo image, where C is the feature dimension, H and W are the height and width of the pseudo image, that is, the voxel grid dimension in the top view.
[0058] The embodiment of the present application can reduce the data dimension by voxelizing the point cloud data and expand the data dimension of each point, which is helpful for data compression and information extraction, and effectively improves data utilization. Therefore, by voxelizing and extracting features from the point cloud data, the data structure can be simplified and the data volume can be reduced, which helps to speed up the speed and efficiency of data processing.
[0059] Optionally, in one embodiment of the present application, a preset 2D target detection network backbone and a detection head are used to detect a pseudo image to obtain pseudo image features, and three-dimensional coordinates and heading angles of target points are generated based on the pseudo image features, including: based on a preset pyramid network, extracting pseudo image features layer by layer from top to bottom to obtain feature maps of multiple scales in the pseudo image features; decoding and splicing feature maps of each scale of the feature maps at the channel scale to generate a result of the target dimension.
[0060] Specifically, combined Figure 5 As shown, the embodiment of the present application can use the 2D target detection network backbone and the detection head to detect pseudo images. By utilizing the feature pyramid network, the features of the pseudo images can be extracted layer by layer from top to bottom, and feature maps of three scales of 1 / 2, 1 / 4, and 1 / 8 are formed. The feature maps of each scale are decoded, and the feature maps of the same size are spliced at the channel scale. The dimension of the final result is (6C, H / 2, W / 2).
[0061] The embodiment of the present application is based on a preset pyramid network, which can extract features at different scales, which is conducive to improving the comprehensiveness of the data. In addition, the decoding and feature splicing operations are combined to effectively improve the detection capability of targets of different scales, which helps to accurately and quickly detect targets and improve the accuracy of target positioning and recognition.
[0062] Optionally, in one embodiment of the present application, a preset 2D target detection network backbone and a detection head are used to detect a pseudo image to obtain pseudo image features, and three-dimensional coordinates and heading angles of the target points are generated based on the pseudo image features. The method also includes: based on the results, generating a corresponding detection head for each voxel coordinate; and obtaining the three-dimensional coordinates and heading angles of the target points based on the detection head.
[0063] Specifically, combined Figure 5 and Figure 6 As shown, in the embodiment of the present application, since the size of the object to be detected is fixed, only two directions of the prior frame need to be considered during detection, such as 0 degrees and 90 degrees under the BEV perspective. Therefore, for each voxel coordinate, two corresponding detection heads will be generated, wherein the detection head has three branches, including: conv_cls is used to predict the probability of the preceding vehicle, conv_box is used to predict seven-dimensional data (x, y, z, w, l, h, θ), which are respectively the offset between the 3D frame and the anchor center coordinate, the length, width and height of the 3D frame, and the heading angle of the 3D frame, and conv_dir_cls predicts two parameters for determining the direction of the 3D frame, wherein, since the prediction of θ cannot distinguish between two 3D frames with an angle difference of 180 degrees, for the generated 3D frames, the result with the highest score can be selected as the final output.
[0064] It is worth noting that when calculating the loss, the classification and orientation losses can use the cross entropy loss function, and the bounding box loss can use the Smooth L1 function, which helps to stabilize the bounding box regression process and reduce the impact of outliers.
[0065] The embodiment of the present application is based on generating a corresponding detection head for each voxel coordinate, which can achieve relatively accurate detection of the three-dimensional coordinates and heading angle of the target point in complex scenes. It has good accuracy and robustness, can improve the computing speed, and can provide reliable results for multi-vehicle docking scenarios.
[0066] Combination Figure 7 As shown, the laser radar-based intelligent connected bus multi-vehicle formation perception method of the embodiment of the present application is described in detail with an embodiment.
[0067] Specifically, the embodiment of the present application can realize the multi-vehicle formation perception of intelligent networked buses based on laser radar, wherein the specific steps are as follows:
[0068] Step S701: Clean and filter the images, point clouds and RTK positioning information of the front and rear vehicles at the same time, and establish a corresponding relationship between the images and point clouds based on the filtered data.
[0069] Step S702: voxelize the front area detected by the laser radar to classify the points in the point cloud into corresponding voxels to obtain multiple points.
[0070] Step S703: Expand the data dimension of each of the multiple points to obtain expanded voxels, and extract voxel features of the expanded voxels.
[0071] Step S704: By performing uniform feature extraction on all voxels, they can be re-stretched into a (C, H, W) shape to form a pseudo image.
[0072] Step S705: Use the 2D object detection network backbone to detect the pseudo image, and use the feature pyramid network to extract the features of the pseudo image layer by layer from top to bottom, and form feature maps of three scales of 1 / 2, 1 / 4, and 1 / 8. The feature maps of each scale are decoded, and the feature maps of the same size are spliced on the channel scale to finally generate the dimension of the result.
[0073] Step S706: Based on the result, a corresponding detection head is generated for each voxel coordinate; and the three-dimensional coordinates and heading angle of the target point are obtained according to the detection head.
[0074] Step S707: Generate a target 3D frame.
[0075] In summary, the embodiments of the present application can use lidar as a sensor, and determine the three-dimensional coordinates and heading angle of the target point on the front vehicle through deep learning related technologies. It can achieve relatively accurate detection of the three-dimensional coordinates and heading angle of the target point in complex scenarios, and has a faster computing speed. By utilizing lidar sensing technology, intelligent connected buses can achieve accurate monitoring and control of the movement status of vehicle formations, thereby effectively improving the safety and stability of the fleet's driving, and contributing to the intelligent development of urban transportation.
[0076] According to the laser radar-based intelligent networked bus multi-vehicle formation perception method proposed in the embodiment of the present application, the laser radar data of the multi-vehicle formation of the intelligent networked bus can be obtained, and the environmental data around the vehicles of the multi-vehicle formation can be obtained based on the laser radar data analysis, so that the three-dimensional coordinates and heading angle of the target point on the front vehicle can be determined, and the multi-vehicle formation perception result of the intelligent networked bus can be generated, which can provide reliable results for multi-vehicle docking scenarios in complex scenarios, has a faster computing speed, enhances the robustness of the system, and improves the accuracy of detection. Therefore, the problem in the related art that during the intelligent networked bus docking process, the target on the front vehicle may be blocked by other vehicles or obstacles, resulting in inaccurate or failed perception of the target point position, and erroneous target detection or tracking may occur, causing problems or failures in the docking process.
[0077] Next, a laser radar-based intelligent connected bus multi-vehicle platoon perception device proposed according to an embodiment of the present application is described with reference to the accompanying drawings.
[0078] Figure 8 It is a block diagram of a laser radar-based intelligent connected bus multi-vehicle formation perception device according to an embodiment of the present application.
[0079] like Figure 8 As shown, the laser radar-based intelligent connected bus multi-vehicle formation perception device 10 includes: an acquisition module 100, an analysis module 200 and a generation module 300.
[0080] Specifically, the acquisition module 100 is used to acquire the laser radar data of a multi-vehicle formation of an intelligent network-connected bus;
[0081] An analysis module 200 is used to obtain environmental data around vehicles in a multi-vehicle formation based on laser radar data analysis;
[0082] The generation module 300 is used to determine the three-dimensional coordinates and heading angle of the target point on the front vehicle according to the environmental data around the vehicle, and generate the multi-vehicle formation perception result of the intelligent connected bus.
[0083] Optionally, in one embodiment of the present application, the analysis module 200 includes: a first acquisition unit, a screening unit and a first generation unit.
[0084] The first acquisition unit is used to synchronize the laser radar data in time to obtain the image, point cloud and real-time differential positioning RTK information of the front and rear vehicles at the same time;
[0085] The screening unit is used to clean and screen the images, point clouds and RTK positioning information of the front and rear vehicles at the same time to obtain the screening data that meets the preset conditions;
[0086] The first generating unit is used to establish a corresponding relationship between the image and the point cloud according to the filtered data, and obtain the true value label of the data according to the RTK positioning information of the front and rear vehicles to generate environmental data.
[0087] Optionally, in one embodiment of the present application, the generation module 300 includes: a processing unit and a second generation unit.
[0088] Wherein, the processing unit is used to perform pseudo image processing based on the environmental data to obtain a pseudo image;
[0089] The second generating unit is used to detect the pseudo image using a preset 2D target detection network backbone and a detection head to obtain pseudo image features, and generate three-dimensional coordinates and heading angles of target points according to the pseudo image features.
[0090] Optionally, in one embodiment of the present application, the processing unit includes: a processing subunit, an expansion subunit and a first extraction subunit.
[0091] The processing subunit is used to voxelize the front area detected by the laser radar, so as to classify the points in the point cloud into corresponding voxels to obtain multiple points;
[0092] An expansion subunit, used for expanding the data dimension of each point of the plurality of points to obtain an expanded voxel;
[0093] The first extraction subunit is used to extract voxel features of the expanded voxels to form a pseudo image.
[0094] Optionally, in one embodiment of the present application, the second generating unit includes: a second extracting subunit and a generating subunit.
[0095] The second extraction subunit is used to extract the features of the pseudo image layer by layer from top to bottom based on a preset pyramid network to obtain feature maps of multiple scales in the pseudo image features;
[0096] The generation subunit is used to decode and concatenate the feature maps of each scale of the feature maps of multiple scales at the channel scale to generate a result of the target dimension.
[0097] Optionally, in one embodiment of the present application, the second generating unit further includes: a third generating unit and a second acquiring unit.
[0098] Wherein, the third generating unit is used to generate a corresponding detection head for each voxel coordinate based on the result;
[0099] The second acquisition unit is used to acquire the three-dimensional coordinates and heading angle of the target point according to the detection head.
[0100] It should be noted that the aforementioned explanation of the embodiment of the method for sensing multiple vehicle formations of intelligent connected buses based on laser radar is also applicable to the device for sensing multiple vehicle formations of intelligent connected buses based on laser radar of this embodiment, and will not be repeated here.
[0101] According to the laser radar-based intelligent networked bus multi-vehicle formation perception device proposed in the embodiment of the present application, the laser radar data of the multi-vehicle formation of the intelligent networked bus can be obtained, and the environmental data around the vehicles of the multi-vehicle formation can be obtained based on the laser radar data analysis, so that the three-dimensional coordinates and heading angle of the target point on the front vehicle can be determined, and the multi-vehicle formation perception results of the intelligent networked bus can be generated. It can provide reliable results for multi-vehicle docking scenarios in complex scenarios, has a faster computing speed, enhances the robustness of the system, and improves the accuracy of detection. Therefore, it solves the problem in the related art that during the docking process of the intelligent networked bus, the target on the front vehicle may be blocked by other vehicles or obstacles, resulting in inaccurate or failed perception of the target point position, and erroneous target detection or tracking may occur, causing problems or failures in the docking process.
[0102] Fig. 9 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. The vehicle may include:
[0103] A memory 901 , a processor 902 , and a computer program stored in the memory 901 and executable on the processor 902 .
[0104] When the processor 902 executes the program, the laser radar-based intelligent connected bus multi-vehicle formation perception method provided in the above embodiment is implemented.
[0105] Furthermore, the vehicle also includes:
[0106] The communication interface 903 is used for communication between the memory 901 and the processor 902 .
[0107] The memory 901 is used to store computer programs that can be executed on the processor 902 .
[0108] The memory 901 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0109] If the memory 901, the processor 902 and the communication interface 903 are implemented independently, the communication interface 903, the memory 901 and the processor 902 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0110] Optionally, in a specific implementation, if the memory 901, the processor 902 and the communication interface 903 are integrated on a chip, the memory 901, the processor 902 and the communication interface 903 can communicate with each other through an internal interface.
[0111] The processor 902 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.
[0112] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned laser radar-based intelligent connected bus multi-vehicle formation perception method.
[0113] An embodiment of the present application also provides a computer program product, which can run computer instructions. When the computer instructions are executed by a processor, the above-mentioned laser radar-based intelligent connected bus multi-vehicle formation perception method is implemented.
[0114] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0115] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0116] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0117] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.
[0118] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0119] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0120] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0121] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A LiDAR-based multi-vehicle platoon perception method for intelligent connected buses. It is characterized in that The following steps are involved: Obtain LiDAR data for multi-vehicle platooning of intelligent connected buses; Obtaining environmental data around vehicles in the multi-vehicle formation according to the laser radar data analysis; Determine the three-dimensional coordinates and heading angle of the target point on the preceding vehicle according to the environmental data around the vehicle, and generate a multi-vehicle formation perception result of the intelligent network-connected bus; The step of obtaining the environmental data around the vehicles of the multi-vehicle formation according to the laser radar data analysis includes: performing time synchronization on the laser radar data to obtain images, point clouds and real-time differential positioning RTK information of the front and rear vehicles at the same time; performing data cleaning and screening on the images, point clouds and RTK positioning information of the front and rear vehicles at the same time to obtain screening data that meets preset conditions; establishing a corresponding relationship between the images and the point clouds according to the screening data, and obtaining the true value label of the data according to the RTK positioning information of the front and rear vehicles to generate the environmental data; Among them, the determining of the three-dimensional coordinates and heading angle of the target point on the preceding vehicle according to the environmental data around the vehicle and generating the multi-vehicle formation perception result of the intelligent network-connected bus includes: performing pseudo-image processing based on the environmental data to obtain a pseudo-image; detecting the pseudo-image using a preset 2D target detection network backbone and a detection head to obtain pseudo-image features, and generating the three-dimensional coordinates and heading angle of the target point according to the pseudo-image features; The pseudo image processing based on the environmental data to obtain the pseudo image includes: voxelizing the front area detected by the laser radar to classify the points in the point cloud into corresponding voxels to obtain multiple points; expanding the data dimension of each of the multiple points to obtain an expanded voxel; extracting the voxel features of the expanded voxel to form the pseudo image; The method of detecting the pseudo image using a preset 2D target detection network backbone and a detection head to obtain pseudo image features, and generating the three-dimensional coordinates and heading angle of the target point according to the pseudo image features, includes: extracting the features of the pseudo image layer by layer from top to bottom based on a preset pyramid network to obtain feature maps of multiple scales in the pseudo image features; decoding the feature maps of each scale of the feature maps of the multiple scales and splicing them at the channel scale to generate a result of the target dimension; Among them, the pseudo image is detected using a preset 2D target detection network backbone and a detection head to obtain pseudo image features, and the three-dimensional coordinates and heading angle of the target point are generated according to the pseudo image features. It also includes: based on the results, a corresponding detection head is generated for each voxel coordinate; and the three-dimensional coordinates and heading angle of the target point are obtained according to the detection head.
2. A laser radar-based intelligent connected bus multi-vehicle platoon perception device, It is characterized in that include: Acquisition module, used to acquire the lidar data of multiple vehicles in a platoon of intelligent connected buses; An analysis module, configured to obtain environmental data around the vehicles of the multi-vehicle formation according to the laser radar data analysis; A generation module, used to determine the three-dimensional coordinates and heading angle of the target point on the preceding vehicle according to the environmental data around the vehicle, and generate a multi-vehicle formation perception result of the intelligent network-connected bus; The analysis module includes: a first acquisition unit, which is used to synchronize the laser radar data in time to obtain the image, point cloud and real-time differential positioning RTK information of the front and rear vehicles at the same time; a screening unit, which is used to clean and screen the image, point cloud and RTK positioning information of the front and rear vehicles at the same time to obtain the screening data that meets the preset conditions; a first generation unit, which is used to establish the corresponding relationship between the image and the point cloud according to the screening data, and obtain the true value label of the data according to the RTK positioning information of the front and rear vehicles to generate the environmental data; The generation module includes: a processing unit, which is used to perform pseudo image processing based on the environmental data to obtain a pseudo image; a second generation unit, which is used to detect the pseudo image using a preset 2D target detection network backbone and a detection head to obtain pseudo image features, and generate the three-dimensional coordinates and heading angle of the target point according to the pseudo image features; The processing unit includes: a processing subunit, which is used to voxelize the front area detected by the laser radar, so as to classify the points in the point cloud into corresponding voxels to obtain multiple points; an expansion subunit, which is used to expand the data dimension of each point of the multiple points to obtain an expanded voxel; a first extraction subunit, which is used to extract the voxel features of the expanded voxel to form the pseudo image; The second generating unit includes: a second extracting subunit, which is used to extract the features of the pseudo image layer by layer from top to bottom based on a preset pyramid network to obtain feature maps of multiple scales in the pseudo image features; a generating subunit, which is used to decode and splice the feature maps of each scale of the feature maps of the multiple scales at the channel scale to generate a result of the target dimension; Among them, the second generating unit also includes: a third generating unit, used to generate a corresponding detection head for each voxel coordinate based on the result; and a second acquiring unit, used to acquire the three-dimensional coordinates and heading angle of the target point according to the detection head.
3. A vehicle, It is characterized in that include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the laser radar-based intelligent connected bus multi-vehicle formation perception method as claimed in claim 1.
4. A computer-readable storage medium having a computer program stored thereon, It is characterized in that The program is executed by a processor to implement the laser radar-based intelligent connected bus multi-vehicle platoon perception method as described in claim 1.
5. A computer program product, It is characterized in that When the computer program is executed, it is used to implement the laser radar-based intelligent connected bus multi-vehicle formation perception method as claimed in claim 1.
