Target vehicle detection system and method for closed highways

Abstract

The present invention discloses a target vehicle detection system for closed highways, comprising a data acquisition module, a point cloud data processing module, an image data processing module, and a point cloud data and image data fusion module; the point cloud data and image data fusion module utilizes lane lines, target point cloud shape, and target distance information when fusing, and fuses them within the point cloud space, thereby solving the problem of multiple close targets not being correctly matched and farther targets not being matched. The present invention also provides a target vehicle detection method for closed highways. The present invention effectively improves the accuracy of target vehicle detection while shortening the target vehicle detection time, and is more suitable for unmanned vehicles; furthermore, the method provided by the present invention does not require the accuracy of the external parameters of the entire system.

Description

Technical Field

[0001] The present invention belongs to the field of data processing, and in particular relates to a target vehicle detection system and method for closed highways. Background Art

[0002] Closed highways include freeways and urban expressways. Compared to ordinary urban roads, vehicles on closed highways travel at higher speeds and have longer safe braking distances. Therefore, autonomous vehicles must be as aware of potentially dangerous objects as possible and take appropriate action as soon as possible.

[0003] Traditional advanced driver-assisted driving solutions for closed highways use high-definition cameras and millimeter-wave radar to detect targets. While camera-based target recognition is highly mature, with detection rates even exceeding those of humans, cameras cannot perceive target distance. While some methods have emerged in recent years that use camera data to infer target distance, neither the detection range nor the accuracy are suitable for highway scenarios. While millimeter-wave radar has a longer detection range, neither its accuracy nor its sensitivity for detecting static targets can cover all the unique situations found on highways.

[0004] In recent years, perception solutions that use lidar and camera fusion have gradually gained popularity. Lidar, due to its superior ranging capabilities, plays a crucial role as a distance observer in fusion systems. However, targets detected by lidar are located in 3D space, while those detected by cameras are located in 2D space. Correctly associating the targets detected by these two sensors is a key challenge in fusion solutions. Traditionally, based on the lidar-to-camera extrinsic parameters and the camera's intrinsic parameters, a pinhole imaging model is used to project the lidar target point cloud into the 2D image space. The intersection-and-union (IoU) is then calculated with the bounding box of the target detected by the camera. A higher IoU indicates a higher probability of identifying the same target. However, this approach has drawbacks: It requires high precision for the extrinsic parameters, resulting in overlapping point clouds of targets that are too close together when projected into the image space, making them difficult to accurately match. Furthermore, sparse point clouds of distant targets occupy fewer pixels when projected into the image space, making them difficult to accurately match with the camera's targets. Summary of the Invention

[0005] Purpose of the invention: To address the problems existing in the prior art, the present invention proposes a target vehicle detection system for closed highways that can quickly and accurately complete detection.

[0006] Technical Solution: To achieve the above-mentioned purpose, the present invention provides a target vehicle detection system for closed highways, comprising a data acquisition module, a point cloud data processing module, an image data processing module, and a point cloud data and image data fusion module;

[0007] The data acquisition module is provided on the main vehicle and is used to collect image data and point cloud data around the main vehicle;

[0008] The point cloud data processing module generates a ground point cloud map based on the point cloud data collected by the data acquisition module, identifies lane lines in the ground point cloud map, and clusters non-ground point clouds. Based on the clusters within the lane lines, the target point cloud cluster set is obtained, and the lane labels of the lanes where the clusters are located are identified. Based on the shapes of the bird's-eye view clusters in the target point cloud cluster set, the corresponding shape labels are identified on each cluster. The obtained results are input into the point cloud data and image data fusion module.

[0009] The image data processing module processes the image data collected by the camera, identifying lane lines and target vehicles around the vehicle in the collected images; classifying the positional relationship between the detected target vehicles and the host vehicle; and obtaining the sets of target vehicles in the host vehicle's current lane, its left lane, and its right lane. The obtained results are input into the point cloud data and image data fusion module.

[0010] The point cloud data and image data fusion module fuses the output results of the point cloud data processing module with the output results of the image data processing module to obtain the target vehicle information around the main vehicle.

[0011] Furthermore, the data acquisition module includes a solid-state laser radar, a camera, an integrated navigation unit, and a synchronization signal generator. The solid-state laser radar is mounted horizontally above the main vehicle's front windshield; the camera is located within the main vehicle's front windshield; and the synchronization signal generator and integrated navigation are both located inside the vehicle. This makes the collected data more convenient to process and can effectively improve the system's processing speed.

[0012] Furthermore, the method in which the point cloud data and image data fusion module fuses the result output by the point cloud data processing module with the result output by the image data processing module comprises the following steps:

[0013] Step 301: Obtain the distance from the centroid of each cluster in the target vehicle point cloud cluster set marked with lane labels and shape labels to the coordinate origin of the lidar coordinate system;

[0014] Step 302: Compare the distance from the centroid of each cluster obtained in step 301 to the coordinate origin of the laser radar coordinate system with the distance threshold; obtain the point cloud cluster set Cluster_object_in_lane_near less than the distance threshold and the point cloud cluster set Cluster_object_in_lane_far greater than the distance threshold respectively; for the point cloud cluster set Cluster_object_in_lane_near less than the distance threshold, execute step 303; for the point cloud cluster set Cluster_object_in_lane_far greater than the distance threshold, execute steps 304-305;

[0015] Step 303: Calculate the score of the fusion result of the close-range point cloud and the image according to the formula near ; Execute step 306;

[0016] Score near =Score near (left)+Score near (right)+Score near (current);

[0017] in,

[0018]

[0019]

[0020]

[0021] Where M represents the maximum effective detection distance of the laser radar; Cluster_distance_near[i] represents the distance from the i-th cluster in the point cloud cluster set Cluster_object_in_lane_near that is less than the distance threshold to the coordinate origin of the laser radar coordinate system; I represents the total number of clusters in the point cloud cluster set Cluster_object_in_lane_near that is less than the distance threshold;

[0022] L i-left Indicates that the lane marked on the i-th cluster is labeled as the judgment value of lable_left_lane; s i-left Indicates the judgment value of the shape label marked on the i-th cluster as lable_shape_lefi_lane; L i-right Indicates the judgment value that the lane marked on the i-th cluster is labeled as label_right_lane; s i-rightIndicates the judgment value of the shape label labeled on the i-th cluster as lable_shape_right_lane; s i-current Indicates the judgment value of the shape label identified on the i-th cluster as lable_shape_current_lane;

[0023] Step 304: randomly assign the clusters in the point cloud cluster set Cluster_object_in_lane_far that are greater than the distance threshold to the lanes; and mark the lane label of each cluster after assignment on the cluster;

[0024] Step 305: Calculate the score of the fusion result of the second point cloud and the image according to the formula far ; Execute step 306;

[0025] Score far =Score far (left)+Score far (right)+Score far (current);

[0026] in,

[0027]

[0028]

[0029]

[0030] In the formula, M represents the maximum effective detection distance of the lidar; Cluster_distance_far[j] represents the distance from the jth cluster in the point cloud cluster set Cluster_object_in_lane_far that is greater than the distance threshold to the coordinate origin of the lidar coordinate system; J represents the total number of clusters in the point cloud cluster set Cluster_object_in_lane_far that is greater than the distance threshold;

[0031] L j-left Indicates that the lane marked on the jth cluster is labeled as the judgment value of lable_left_lane; s j-left Indicates the judgment value of the shape label marked on the j-th cluster as lable_shape_left_lane; L j-right Indicates the judgment value that the lane marked on the jth cluster is labeled as label_right_lane; s j-right Indicates the judgment value of the shape label identified on the j-th cluster as lable_shape_right_lane; Lj-current Indicates that the lane marked on the jth cluster is labeled as the judgment value of label_current_lane; s j-current The judgment value indicating that the shape label identified on the j-th cluster is label_shape_current_lane; wherein the label of the lane identified on the j-th cluster is obtained in step 304;

[0032] Step 306: According to the formula

[0033] Score=Score far +Score near ;

[0034] Calculate the total score Score;

[0035] Step 307: Compare the total score Score with the number of target vehicles detected by the image data processing module and Camera_object_count; if the total score Score is not less than the number of target vehicles detected by the image data processing module and Camera_object_count, fuse the target vehicle information obtained by the point cloud data processing module with the corresponding target vehicle information obtained by the image data processing module; if the total score Score is less than the number of target vehicles detected by the image data processing module and Camera_object_count, repeat steps 302 to 306.

[0036] The present invention also provides a method for detecting a target vehicle on a closed highway based on the target vehicle detection system for a closed highway, comprising the following steps:

[0037] Step 1: The data acquisition module collects environmental information around the main vehicle; the data acquisition module includes a laser radar and a camera;

[0038] Step 2: The point cloud data processing module and the image data processing module simultaneously process the collected data respectively; obtain the target vehicle point cloud cluster set marked with lane labels and shape labels and the number of target vehicles detected in different lanes through the image;

[0039] Step 3: The point cloud data and image data fusion module fuses and judges the results processed by the point cloud data processing module and the image data processing module to obtain the target vehicle information around the main vehicle.

[0040] Furthermore, the laser radar is a solid-state laser radar, which is horizontally installed on the upper end of the front windshield of the main vehicle; the camera is set inside the front windshield of the main vehicle.

[0041] Furthermore, the data acquisition module also includes a synchronization signal generator, which is used to synchronize the frequency and time of data acquisition by all sensors in the data acquisition module. Synchronizing the frequency of data acquisition by sensors can effectively improve detection efficiency.

[0042] Furthermore, the method in step 3 in which the point cloud data and image data fusion module fuses and judges the results processed by the point cloud data processing module and the image data processing module includes the following steps:

[0043] Step 301: Obtain the distance from the centroid of each cluster in the target vehicle point cloud cluster set marked with lane labels and shape labels to the coordinate origin of the lidar coordinate system;

[0044] Step 302: Compare the distance from the centroid of each cluster obtained in step 301 to the coordinate origin of the laser radar coordinate system with the distance threshold; obtain the point cloud cluster set Cluster_object_in_lane_near less than the distance threshold and the point cloud cluster set Cluster_object_in_lane_far greater than the distance threshold respectively; for the point cloud cluster set Cluster_object_in_lane_near less than the distance threshold, execute step 303; for the point cloud cluster set Cluster_object_in_lane_far greater than the distance threshold, execute steps 304-305;

[0045] Step 303: Calculate the score of the fusion result of the close-range point cloud and the image according to the formula near ; Execute step 306;

[0046] Score near =Score near (left)+Score near (right)+Score near (current);

[0047] in,

[0048]

[0049]

[0050]

[0051] Where M represents the maximum effective detection distance of the laser radar; Cluster_distance_near[i] represents the distance from the i-th cluster in the point cloud cluster set Cluster_object_in_lane_near that is less than the distance threshold to the coordinate origin of the laser radar coordinate system; I represents the total number of clusters in the point cloud cluster set Cluster_object_in_lane_near that is less than the distance threshold;

[0052] L i-left Indicates that the lane marked on the i-th cluster is labeled as the judgment value of lable_left_lane; s i-left Indicates the judgment value of the shape label marked on the i-th cluster as lable_shape_left_lane; L i-right Indicates the judgment value that the lane marked on the i-th cluster is labeled as label_right_lane; s i-right Indicates the judgment value of the shape label labeled on the i-th cluster as lable_shape_right_lane; s i-current Indicates the judgment value of the shape label identified on the i-th cluster as lable_shape_current_lane;

[0053] Step 304: randomly assign the clusters in the point cloud cluster set Cluster_object_in_lane_far that are greater than the distance threshold to the lanes; and mark the lane label of each cluster after assignment on the cluster;

[0054] Step 305: Calculate the score of the fusion result of the second point cloud and the image according to the formula far ; Execute step 306;

[0055] Score far =Score far (left)+Score far (right)+Score far (current);

[0056] in,

[0057]

[0058]

[0059]

[0060] In the formula, M represents the maximum effective detection distance of the lidar; Cluster_distance_far[j] represents the distance from the jth cluster in the point cloud cluster set Cluster_object_in_lane_far that is greater than the distance threshold to the coordinate origin of the lidar coordinate system; J represents the total number of clusters in the point cloud cluster set Cluster_object_in_lane_far that is greater than the distance threshold;

[0061] L j-left Indicates that the lane marked on the jth cluster is labeled as the judgment value of lable_left_lane; s j-left Indicates the judgment value of the shape label marked on the j-th cluster as lable_shape_left_lane; L j-right Indicates the judgment value that the lane marked on the jth cluster is labeled as label_right_lane; s j-right Indicates the judgment value of the shape label identified on the j-th cluster as lable_shape_right_lane; L j-current Indicates that the lane marked on the jth cluster is labeled as the judgment value of label_current_lane; s j-current The judgment value indicating that the shape label identified on the j-th cluster is label_shape_current_lane; wherein the label of the lane identified on the j-th cluster is obtained in step 304;

[0062] Step 306: According to the formula

[0063] Score=Score far +Score near ;

[0064] Calculate the total score Score;

[0065] Step 307: Compare the total score Score with the number of target vehicles detected by the image data processing module and Camera_object_count; if the total score Score is not less than the number of target vehicles detected by the image data processing module and Camera_object_count, fuse the target vehicle information obtained by the point cloud data processing module with the corresponding target vehicle information obtained by the image data processing module; if the total score Score is less than the number of target vehicles detected by the image data processing module and Camera_object_count, repeat steps 302 to 306.

[0066] Furthermore, the method for identifying the shape label in step 2 is: comparing the shape of each cluster in the target vehicle point cloud cluster set from a bird's-eye view with a preset shape and then identifying the corresponding label; the preset shapes include inverted L-shape, L-shape and straight shape.

[0067] The present invention also provides a computer system, comprising:

[0068] one or more processors;

[0069] A memory stores operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors are caused to perform operations, wherein the operations include the process of the target vehicle detection method for a closed highway as described above.

[0070] The present invention also provides a computer-readable medium storing software, wherein the software includes instructions that can be executed by one or more computers, and the instructions, through such execution, cause the one or more computers to perform operations, wherein the operations include the process of the above-mentioned target vehicle detection method for closed highways.

[0071] Working principle: It uses an easily overlooked attribute in the LiDAR scanning data - shape, and utilizes the lane lines, target point cloud shape and target distance information during fusion, and fuses them in the point cloud space. It solves the problem that multiple close targets cannot be correctly matched and that farther targets cannot be matched. It directly matches in 3D space and introduces weights based on target distance to balance the effects of the two attributes of shape and lane lines on fusion. The present invention uses both 2D and 3D information for target detection. Unmanned driving requires the 3D spatial position of the target, so the present invention associates the detection results of the successfully matched camera with the LiDAR data. It effectively improves the accuracy of the target vehicle information.

[0072] Beneficial effects: Compared with the existing technology, the present invention effectively improves the accuracy of target vehicle detection and shortens the target vehicle detection time, and is more suitable for unmanned vehicles; furthermore, the method provided by the present invention has no requirements on the accuracy of the external parameters of the entire system. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 This is a schematic diagram of the structure of the target detection system for highways provided by the present invention;

[0074] Figure 2 A flow chart of the target detection method for highways provided by the present invention;

[0075] Figure 3 The clusters in the second target point cloud cluster set Cluster_object_in_lane are projected into the shape diagram of the bird's-eye view;

[0076] Figure 4 The preset shape is an L-shaped schematic diagram;

[0077] Figure 5 The preset shape is a straight line diagram;

[0078] Figure 6 The preset shape is an inverted L-shaped diagram. DETAILED DESCRIPTION

[0079] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the examples of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0080] like Figure 1 As shown, this embodiment provides a target detection system for highways, which is used in mass-produced cars with autonomous driving functions. It mainly includes a data acquisition module, a point cloud data processing module, an image data processing module, and a point cloud data and image data fusion module.

[0081] Among them, the data acquisition module includes a solid-state laser radar, a camera, a combined navigation unit and a synchronization signal generator; the solid-state laser radar is horizontally installed on the upper end of the main vehicle's front windshield; the camera is 8 million pixels and is set inside the main vehicle's front windshield; the synchronization signal generator and combined navigation are both set inside the vehicle.

[0082] The integrated navigation unit transmits a 1 Hz PPS satellite synchronization signal to the synchronization signal generator. Based on this 1 Hz PPS signal, the synchronization signal generator generates a 20 Hz sensor data acquisition synchronization signal. Upon receiving the 20 Hz synchronization signal, the lidar begins emitting laser light and collecting a point cloud of the surrounding environment. The camera, upon receiving the 20 Hz synchronization signal, captures a color image of the surrounding environment.

[0083] The point cloud data processing module generates a ground point cloud map based on the point cloud data collected by the data acquisition module, identifies lane lines in the ground point cloud map, and clusters non-ground point clouds. It obtains a target point cloud cluster set based on the clusters within the lane lines, and then identifies the lane label according to the lane where the cluster is located. It also identifies the corresponding shape label on each cluster based on the shape of the bird's-eye view of the clusters in the target point cloud cluster set. The obtained results are input into the point cloud data and image data fusion module.

[0084] The image data processing module mainly processes the image data collected by the camera, identifies lane lines and target vehicles around the vehicle in the collected images; and classifies the positional relationship between the detected target vehicles and the main vehicle; and obtains the sets of target vehicles in the current lane where the main vehicle is located, its left lane, and its right lane.

[0085] The point cloud data and image data fusion module fuses the output results of the point cloud data processing module with the output results of the image data processing module to obtain the target vehicle information around the main vehicle.

[0086] like Figure 2 As shown, a host vehicle equipped with the target detection system for highways provided by this embodiment is traveling on a highway, and detecting vehicles around the host vehicle specifically includes the following steps:

[0087] Step 1: The data acquisition module starts collecting data:

[0088] The integrated navigation continuously collects the 3D posture of the vehicle during driving, and forms a vehicle posture sequence Position_queue_rtk based on the data collected at each collection moment. The posture sequence Position_queue_rtk includes the coordinate value of the vehicle in the geodetic coordinate system at each collection moment.

[0089] The lidar continuously collects point clouds of the vehicle's surroundings while the vehicle is driving, and composes a surrounding point cloud frame sequence Pointcloud_queue_lidar based on the point cloud frames collected at each collection moment.

[0090] The camera continuously collects color images of the vehicle's surroundings while the vehicle is driving, and composes a surrounding color image sequence Image_queue_camera based on the color images collected at each collection moment.

[0091] Since a synchronization signal generator is set in the data acquisition module, the data collected by the integrated navigation, lidar, and camera are collected at the same time. Therefore, within the same data acquisition cycle, the total amount of data in the pose sequence Position_queue_rtk, the point cloud sequence Pointcloud_queue_lidar, and the color image sequence Image_queue_camera is the same.

[0092] Step 2: The point cloud data processing module and the image data processing module process the collected data respectively at the same time.

[0093] Among them, the point cloud data processing module mainly includes the following steps for processing point cloud data:

[0094] Step 2A-1: Traverse each frame in the surrounding point cloud frame sequence Pointcloud_queue_lidar, extract the ground points in each frame, and store them in the ground point cloud sequence Pointcloud_ground_queue_lidar in ascending order of the point cloud frame acquisition time. The remaining non-ground points are stored in the non-ground point cloud sequence Pointcloud_none_ground_queue_lidar in ascending order of the point cloud frame acquisition time.

[0095] The method for extracting ground point cloud in each frame point cloud is the ray_ground_filter algorithm.

[0096] Step 2A-2: The ground point cloud map generation module uses the 3D pose at the corresponding acquisition time in the pose sequence Position_queue_rtk as the initial position. It then uses a point cloud registration method to stitch adjacent point cloud frames in the point cloud sequence Pointcloud_queue_lidar in real time, generating a pose change matrix sequence Matrix_odometry_lidar sorted by ascending acquisition time. Point cloud registration methods include ICP point cloud registration or NDT point cloud registration.

[0097] Step 2A-3: The ground point cloud map generation module stitches the ground point clouds in the ground point cloud sequence Pointcloud_ground_queue_lidar according to the pose change matrix sequence Matrix_odometry_lidar of adjacent point cloud frames obtained in step 2A-2 at the corresponding acquisition time to obtain the ground point cloud map PointMap_ground_lidar.

[0098] Step 2A-4: Identify lane lines in the ground point cloud map PointMap_ground_lidar.

[0099] Lane line recognition mainly includes the following steps:

[0100] Step 2A-401: Set a reflection intensity threshold, filter out points in the ground point cloud map PointMap_ground_lidar whose reflection intensity exceeds the reflection intensity threshold, and save the filtered points in the lane point set Set_points_lane_lidar;

[0101] Step 2A-402: Project all points in the lane point set Set_points_lane_lidar onto the two-dimensional plane where the ground plane resides, generating an image, Image_points_lane_lidar. Since the lidar is mounted horizontally on the vehicle, in this embodiment, projecting all points in the lane point set Set_points_lane_lidar onto the two-dimensional plane where the ground plane resides actually preserves the x-axis and y-axis information of all points in the lane point set Set_points_lane_lidar to generate the resulting two-dimensional image.

[0102] Step 2A-403: Perform a dilation operation on the image Image_points_lane_lidar, and then perform a Hough transform to fit the lane lines.

[0103] Step 2A-404: Based on the conversion relationship between the points in the lane line point set Set_points_lane_lidar and the resulting image Image_points_lane_lidar, the lane lines obtained in step 2A-403 are projected onto the ground point cloud map PointMap_ground_lidar. In other words, the lane lines obtained in step 2A-403 are converted from 2D to 3D space, thus recovering the Z-axis information of each point that makes up the lane line.

[0104] Step 2A-5: Identify the host vehicle's current lane, the host vehicle's current left lane, and the host vehicle's current right lane in the ground point cloud map PointMap_ground_lidar;

[0105] Step 2A-6: Execute the Euclidean clustering algorithm in the current frame of the non-ground point cloud sequence Pointcloud_none_ground_queue_lidar to obtain the first target point cloud cluster set Cluster_object.

[0106] Step 2A-7: traverse the first target point cloud cluster set Cluster_object, and determine whether the clusters in the first target point cloud cluster set Cluster_object are located within the multiple lane lines obtained in step 2A-4;

[0107] Only the clusters within the lane line are retained to obtain the filtered second target point cloud cluster set Cluster_object_in_lane.

[0108] According to the lane where each cluster in the second target point cloud cluster set Cluster_object_in_lane is located, the label lane_lable (lable_left_lane, lable_right_lane, lable_current_lane) of the lane is marked on each cluster; lable_left_lane represents the cluster in the left lane of the main vehicle; lable_right_lane represents the cluster in the right lane of the main vehicle; lable_current_lane represents the cluster in the lane where the main vehicle is located.

[0109] Among them, the method for determining whether the clustered point cloud in the set is located within the multiple lane lines obtained in step 2A-4 is: the geometric center point Point_x of the clustered point cloud is within the lane line range, that is, the geometric center point Point_x of the clustered point cloud satisfies that the x-coordinate and y-coordinate of Point_x are both within the lane line range, then the clustered point cloud is located within the lane line.

[0110] Step 2A-8: Detect the shape of each cluster in the second target point cloud cluster set Cluster_object_in_lane from a bird's-eye view, compare it with a preset shape, and record the corresponding label shape_lable.

[0111] The specific method includes the following steps:

[0112] Step 2A-801: Project all points constituting the clusters in the second target point cloud cluster set Cluster_object_in_lane into the bird's-eye view image; Figure 3 As shown, 1 represents the main vehicle; 2 is the shape of the cluster in the left lane of the main vehicle in the bird's-eye view, 3 is the shape of the cluster in the right lane of the main vehicle in the bird's-eye view, and 4 is the shape of the cluster in the same lane as the main vehicle in the bird's-eye view.

[0113] Step 2A-802: Compare the shape obtained in step 2A-801 with the preset shape, such as Figures 4-6 As shown, the preset shapes include inverted L-shape, L-shape and straight-line shape, and the label of the inverted L-shape is lable_shape_left_lane; and the comparison results are recorded in the form of labels; the label corresponding to the inverted L-shape is lable_shape_left_lane, the label corresponding to the L-shape is lable_shape_right_lane, and the label corresponding to the straight-line shape is lable_shape_current_lane.

[0114] The method for processing the collected images by the image data processing module includes the following steps:

[0115] Step 2B-1: Execute an object detection algorithm on the last frame of the color image sequence Image_queue_camera to obtain the target vehicle set Camera_object_queue. This algorithm primarily uses an object detection algorithm, such as the YOLO algorithm, to detect target vehicles surrounding the main vehicle in the color image of the current frame. The current frame is the last frame of the color image sequence Image_queue_camera.

[0116] Step 2B-2: Execute a lane line detection algorithm in the last frame of the color image sequence Image_queue_camera to obtain lane lines. In this embodiment, the Lane-net algorithm is used to detect lane lines in the color image of the current frame.

[0117] Step 2B-3: Divide the target vehicle set Camera_object_queue obtained in Step 2B-1 into three sets based on the lanes they are in. The target vehicles in the left lane of the host vehicle are stored in the left lane target vehicle set Camera_object_left_lane_queue; the target vehicles in the right lane of the host vehicle are stored in the right lane target vehicle set Camera_object_right_lane_queue; and the target vehicles in the same lane as the host vehicle are stored in the current lane target vehicle set Camera_object_current_lane_queue. The number of target vehicles in each set is counted separately, where Camera_count_current represents the number of target vehicles in the same lane as the host vehicle, as obtained by the image data processing module; Camera_count_left represents the number of target vehicles in the left lane of the host vehicle, as obtained by the image data processing module; and Camera_count_right represents the number of target vehicles in the right lane of the host vehicle, as obtained by the image data processing module.

[0118] Step 3: The point cloud data and image data fusion module fuses and judges the results processed by the point cloud data processing module and the image data processing module to obtain the target vehicle information around the main vehicle.

[0119] Specific methods include:

[0120] Step 301: Calculate the distance between the centroid of each cluster in the second target point cloud cluster set Cluster_object_in_lane and the coordinate origin of the laser radar coordinate system;

[0121] Step 302: Compare the distance from the centroid of each cluster obtained in step 301 to the origin of the LiDAR coordinate system with a distance threshold. Obtain the point cloud clusters Cluster_object_in_lane_near and Cluster_object_in_lane_far for clusters smaller than the distance threshold. For clusters smaller than the distance threshold, execute step 303; for clusters larger than the distance threshold, execute steps 304-305. In this embodiment, the distance threshold is 40 meters.

[0122] Step 303: Calculate the score of the fusion result of the close-range point cloud and the image according to the formula near ; Execute step 306;

[0123] Score near =Score near (left)+Score near (right)+Score near (current);

[0124] in,

[0125]

[0126]

[0127]

[0128] In the formula, M represents the maximum effective detection distance of the lidar; Cluster_distance_near[i] represents the distance from the i-th cluster in the point cloud cluster set Cluster_object_in_lane_near that is less than the distance threshold to the coordinate origin of the lidar coordinate system; I represents the total number of clusters in the point cloud cluster set Cluster_object_in_lane_near that is less than the distance threshold.

[0129] L i-left Indicates the judgment value that the label of the lane marked on the i-th cluster is lable_left_lane. If the label of the lane marked on the i-th cluster is lable_left_lane, then L i-left =1; if the label of the lane marked on the i-th cluster is not label_left_lane, then L i-left =0;s i-leftIndicates the judgment value of the shape label identified on the i-th cluster as lable_shape_left_lane. If the shape label identified on the i-th cluster is lable_shape_left_lane, then s i-left =1; if the shape label marked on the i-th cluster is not lable_shape_left_lane, then s i-left =0.

[0130] L i-right Indicates the judgment value that the label of the lane marked on the i-th cluster is lable_right_lane. If the label of the lane marked on the i-th cluster is lable_right_lane, then L i-right =1; if the lane label of the i-th cluster is not label_right_lane, then L i-right =0;s i-right Indicates the judgment value of the shape label identified on the i-th cluster as lable_shape_right_lane. If the shape label identified on the i-th cluster is lable_shape_right_lane, then s i-right =1; if the shape label marked on the i-th cluster is not label_shape_right_lane, then s i-right =0.

[0131] L i-current Indicates the judgment value that the label of the lane marked on the i-th cluster is lable_current_lane. If the label of the lane marked on the i-th cluster is lable_current_lane, then L i-current =1; if the label of the lane marked on the i-th cluster is not label_current_lane, then L i-current =0;s i-current Indicates the judgment value of the shape label identified on the i-th cluster as lable_shape_current_lane. If the shape label identified on the i-th cluster is lable_shape_current_lane, then s i-current =1; if the shape label marked on the i-th cluster is not label_shape_current_lane, then s i-current =0.

[0132] L i-left 、L i-right 、L i-current 、s i-left 、si-right and s i-current All calculations can be performed using the ternary operator; that is:

[0133] L i-left =(Cluster_object_in_lane_near[i].lane_lable==lable_left_lane)? 1:0;

[0134] L i-right =(Cluster_object_in_lane_near[i].lane_lable==lable_right_lane)? 1:0;

[0135] L i-current =(Cluster_object_in_lane_near[i].lane_lable==lable_current_lane)? 1:0;

[0136] s i-left =(Cluster_object_in_lane_near[i].shape_lable==lable_shape_left_lane)? 1:0;

[0137] s i-right =(Cluster_object_in_lane_near[i].shape_lable==lable_shape_right_lane)? 1:0;

[0138] s i-current =(Cluster_object_in_lane_near[i].shape_lable==lable_shape_current_lane)? 1:0;

[0139] Step 304: randomly assign the clusters in the point cloud cluster set Cluster_object_in_lane_far that are greater than the distance threshold to the lanes; and mark the lane label of each cluster after assignment on the cluster;

[0140] Step 305: Calculate the score of the fusion result of the second point cloud and the image according to the formula far ; Execute step 306;

[0141] Score far =Score far (left)+Score far (right)+Scorefar (current);

[0142] in,

[0143]

[0144]

[0145]

[0146] In the formula, M represents the maximum effective detection distance of the lidar; Cluster_distance_far[j] represents the distance from the jth cluster in the point cloud cluster set Cluster_object_in_lane_far that is greater than the distance threshold to the coordinate origin of the lidar coordinate system; J represents the total number of clusters in the point cloud cluster set Cluster_object_in_lane_far that is greater than the distance threshold.

[0147] L j-left Indicates the judgment value that the label of the lane marked on the jth cluster is lable_left_lane. If the label of the lane marked on the jth cluster is lable_left_lane, then L j-left =1; if the label of the lane marked on the jth cluster is not label_left_lane, then L j-left =0;s j-left Indicates the judgment value of the shape label identified on the j-th cluster as lable_shape_left_lane. If the shape label identified on the j-th cluster is lable_shape_left_lane, then s j-left =1; if the shape label marked on the jth cluster is not lable_shape_left_lane, then s j-left =0.

[0148] L j-right Indicates the judgment value that the label of the lane marked on the jth cluster is lable_right_lane. If the label of the lane marked on the jth cluster is lable_right_lane, then L j-right =1; if the label of the lane marked on the jth cluster is not label_right_lane, then L j-right =0;s j-rightIndicates the judgment value of the shape label identified on the j-th cluster as lable_shape_right_lane. If the shape label identified on the j-th cluster is lable_shape_right_lane, then s j-right =1; if the shape label marked on the jth cluster is not label_shape_right_lane, then s j-right =0.

[0149] L j-current Indicates the judgment value that the label of the lane marked on the jth cluster is lable_current_lane. If the label of the lane marked on the jth cluster is lable_current_lane, then L j-current =1; if the label of the lane marked on the jth cluster is not label_current_lane, then L j-current =0;s j-current Indicates the judgment value of the shape label identified on the j-th cluster as lable_shape_current_lane. If the shape label identified on the j-th cluster is lable_shape_current_lane, then s j-current =1; if the shape label marked on the jth cluster is not label_shape_current_lane, then s j-current =0.

[0150] The label of the lane marked on the jth cluster is obtained in step 304. j-left 、L j-right 、L j-current 、s j-left 、s j-rigght and s j-current All of them can be calculated using the ternary operator.

[0151] Step 306: According to the formula

[0152] Score=Score far +Score near ;

[0153] Calculate the total score Score.

[0154] Step 307: Compare the total score Score with the number of target vehicles detected by the image data processing module and Camera_object_count; if the total score Score is not less than the number of target vehicles detected by the image data processing module and Camera_object_count, fuse the target vehicle information obtained by the point cloud data processing module with the corresponding target vehicle information obtained by the image data processing module, that is, associate the corresponding target vehicle information in the point cloud data processing module and the image data processing module; if the total score Score is less than the number of target vehicles detected by the image data processing module and Camera_object_count, repeat steps 302 to 306.

[0155] Among them, Camera_object_count=Camera_count_left+Camera_count_right+Camera_count_current.

[0156] The present invention also provides a computer system comprising: one or more processors; and a memory storing operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors are caused to perform operations, wherein the operations include the process of the aforementioned method for detecting a target vehicle on a closed highway.

[0157] It should be understood that the aforementioned examples of the closed highway target vehicle detection method of the present invention can be implemented in any computer system that includes data storage and data processing capabilities. The aforementioned computer system can be at least one electronic processing system or electronic device including a processor and memory, such as a personal computer, whether a personal computer, a commercial computer, a graphics processing computer, or a server-class computer. These PCs can implement wired and / or wireless data transmission, particularly image data, by having data interfaces and / or network interfaces.

[0158] In other embodiments, the computer system may also be a server, in particular a cloud server, having data storage, processing, and network communication functions.

[0159] As an example, a computer system generally includes at least one processor, a memory, and a network interface connected by a system bus. The network interface is used to communicate with other devices / systems.

[0160] The processor is used to provide calculation and control of the system.

[0161] The memory includes non-volatile memory and cache.

[0162] Non-volatile memory typically has a massive storage capacity and can store operating systems and computer programs. These computer programs may include operable instructions that, when executed by one or more processors, enable the one or more processors to perform the process of the target vehicle detection method for closed highways of the aforementioned embodiment of the present invention.

[0163] In necessary or reasonable implementations, the aforementioned computer system, whether a PC device or a server, may also include more or fewer components than shown in the figure, or a combination, or adopt different hardware, software and other components or different deployment methods.

Claims

1. A target vehicle detection system for closed highways, characterized by: It includes data acquisition module, point cloud data processing module, image data processing module, point cloud data and image data fusion module; The data acquisition module is provided on the main vehicle and is used to collect image data and point cloud data around the main vehicle; The point cloud data processing module generates a ground point cloud map based on the point cloud data collected by the data acquisition module, identifies lane lines in the ground point cloud map, and clusters non-ground point clouds. Based on the clusters within the lane lines, the target point cloud cluster set is obtained, and the lane labels of the lanes where the clusters are located are identified. Based on the shapes of the bird's-eye view clusters in the target point cloud cluster set, the corresponding shape labels are identified on each cluster. The obtained results are input into the point cloud data and image data fusion module. The image data processing module processes the image data collected by the camera, identifying lane lines and target vehicles around the vehicle in the collected images; classifying the positional relationship between the detected target vehicles and the host vehicle; and obtaining the target vehicles in the host vehicle's current lane, its left lane, and its right lane. The obtained results are input into the point cloud data and image data fusion module. The point cloud data and image data fusion module fuses the output of the point cloud data processing module with the output of the image data processing module to obtain the target vehicle information around the main vehicle; The method for the point cloud data and image data fusion module to fuse the result output by the point cloud data processing module with the result output by the image data processing module comprises the following steps: Step 301: Obtain the distance from the centroid of each cluster in the target vehicle point cloud cluster set marked with lane labels and shape labels to the coordinate origin of the lidar coordinate system; Step 302: Compare the distance from the centroid of each cluster obtained in step 301 to the coordinate origin of the laser radar coordinate system with the distance threshold; obtain the point cloud cluster set Cluster_object_in_lane_near less than the distance threshold and the point cloud cluster set Cluster_object_in_lane_far greater than the distance threshold respectively; for the point cloud cluster set Cluster_object_in_lane_near less than the distance threshold, execute step 303; for the point cloud cluster set Cluster_object_in_lane_far greater than the distance threshold, execute steps 304-305; Step 303: Calculate the score of the fusion result of the close-range point cloud and the image according to the formula near ; Execute step 306; Score near =Score near (left)+Score near (right)+Score near (current); in, Where M represents the maximum effective detection distance of the laser radar; Cluster_distance_near[i] represents the distance from the i-th cluster in the point cloud cluster set Cluster_object_in_lane_near that is less than the distance threshold to the coordinate origin of the laser radar coordinate system; I represents the total number of clusters in the point cloud cluster set Cluster_object_in_lane_near that is less than the distance threshold; Camera_count_current represents the number of target vehicles in the same lane as the main vehicle obtained by the image data processing module, Camera_count_left represents the number of target vehicles in the left lane of the main vehicle obtained by the image data processing module, and Camera_count_right represents the number of target vehicles in the right lane of the main vehicle obtained by the image data processing module; L i-left Indicates that the lane marked on the i-th cluster is labeled as the judgment value of lable_left_lane; s i-left Indicates the judgment value of the shape label marked on the i-th cluster as lable_shape_left_lane; L i-right Indicates the judgment value that the lane marked on the i-th cluster is labeled as label_right_lane; s i-right Indicates the judgment value of the shape label labeled on the i-th cluster as lable_shape_right_lane; s i-current Indicates the judgment value of the shape label identified on the i-th cluster as lable_shape_current_lane; L i-current Indicates that the lane labeled on the i-th cluster is labeled as the judgment value of label_current_lane; Step 304: randomly assign the clusters in the point cloud cluster set Cluster_object_in_lane_far that are greater than the distance threshold to the lanes; and mark the lane label of each cluster after assignment on the cluster; Step 305: Calculate the score of the fusion result of the second point cloud and the image according to the formula far ; Execute step 306; Score far =Score far (left)+Score far (right)+Score far (current); in, In the formula, M represents the maximum effective detection distance of the lidar; Cluster_distance_far[j] represents the distance from the jth cluster in the point cloud cluster set Cluster_object_in_lane_far that is greater than the distance threshold to the coordinate origin of the lidar coordinate system; J represents the total number of clusters in the point cloud cluster set Cluster_object_in_lane_far that is greater than the distance threshold; L j-left Indicates that the lane marked on the jth cluster is labeled as the judgment value of lable_left_lane; s j-left Indicates the judgment value of the shape label marked on the j-th cluster as lable_shape_left_lane; L j-right Indicates the judgment value that the lane marked on the jth cluster is labeled as label_right_lane; s j-right Indicates the judgment value of the shape label identified on the j-th cluster as lable_shape_right_lane; L j-current Indicates that the lane marked on the jth cluster is labeled as the judgment value of label_current_lane; s j-current The judgment value indicating that the shape label identified on the j-th cluster is label_shape_current_lane; wherein the label of the lane identified on the j-th cluster is obtained in step 304; Step 306: According to the formula Score=Score far +Score near ; Calculate the total score Score; Step 307: Compare the total score Score with the total number of target vehicles Camera_object_count detected by the image data processing module; if the total score Score is not less than the total number of target vehicles Camera_object_count detected by the image data processing module, fuse the target vehicle information obtained by the point cloud data processing module with the corresponding target vehicle information obtained by the image data processing module; if the total score Score is less than the total number of target vehicles Camera_object_count detected by the image data processing module, repeat steps 302 to 306.

2. The target vehicle detection system for closed highways according to claim 1, characterized in that: The data acquisition module includes a solid-state laser radar, a camera, a combined navigation unit and a synchronization signal generator; the solid-state laser radar is horizontally installed on the upper end of the main vehicle's front windshield; the camera is set inside the main vehicle's front windshield; the synchronization signal generator and the combined navigation are both set inside the vehicle.

3. A method for detecting a target vehicle on a closed highway based on the target vehicle detection system for a closed highway according to claim 1, characterized in that: The following steps are involved: Step 1: The data acquisition module collects environmental information around the main vehicle; the data acquisition module includes a laser radar and a camera; Step 2: The point cloud data processing module and the image data processing module process the collected data simultaneously; Obtain target vehicle point cloud cluster sets marked with lane labels and shape labels and the number of target vehicles detected in different lanes through images respectively; Step 3: The point cloud data and image data fusion module fuses and judges the results processed by the point cloud data processing module and the image data processing module to obtain the target vehicle information around the main vehicle; The method for the point cloud data and image data fusion module in step 3 to fuse and judge the results processed by the point cloud data processing module and the image data processing module includes the following steps: Step 301: Obtain the distance from the centroid of each cluster in the target vehicle point cloud cluster set marked with lane labels and shape labels to the coordinate origin of the lidar coordinate system; Step 302: Compare the distance from the centroid of each cluster obtained in step 301 to the coordinate origin of the laser radar coordinate system with the distance threshold; obtain the point cloud cluster set Cluster_object_in_lane_near less than the distance threshold and the point cloud cluster set Cluster_object_in_lane_far greater than the distance threshold respectively; for the point cloud cluster set Cluster_object_in_lane_near less than the distance threshold, execute step 303; for the point cloud cluster set Cluster_object_in_lane_far greater than the distance threshold, execute steps 304-305; Step 303: Calculate the score of the fusion result of the close-range point cloud and the image according to the formula near ; Execute step 306; Score near =Score near (left)+Score near (right)+Score near (current); in, Where M represents the maximum effective detection distance of the laser radar; Cluster_distance_near[i] represents the distance from the i-th cluster in the point cloud cluster set Cluster_object_in_lane_near that is less than the distance threshold to the coordinate origin of the laser radar coordinate system; I represents the total number of clusters in the point cloud cluster set Cluster_object_in_lane_near that is less than the distance threshold; Camera_count_current represents the number of target vehicles in the same lane as the main vehicle obtained by the image data processing module, Camera_count_left represents the number of target vehicles in the left lane of the main vehicle obtained by the image data processing module, and Camera_count_right represents the number of target vehicles in the right lane of the main vehicle obtained by the image data processing module; L i-left Indicates that the lane marked on the i-th cluster is labeled as the judgment value of lable_left_lane; s i-left Indicates the judgment value of the shape label marked on the i-th cluster as lable_shape_left_lane; L i-right Indicates the judgment value that the lane marked on the i-th cluster is labeled as label_right_lane; s i-right Indicates the judgment value of the shape label labeled on the i-th cluster as lable_shape_right_lane; s i-current Indicates the judgment value of the shape label identified on the i-th cluster as lable_shape_current_lane; L i-current Indicates that the lane labeled on the i-th cluster is labeled as the judgment value of label_current_lane; Step 304: randomly assign the clusters in the point cloud cluster set Cluster_object_in_lane_far that are greater than the distance threshold to the lanes; and mark the lane label of each cluster after assignment on the cluster; Step 305: Calculate the score of the fusion result of the second point cloud and the image according to the formula far ; Execute step 306; Score far =Score far (left)+Score far (right)+Score far (current); in, In the formula, M represents the maximum effective detection distance of the lidar; Cluster_distance_far[j] represents the distance from the jth cluster in the point cloud cluster set Cluster_object_in_lane_far that is greater than the distance threshold to the coordinate origin of the lidar coordinate system; J represents the total number of clusters in the point cloud cluster set Cluster_object_in_lane_far that is greater than the distance threshold; L j-left Indicates that the lane marked on the jth cluster is labeled as the judgment value of lable_left_lane; s j-left Indicates the judgment value of the shape label marked on the j-th cluster as lable_shape_left_lane; L j-right Indicates the judgment value that the lane marked on the jth cluster is labeled as label_right_lane; s j-right Indicates the judgment value of the shape label identified on the j-th cluster as lable_shape_right_lane; L j-current Indicates that the lane marked on the jth cluster is labeled as the judgment value of label_current_lane; s j-current The judgment value indicating that the shape label identified on the j-th cluster is label_shape_current_lane; wherein the label of the lane identified on the j-th cluster is obtained in step 304; Step 306: According to the formula Score=Score far +Score near ; Calculate the total score Score; Step 307: Compare the total score Score with the total number of target vehicles Camera_object_count detected by the image data processing module; if the total score Score is not less than the total number of target vehicles Camera_object_count detected by the image data processing module, fuse the target vehicle information obtained by the point cloud data processing module with the corresponding target vehicle information obtained by the image data processing module; if the total score Score is less than the total number of target vehicles Camera_object_count detected by the image data processing module, repeat steps 302 to 306.

4. The method for detecting a target vehicle on a closed highway according to claim 3, wherein: The laser radar is a solid-state laser radar, which is horizontally installed on the upper end of the front windshield of the main vehicle; the camera is set in the front windshield of the main vehicle.

5. The method for detecting a target vehicle on a closed highway according to claim 3, wherein: The data acquisition module also includes a synchronization signal generator, which is used to synchronize the frequency and time of data acquisition by all sensors in the data acquisition module.

6. The method for detecting a target vehicle on a closed highway according to claim 3, wherein: The method for identifying the shape label in step 2 is: comparing the shape of each cluster in the target vehicle point cloud cluster set from a bird's-eye view with a preset shape and then identifying the corresponding label; The preset shapes include reverse L-shape, L-shape and straight shape.

7. A computer system, characterized in that: include: one or more processors; A memory storing operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors perform operations, wherein the operations include the process of the target vehicle detection method for a closed highway as described in any one of claims 3 to 6.

8. A computer-readable medium storing software, characterized in that: The software includes instructions that can be executed by one or more computers, and the instructions, through such execution, enable the one or more computers to perform operations, and the operations include the process of the target vehicle detection method for closed highways as described in any one of claims 3 to 6.

Images