A target post-processing method, system and product based on in-frame point cloud

Abstract

The application provides a target post-processing method, system and product based on in-frame point cloud, which comprises the following steps: processing point cloud data to determine a target frame and target drivable area points in the frame; determining a convex hull point set comprising all target drivable area points in the frame according to the target drivable area points; determining the longitudinal maximum and transverse minimum convex hull points in the set; when the target frame is on the left side of the vehicle, the right convex hull point set is constructed by the convex hull points which meet the first condition in the longitudinal distance between the longitudinal maximum and transverse minimum convex hull points in the frame; when the target frame is on the right side of the vehicle, the left convex hull point set is constructed by the convex hull points which meet the second condition in the longitudinal distance between the longitudinal maximum and transverse maximum convex hull points in the frame; when the frame is on the left side of the vehicle and the vehicle turns left, the right boundary line is corrected by the right convex hull points; and when the frame is on the right side of the vehicle and the vehicle turns right, the left boundary line is corrected by the left convex hull points. The application aims to improve the accuracy of the output results of the radar.

Description

Technical Field

[0001] This application relates to the field of point cloud processing technology, specifically to a target post-processing method, system, and product based on in-frame point clouds. Background Technology

[0002] Autonomous vehicles rely on onboard sensors to acquire information about their surroundings. The environmental perception module processes this sensor information and transmits it to subsequent modules for vehicle behavior planning. Therefore, the reliability and accuracy of sensor information are crucial for ensuring the safety of autonomous vehicles. Currently, the mainstream sensors include cameras, millimeter-wave radar, and LiDAR. However, because LiDAR has high detection accuracy, it can detect dust, water vapor, and small particles in vehicle exhaust. In these scenarios, as well as in adverse weather conditions such as rain, snow, and fog, LiDAR performance may be somewhat reduced. Furthermore, the output of the LiDAR deep learning model and the fitting of traditional clustering bounding boxes inevitably exhibit some deviation. To address these two issues, strategies are needed to minimize the decline in perception performance caused by adverse weather or environmental factors and improve the accuracy of LiDAR output information. Summary of the Invention

[0003] In view of this, this application provides a target post-processing method, system, and product based on in-frame point clouds. The aim is to improve the accuracy of radar output results.

[0004] The first aspect of this application provides a target post-processing method based on in-frame point clouds, the method comprising:

[0005] The point cloud data is used to identify targets using a target recognition model, resulting in multiple bounding boxes.

[0006] By analyzing and processing the point cloud data, the target drivable area points located within the target frame are determined;

[0007] Based on the target drivable area points in the target box, determine the convex hull of the target box and the set of convex hull points corresponding to the convex hull, wherein the convex hull points in the set of convex hull points are all the target drivable area points in the target box;

[0008] Determine the vertical maximum convex hull point, the horizontal maximum convex hull point, and the horizontal minimum convex hull point in the set of convex hull points of the target bounding box;

[0009] When the target box is located on the left side of the vehicle, the right convex hull point set of the target box is constructed by taking the convex hull points in the target box as being located in the order of the longitudinal maximum convex hull point and the transverse minimum convex hull point, and the right convex hull point whose distance from the transverse minimum convex hull point in the longitudinal direction satisfies the first condition.

[0010] When the target box is located on the right side of the vehicle, the left convex hull point set of the target box is constructed by taking the convex hull points in the target box as being located in the order of the longitudinal maximum convex hull point and the transverse maximum convex hull point, and the left convex hull point whose distance from the transverse maximum convex hull point in the longitudinal direction satisfies the second condition.

[0011] When the target frame is located on the left side of the vehicle and the vehicle is turning left, the right boundary line of the target frame is corrected using the right convex hull points in the set of right convex hull points of the target frame.

[0012] When the target frame is located on the right side of the vehicle and the vehicle is turning right, the left boundary line of the target frame is corrected using the left convex hull points in the set of left convex hull points of the target frame.

[0013] Optionally, by analyzing and processing the point cloud data, the target drivable area points located within the target frame are determined, including:

[0014] By analyzing and processing the point cloud data, drivable areas in the point cloud data are determined.

[0015] Based on the location of the target frame and the location of the drivable area point, determine the target drivable area point located within the target frame.

[0016] Optionally, before analyzing and processing the point cloud data to determine the target drivable area points located within the target bounding box, the method further includes:

[0017] Based on the attribute information of the target bounding boxes, determine the virtual target bounding boxes among the multiple target bounding boxes;

[0018] Filter the dummy target boxes among the multiple target boxes to obtain the remaining target boxes;

[0019] The step of analyzing and processing the point cloud data to determine the target drivable area points located within the target frame includes:

[0020] By analyzing and processing the point cloud data, the drivable area points of the target located in the remaining target boxes after filtering are determined.

[0021] Optionally, before determining the convex hull of the target bounding box and the set of convex hull points corresponding to the convex hull based on the target drivable area points in the target bounding box, the method further includes:

[0022] Based on the attribute information of the target bounding boxes, determine the target bounding boxes that need orientation correction from all the target bounding boxes;

[0023] The step of determining the convex hull of the target frame and the set of convex hull points corresponding to the convex hull based on the target drivable area points in the target frame includes: determining the convex hull of the target frame that needs to be oriented corrected and the set of convex hull points corresponding to the convex hull based on the target drivable area points in the target frame that need to be oriented corrected.

[0024] Optionally, when the target frame is located on the left side of the vehicle and the vehicle is turning left, the right boundary line of the target frame is corrected using the right convex hull points in the set of right convex hull points of the target frame, including:

[0025] When the target box is located on the left side of the vehicle and the vehicle is turning left, the right convex hull points in the set of right convex hull points of the target box are traversed, and the right convex hull point currently traversed is connected with the horizontal minimum convex hull point to form the right prediction boundary line.

[0026] If the angle deviation between the obtained right predicted boundary line and the right boundary line of the target box meets the set conditions, the traversal ends and the right predicted boundary line of the target box is corrected with the right predicted boundary line whose deviation meets the set conditions.

[0027] The step of correcting the left boundary line of the target frame by using the left convex hull points from the set of left convex hull points of the target frame when the target frame is located on the right side of the vehicle and the vehicle is turning right includes:

[0028] When the target box is located on the right side of the vehicle and turns right, the left convex hull points in the set of left convex hull points of the target box are traversed, and the currently traversed left convex hull point is connected with the largest lateral convex hull point to form the left prediction boundary line.

[0029] If the angle deviation between the obtained left predicted boundary line and the left boundary line of the target box meets the set conditions, the traversal ends and the left boundary line of the target box is corrected using the left predicted boundary line whose deviation meets the set conditions.

[0030] Optionally, the method further includes:

[0031] The point cloud data is clustered using a clustering algorithm to obtain the target bounding boxes for each cluster.

[0032] Determine the proportion of point clouds for different semantic categories of the clustering target box, determine the size of the clustering target box, determine the reflectance information of the point cloud within the clustering target box, and determine the first point cloud density of the point cloud within a set range of the centroid of the clustering target box.

[0033] The clustering target box is divided into multiple segments vertically, and the cloud density of the second point around the center point of the first segment in the direction of movement of the clustering target box is calculated.

[0034] Based on the point cloud proportion, the size, the reflectivity information, the first point cloud density, and the second point cloud density, the virtual clustering target boxes in each clustering target box are determined;

[0035] The dummy clustering boxes in each clustering target box are filtered to obtain the clustering result of the clustering algorithm.

[0036] Optionally, determine the reflectance information of the point cloud within the clustering target bounding box, including:

[0037] Based on the attribute information of the point cloud within the clustering target box, determine the number of low reflectivity point clouds within the clustering target box, as well as the maximum reflectivity and total reflectivity of the point clouds within the clustering target box.

[0038] The proportion of low-reflectivity point clouds within the clustering target box is determined based on the total number of point clouds within the clustering target box and the number of low-reflectivity point clouds.

[0039] The average reflectance of the point cloud within the clustering target box is determined based on the total number of point clouds and the total reflectance value within the clustering target box.

[0040] Optionally, based on the point cloud proportion, the size, the reflectivity information, the first point cloud density, and the second point cloud density, the virtual clustering target boxes in each clustering target box are determined, including:

[0041] When the proportion of low reflectivity point cloud corresponding to the clustered target box is a set value, the clustered target box is determined to be a virtual clustered target box;

[0042] If the proportion of low reflectivity point cloud corresponding to the clustered target box exceeds the first reflectivity threshold and the corresponding average reflectivity does not exceed the second reflectivity threshold, the clustered target box is determined to be a virtual clustered target box.

[0043] If the proportion of low reflectivity point cloud corresponding to the clustered target box exceeds the third reflectivity threshold, and the corresponding average reflectivity does not exceed the fourth reflectivity threshold and the corresponding maximum reflectivity does not exceed the fifth reflectivity threshold, the clustered target box is determined to be a virtual clustered target box.

[0044] When the proportion of point clouds of noisy semantic categories within the clustering target box exceeds the category threshold, the size relationship between the size and the first size threshold is determined, the density relationship between the first point cloud density and the first density threshold is determined, and the quantity relationship between the total number of point clouds and the total number threshold is determined.

[0045] Based on the size relationship, the density relationship, and the quantity relationship, determine whether the clustering target box is a dummy clustering target box;

[0046] If the proportion of point clouds representing noise semantic categories within the clustering target box does not exceed the category threshold, the clustering target box is determined to be a virtual clustering target box based on the proportion of point clouds representing background semantic categories within the clustering target box, the size, and the total number of point clouds. Furthermore, the clustering target box is also determined to be a virtual clustering target box based on the proportion of point clouds representing general obstacle semantic categories within the clustering target box, the location of the clustering target box, and the size.

[0047] Optionally, the method further includes:

[0048] Determine whether there are small single clustering boxes within a preset range around the vehicle category clustering target box;

[0049] If there are small-sized single clustering target boxes within a preset range around the vehicle category clustering target box, the vehicle category clustering target box is determined to be a dummy clustering target box.

[0050] A second aspect of this application provides a target post-processing system based on in-frame point clouds, the system comprising:

[0051] The target recognition module is used to identify targets in point cloud data using a target recognition model to obtain multiple target bounding boxes.

[0052] The analysis and processing module is used to determine the target drivable area points located in the target box by analyzing and processing the point cloud data;

[0053] The convex hull point set determination module is used to determine the convex hull of the target frame and the convex hull point set corresponding to the convex hull based on the target drivable area points in the target frame. The convex hull points in the convex hull point set are all the target drivable area points in the target frame.

[0054] The extreme convex hull point determination module is used to determine the maximum vertical convex hull point, the maximum horizontal convex hull point, and the minimum horizontal convex hull point in the set of convex hull points of the target box;

[0055] The right convex hull point set determination module is used to construct the right convex hull point set of the target box when the target box is located on the left side of the vehicle, based on the order of the convex hull points in the target box being located between the longitudinal maximum convex hull point and the lateral minimum convex hull point, and the right convex hull points whose distance from the lateral minimum convex hull point in the longitudinal direction satisfies the first condition.

[0056] The left convex hull point set determination module is used to construct the left convex hull point set of the target box when the target box is located on the right side of the vehicle, based on the left convex hull points in the target box being located in the order of the longitudinal maximum convex hull point and the transverse maximum convex hull point, and whose longitudinal distance from the transverse maximum convex hull point satisfies the second condition.

[0057] The first correction module is used to correct the right boundary line of the target frame by using the right convex hull points in the set of right convex hull points of the target frame when the target frame is located on the left side of the vehicle and turns to the left.

[0058] The second correction module is used to correct the left boundary line of the target frame by using the left convex hull points in the set of left convex hull points of the target frame when the target frame is located on the right side of the vehicle and turns to the right.

[0059] A third aspect of this application provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and running on the processor, wherein the computer program, when executed by the processor, implements the steps of a target post-processing method based on in-frame point clouds as described in the first aspect of this application.

[0060] The fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a target post-processing method based on in-frame point clouds as described in the first aspect of this application.

[0061] The target post-processing method based on in-frame point clouds provided in this application has the following advantages:

[0062] This application provides a target post-processing method based on in-frame point clouds. First, target recognition is performed on the point cloud data using a target recognition model to obtain multiple target bounding boxes. Then, the point cloud data is analyzed to determine the drivable area points of the targets within the target bounding boxes. Based on these drivable area points, the convex hull of the target bounding box and the corresponding set of convex hull points are determined, where the convex hull points in the set represent all drivable area points within the target bounding box. Next, the maximum vertical convex hull point, the maximum horizontal convex hull point, and the minimum horizontal convex hull point in the set of convex hull points are determined. When the target bounding box is located on the left side of the vehicle, the convex hull points are positioned sequentially between the maximum vertical convex hull point and the minimum horizontal convex hull point. The right convex hull point set of the target box is constructed by selecting the right convex hull points whose vertical distance from the minimum lateral convex hull point satisfies the first condition. When the target box is located on the right side of the vehicle, the left convex hull point set of the target box is constructed by selecting the left convex hull points whose vertical distance from the maximum lateral convex hull point satisfies the second condition. When the target box is located on the left side of the vehicle and turns left, the right boundary line of the target box is corrected using the right convex hull points in the right convex hull point set of the target box. When the target box is located on the right side of the vehicle and turns right, the left boundary line of the target box is corrected using the left convex hull points in the left convex hull point set of the target box. Therefore, this application determines the convex hull of the target box based on the drivable points within the target box. This can reduce the amount of computation and improve the efficiency of convex hull calculation, while supplementing the target box with more complete information. In addition, using the convex hull points in the convex hull of the target box to correct the boundary line of the target box can reduce the information bias of some target boxes in the model output or traditional clustering fitting, and provide more accurate target information for the backend. Attached Figure Description

[0063] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 A flowchart illustrating a target post-processing method based on in-frame point clouds is shown in one embodiment of this application;

[0065] Figure 2 This is a schematic diagram illustrating the target recognition result in a target post-processing method based on in-frame point clouds, as shown in one embodiment of this application.

[0066] Figure 3This is a schematic diagram of the convex hull of the target bounding box in a target post-processing method based on in-box point cloud, as shown in one embodiment of this application.

[0067] Figure 4 This is a schematic diagram illustrating the optimization of the target bounding box in a target post-processing method based on in-box point clouds, as shown in one embodiment of this application.

[0068] Figure 5 This is another flowchart illustrating a target post-processing method based on in-frame point clouds, as shown in one embodiment of this application;

[0069] Figure 6 This is a schematic diagram illustrating a target post-processing system based on in-frame point clouds, as shown in one embodiment of this application. Detailed Implementation

[0070] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0071] refer to Figure 1 , Figure 1 This is a schematic diagram illustrating a target post-processing method based on in-frame point clouds, as shown in one embodiment of this application. Figure 1 As shown, the method includes:

[0072] Step S1: Use the target recognition model to identify targets in the point cloud data and obtain multiple target boxes.

[0073] In this embodiment, point cloud data of the current driving scene is collected by the radar on the vehicle. The collected point cloud data of the current frame is then input into a target recognition model for target identification, obtaining the bounding box for each target in the current driving scene. The bounding box corresponds to the target, which refers to various objects in the current driving scene, including cars, bicycles, motorcycles, pedestrians, roadblocks, etc. The radar used to collect point cloud data includes, but is not limited to, lidar, millimeter-wave radar, continuous-wave radar, and pulse radar. If there are two vehicles in front of the radar, the collected point cloud data is input into the target recognition model for target identification, obtaining the bounding boxes corresponding to each of the two vehicles. It should be understood that this is only a simplified explanation for understanding the meaning of the bounding box; in reality, the bounding boxes obtained through the target recognition model may contain misidentifications, such as a non-existent target being identified as a target. The target recognition model is a radar model used to perform target identification on the point cloud data to obtain the bounding boxes corresponding to the targets. Figure 2As shown, Figure 2 An example is shown of the target bounding box recognition result obtained by performing target recognition on point cloud data through a target recognition model, where the green part is the target bounding box recognition result obtained by the target recognition model.

[0074] Step S2: By analyzing and processing the point cloud data, determine the target drivable area points located in the target box.

[0075] In this embodiment, the target box recognition results output by the target recognition model have a certain degree of deviation. Therefore, in order to avoid the impact of the target box corresponding to the vehicle target with the deviation on the vehicle's driving when the surrounding vehicles (i.e., vehicle targets) turn, this application will optimize the target boxes in the target box recognition results output by the target recognition model.

[0076] Specifically, the point cloud data of the current frame is analyzed and processed to determine the freespace points, i.e., drivable area points, within a certain height range, which, without considering traffic rules, provide unobstructed, flat road surface areas for vehicles to freely drive. There are various and well-established methods for determining freespace points in point cloud data, which will not be detailed here. Then, based on the positional relationship between all determined drivable area points and each target bounding box, the drivable area points within each target bounding box are determined, and these drivable area points within the target bounding box are designated as target drivable area points.

[0077] Step S3: Based on the target drivable area points in the target box, determine the convex hull of the target box and the set of convex hull points corresponding to the convex hull. The convex hull points in the set of convex hull points are all the target drivable area points in the target box.

[0078] In this embodiment, after determining the target drivable area points located within the target frame, for each target frame, using the target drivable area points within that target frame as boundaries, the convex hull of the target frame is calculated using a corresponding algorithm. This convex hull will encompass all target drivable area points within the target frame. The set of all target drivable area points within this convex hull is defined as the convex hull point set of the convex hull corresponding to the target frame. A convex hull point in this set corresponds to a target drivable area point within the convex hull. The preferred algorithm for calculating the convex hull of the target frame is the Graham scan algorithm. It should be understood that this is only a preferred algorithm, and other algorithms can also be used to determine the convex hull of the target frame; no specific limitation is made here.

[0079] Step S4: Determine the vertical maximum convex hull point, the horizontal maximum convex hull point, and the horizontal minimum convex hull point in the set of convex hull points of the target bounding box.

[0080] In this embodiment, for ease of description and understanding, the coordinate system of each target box is defined with the vertical direction of travel along its own target box as the x-axis and the horizontal direction as the y-axis. After obtaining the set of convex hull points corresponding to each target box in step S3, for each target box, the convex hull point with the largest vertical coordinate value (i.e., the vertical maximum convex hull point) and the convex hull point with the largest horizontal coordinate value (i.e., the horizontal maximum convex hull point) and the convex hull point with the smallest horizontal coordinate value (i.e., the horizontal minimum convex hull point) in its corresponding convex hull set are determined. Thus, for each target box, a vertical maximum convex hull point, a horizontal maximum convex hull point, and a horizontal minimum convex hull point can be determined. Figure 3 As shown, Figure 3 The diagram schematically illustrates the convex hull corresponding to a target bounding box, the set of convex hull points corresponding to the convex hull, and the maximum vertical convex hull point, the maximum horizontal convex hull point, and the minimum horizontal convex hull point in the defined set of convex hull points. The rectangle in the diagram represents the target bounding box, and the area enclosed by the dashed line is the convex hull corresponding to the target bounding box. The dots in the convex hull are the convex hull points. All the convex hull points in the convex hull constitute the set of convex hull points corresponding to the target bounding box. In the diagram, X_max is the maximum vertical convex hull point in the set of convex hull points, Y_max is the maximum horizontal convex hull point in the set of convex hull points, and Y_min is the minimum horizontal convex hull point in the set of convex hull points.

[0081] Step S5: When the target box is located on the left side of the vehicle, construct the set of right convex hull points of the target box by taking the convex hull points in the target box as being located in the order of the longitudinal maximum convex hull point and the transverse minimum convex hull point, and the right convex hull points whose longitudinal distance from the transverse minimum convex hull point satisfies the first condition.

[0082] In this embodiment, since the primary purpose of this application is to avoid the influence of the target bounding box of a vehicle with identification errors on the vehicle's driving when surrounding vehicles (i.e., vehicle targets) turn, and when the target bounding box is located on the left side of the vehicle, the main factor that may affect the vehicle's driving is the right boundary of the target bounding box. Therefore, when the target bounding box to be optimized is located on the left side of the vehicle, this application filters out convex hull points whose scanning order is between the vertical maximum convex hull point and the horizontal minimum convex hull point of the target bounding box, and whose vertical distance from the horizontal minimum convex hull point of the target bounding box satisfies a first condition. These filtered convex hull points are then determined as the right convex hull points of the target bounding box and stored in the right convex hull point set corresponding to the target bounding box. The method for determining the right convex hull point set corresponding to each target bounding box located on the left side of the vehicle is the same and will not be repeated here. The right convex hull point set of the target bounding box is used to optimize the target bounding box if optimization is required. The first condition is preferably that the distance between the convex hull point and the minimum convex hull point of the target box in the longitudinal direction is greater than half of the longitudinal dimension of the target box (that is, half the length of the target box in the longitudinal direction).

[0083] In this embodiment, the LiDAR records and stores point cloud data in the scanning order. In this implementation, when the target bounding box to be optimized is located on the left side of the vehicle, the boundary between the vertical maximum convex hull point and the horizontal minimum convex hull point refers to the convex hull points in the set of convex hull points of the target bounding box that are recorded and stored after the vertical maximum convex hull point and before the horizontal minimum convex hull point. That is, when subsequently determining convex hull points that satisfy the first condition, the determination is made for the convex hull points in the set of convex hull points of the target bounding box that are recorded and stored after the vertical maximum convex hull point and before the horizontal minimum convex hull point.

[0084] Step S6: When the target box is located on the right side of the vehicle, construct the set of left convex hull points of the target box by taking the convex hull points in the target box as being located in the order of the longitudinal maximum convex hull point and the transverse maximum convex hull point, and the left convex hull points whose longitudinal distance from the transverse maximum convex hull point satisfies the second condition.

[0085] In this embodiment, when the target box is located on the right side of the vehicle, the main area affected by the vehicle's movement is its left boundary. Therefore, this application, when the target box to be optimized is located on the right side of the vehicle, filters convex hull points whose scanning order is between the maximum longitudinal and lateral convex hull points of the target box, and whose longitudinal distance from the maximum lateral convex hull point of the target box satisfies the second condition. The filtered convex hull points are then determined as the left convex hull points of the target box and stored in the corresponding left convex hull point set. The method for determining the left convex hull point set for each target box located on the right side of the vehicle is the same and will not be repeated here. The left convex hull point set of the target box is used to optimize the target box if optimization is required. Preferably, the second condition is that the longitudinal distance between the convex hull point and the maximum lateral convex hull point of the target box is greater than half the longitudinal dimension of the target box (i.e., half the longitudinal length of the target box).

[0086] In this embodiment, the LiDAR records and stores point cloud data in the scanning order. In this implementation, when the target bounding box to be optimized is located on the right side of the vehicle, the boundary between the vertical and horizontal maximum convex hull points refers to the convex hull points in the set of convex hull points of the target bounding box that are recorded and stored after and before the vertical and horizontal maximum convex hull points. That is, when subsequently determining convex hull points that satisfy the second condition, the focus is on the convex hull points in the set of convex hull points of the target bounding box that are recorded and stored after and before the vertical and horizontal maximum convex hull points.

[0087] Step S7: When the target frame is located on the left side of the vehicle and the vehicle is turning left, the right boundary line of the target frame is corrected using the right convex hull points in the set of right convex hull points of the target frame.

[0088] In this embodiment, the positional relationship between the target frame and the vehicle is determined. When the target frame is located to the left of the vehicle, it is further determined whether the target frame is turning left. If the target frame located to the left of the vehicle is turning left, and its original right boundary line intrudes into the lane where the vehicle is located, the right boundary line of the target frame is corrected towards the inside of the target frame using the right convex hull points in the set of right convex hull points, to obtain a more accurate target frame. The corrected target frame may no longer intrude into the lane where the vehicle is located, so that the target frame with the original identification error will no longer affect the vehicle's driving, thereby enabling the vehicle to better perform control actions. In real-world scenarios, a vehicle target may not actually intrude into the lane where the vehicle is located (i.e., it will not affect the normal driving of the vehicle). However, due to a deviation in the target bounding box obtained by the target recognition model, the obtained target bounding box may intrude into the lane where the vehicle is located. As a result, a vehicle target that would not normally affect the normal driving of the vehicle is now considered to intrude into the lane and thus affect the normal driving of the vehicle. Consequently, the subsequent control actions of the vehicle will also be deviated (e.g., the vehicle could normally drive along the lane and overtake the vehicle in front, but due to the recognition result, the vehicle cannot drive and overtake normally). The method of this application is used to correct the target bounding box in the recognition result, so that the target bounding box is closer to the real target, thereby enabling the vehicle to perform control actions better (e.g., normally drive along the lane and overtake the vehicle target in front).

[0089] In this embodiment, one method for determining whether a target frame located on the left side of the vehicle is turning left involves determining the angle between the orientation of the target frame on the left side of the vehicle and the orientation of the vehicle. If this angle is greater than zero, the target frame on the left side of the vehicle is determined to be turning left. The vehicle's coordinate system is also based on the longitudinal direction of travel of the vehicle as the x-axis (e.g.,...). Figure 3 As shown), when the angle between the target frame on the left side of the vehicle and the vehicle's orientation is greater than zero, it is determined that the target frame on the left side of the vehicle is turning to the left.

[0090] Step S8: When the target frame is located on the right side of the vehicle and the vehicle is turning right, the left boundary line of the target frame is corrected using the left convex hull points in the set of left convex hull points of the target frame.

[0091] In this embodiment, the positional relationship between the target frame and the vehicle is determined. When the target frame is located to the right of the vehicle, it is further determined whether the target frame is turning right. If the target frame to the right of the vehicle is turning right, and its original left boundary line encroaches on the lane where the vehicle is located, the left boundary line of the target frame is corrected towards the inside of the target frame using the left convex hull points in the set of left convex hull points, to obtain a more accurate target frame. Figure 4 As shown, Figure 4 The green box corresponds to a target box, and the orange box is the target box obtained after correction.

[0092] This application provides a target post-processing method based on in-frame point clouds. First, target recognition is performed on the point cloud data using a target recognition model to obtain multiple target bounding boxes. Then, the point cloud data is analyzed to determine the drivable area points of the targets within the target bounding boxes. Based on these drivable area points, the convex hull of the target bounding box and the corresponding set of convex hull points are determined, where the convex hull points in the set represent all drivable area points within the target bounding box. Next, the maximum vertical convex hull point, the maximum horizontal convex hull point, and the minimum horizontal convex hull point in the set of convex hull points are determined. When the target bounding box is located on the left side of the vehicle, the convex hull points are positioned sequentially between the maximum vertical convex hull point and the minimum horizontal convex hull point. The right convex hull point set of the target box is constructed by selecting the right convex hull points whose vertical distance from the minimum lateral convex hull point satisfies the first condition. When the target box is located on the right side of the vehicle, the left convex hull point set of the target box is constructed by selecting the left convex hull points whose vertical distance from the maximum lateral convex hull point satisfies the second condition. When the target box is located on the left side of the vehicle and turns left, the right boundary line of the target box is corrected using the right convex hull points in the right convex hull point set of the target box. When the target box is located on the right side of the vehicle and turns right, the left boundary line of the target box is corrected using the left convex hull points in the left convex hull point set of the target box. Therefore, this application determines the convex hull of the target box based on the drivable points within the target box. This can reduce the amount of computation and improve the efficiency of convex hull calculation, while supplementing the target box with more complete information. In addition, using the convex hull points in the convex hull of the target box to correct the boundary line of the target box can reduce the information bias of some target boxes in the model output or traditional clustering fitting, and provide more accurate target information for the backend.

[0093] In conjunction with the above embodiments, in one implementation, this application also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, step S2 may include: analyzing and processing the point cloud data to determine drivable area points in the point cloud data; and determining target drivable area points located within the target frame based on the location of the target frame and the locations of the drivable area points.

[0094] In this embodiment, the point cloud data of the current frame is analyzed and processed to determine the drivable area points in the point cloud data. Then, based on the center point and size information of each target box, coordinate transformation is used to calculate the coordinates of the four corner points of each target box in the xy plane. Then, based on the coordinates of its four corner points, the maximum and minimum ranges in the horizontal and vertical directions of each target box in the xy plane are obtained. Then, based on the coordinates of the drivable area points, it is determined whether they fall within any target box. If the coordinates of the drivable area point fall within any target box, the drivable area point is identified as a target drivable area point located within a target box and stored for use in subsequent steps.

[0095] In conjunction with the above embodiments, in one implementation, this application also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, before step S2, the method further includes steps S01 to S02:

[0096] Step S01: Determine the virtual target box among the multiple target boxes based on the attribute information of the target box.

[0097] In this embodiment, as mentioned in step S1 above, there may be misidentifications in the target boxes obtained through the target recognition model, such as a non-existent target being identified as a target. Therefore, to avoid the impact of misidentified target boxes on the vehicle's driving and to reduce the resource consumption associated with optimizing misidentified target boxes, this application filters the target boxes obtained by processing point cloud data through the target recognition model to identify and remove misidentified virtual target boxes. Subsequent optimization only targets the remaining target boxes after filtering out the virtual target boxes. This application mainly determines whether each target box is a virtual target based on the point cloud reflectivity information, the confidence level of the target box, the positional relationship between target boxes, and the point cloud distribution within the target box.

[0098] Specifically, in scenarios with dense dust or high humidity near water trucks, if the proportion of low-reflectivity point clouds within the target bounding box reaches a certain threshold, and the average reflectivity and maximum reflectivity are both below a certain threshold, then the target bounding box is determined to be a false target. To address the issue of low accuracy in target recognition models due to incomplete point clouds, low-confidence target bounding boxes are identified as false targets based on their location, orientation, and point cloud distribution within the box.

[0099] Step S02: Filter the dummy target boxes among the multiple target boxes to obtain the remaining target boxes.

[0100] In this embodiment, the dummy target boxes in all target boxes determined by step S01 are filtered to obtain the remaining target boxes.

[0101] In this application, if the method further includes steps S01 to S02, step S2 may include: determining the target drivable area points located in the remaining target boxes after filtering by analyzing and processing the point cloud data.

[0102] In this embodiment, before step S2, steps S01 to S02 are included to filter the virtual target boxes in all the obtained target boxes. The implementation of step S2 is as follows: by analyzing and processing the point cloud data, the drivable area points in the point cloud data are determined, then it is determined which drivable area points are located in the remaining target boxes after filtering, and finally the drivable area points located in the remaining target boxes after filtering are determined as target drivable area points. The subsequent steps of determining the convex hull point set, determining the right convex hull point set, determining the left convex hull point set, and correcting and optimizing the boundary line are all based on the target drivable area points located in the remaining target boxes after filtering and the remaining target boxes after filtering.

[0103] In conjunction with the above embodiments, in one implementation, this application also provides a target post-processing method based on in-box point clouds. In this target post-processing method based on in-box point clouds, before step S3, the method further includes: determining the target boxes requiring orientation correction from all target boxes based on the attribute information of the target boxes.

[0104] In this embodiment, the target recognition model outputs bounding box recognition results with a certain degree of deviation. This deviation is only considered significant when a vehicle in the adjacent lane intrudes into the lane where the vehicle is located; such intrusion typically occurs only with large vehicles. Therefore, to improve processing efficiency and reduce computational resource consumption, this application only determines whether correction is needed and performs subsequent corrections for relatively large bounding boxes and / or those in a turning state.

[0105] Specifically, before step S3, if this embodiment addresses a scheme without virtual target box filtering, then target boxes requiring orientation correction are determined from all target boxes based on the category, size, and / or orientation in the target box's attribute information. If this embodiment addresses a scheme with virtual target box filtering, then target boxes requiring orientation correction are determined from all remaining target boxes after filtering based on the category, size, and / or orientation in the target box's attribute information. One optional implementation for determining target boxes requiring orientation correction is as follows: determining whether the target box is of the BIG_VEHICLE or HUGE_VEHICLE category, determining whether the target box's length exceeds a certain threshold (e.g., 6 meters), determining whether the target box's width exceeds a certain threshold (e.g., 2 meters), and determining whether the target box's orientation is within a range of ±25°. If any one of these conditions is met, the target box is determined to be a target box requiring orientation correction.

[0106] In this application, before step S3 of the target post-processing method based on in-frame point cloud provided in this application, if the target frame that needs to be oriented corrected is determined from all the target frames according to the attribute information of the target frame, step S3 may include: determining the convex hull of the target frame that needs to be oriented corrected and the set of convex hull points corresponding to the convex hull, based on the target drivable area points in the target frame that needs to be oriented corrected.

[0107] In this embodiment, when the target box that needs orientation correction is determined in this application, subsequent correction operations will all target boxes that need orientation correction. Therefore, in this case, in this implementation, this application will only distinguish the convex hull and the set of convex hull points of the determined target box that needs orientation correction. The method of determination is the same as the implementation of step S3 above, except that at this time, the convex hull of the target box that needs orientation correction and the set of convex hull points corresponding to the convex hull are determined based on the target drivable area point of the target box that needs orientation correction.

[0108] In conjunction with the above embodiments, in one implementation, this application also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, step S7 may include:

[0109] Step S71: When the target box is located on the left side of the vehicle and turns left, traverse the right convex hull points in the set of right convex hull points of the target box, and connect the currently traversed right convex hull point with the horizontal minimum convex hull point to form the right prediction boundary line.

[0110] In this embodiment, when the target box is located on the left side of the vehicle and the target box turns to the left, if there is an error in the model recognition causing its right boundary line to intrude into the lane where the vehicle is located, it will affect the driving of the vehicle (if it does not actually intrude, it is because of the recognition error that the recognition result is intrusion). In this case, this application will traverse the right convex hull points in the set of right convex hull points of the target box, and take out one convex hull point in the set of right convex hull points in turn, and then connect the convex hull point with the horizontal minimum convex hull point of the target box to form a new right predicted boundary line.

[0111] Step S72: If the angle deviation between the obtained right predicted boundary line and the right boundary line of the target box meets the set conditions, end the traversal and correct the right boundary line of the target box with the right predicted boundary line whose deviation meets the set conditions.

[0112] In this embodiment, after forming a new right predicted boundary line of the target box in step S71, the new right predicted boundary line is compared with the right boundary line of the target box (i.e., the right boundary point of the target box obtained by the model recognition) to determine whether the angle deviation between the two meets the set conditions. If the set conditions are met, the new right predicted boundary line replaces the right boundary line of the target box, thereby achieving the correction purpose, and the traversal ends. If the set conditions are not met, a new convex hull point is taken from the set of right convex hull points of the target box, and the newly taken convex hull point is connected to the horizontal minimum convex hull point of the target box to form a new right predicted boundary line. The angle deviation between the new right predicted boundary line and the right boundary line of the target box is then determined to meet the set conditions. If it still does not meet the conditions, the process continues until the angle deviation between the newly taken convex hull point and the right boundary line of the target box meets the set conditions. Preferably, the preset condition is that the angle deviation does not exceed 10°, which is to prevent over-correction of the target box and resulting in more serious errors.

[0113] In conjunction with the above embodiments, in one implementation, this application also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, step S8 may include:

[0114] Step S81: When the target box is located on the right side of the vehicle and turns right, traverse the left convex hull points in the set of left convex hull points of the target box, and connect the currently traversed left convex hull point with the largest lateral convex hull point to form the left prediction boundary line.

[0115] In this embodiment, when the target box is located on the right side of the vehicle and the target box turns to the right, if the model recognition has an error and causes its left boundary line to intrude into the lane where the vehicle is located, it will affect the driving of the vehicle (if it does not actually intrude, it is because the recognition error causes the recognition result to be intrusion). In this case, this application will traverse the left convex hull points in the set of left convex hull points of the target box, take out one convex hull point in the set of left convex hull points in turn, and then connect the convex hull point with the horizontal maximum convex hull point of the target box to form a new left predicted boundary line.

[0116] Step S82: If the angle deviation between the obtained left predicted boundary line and the left boundary line of the target box meets the set condition, end the traversal and correct the left boundary line of the target box with the left predicted boundary line whose deviation meets the set condition.

[0117] In this embodiment, after forming a new left predicted boundary line of the target box in step S81, the new predicted boundary line is compared with the left boundary line of the target box (i.e., the left boundary point of the target box obtained by the model recognition) to determine whether the angle deviation between the two meets the set conditions. If the set conditions are met, the new predicted boundary line replaces the left boundary line of the target box, thereby achieving the correction purpose, and the traversal ends. If the set conditions are not met, a new convex hull point is taken from the set of left convex hull points of the target box, and the new convex hull point is connected to the horizontal maximum convex hull point of the target box to form a new left predicted boundary line. The angle deviation between the new predicted boundary line and the left boundary line of the target box is then determined to meet the set conditions. If it still does not meet the conditions, the process continues until the angle deviation between the new convex hull point taken from the set of left convex hull points of the target box and the left boundary line of the target box meets the set conditions. Preferably, the preset condition is that the angle deviation does not exceed 10°, which is to prevent over-correction of the target box and more serious errors.

[0118] In conjunction with the above embodiments, in one implementation, this application also provides a target post-processing method based on in-frame point clouds. This target post-processing method based on in-frame point clouds further includes:

[0119] Step S101: Cluster the point cloud data using a clustering algorithm to obtain the target boxes for each cluster.

[0120] In this embodiment, in the implementation of processing point cloud data to obtain target boxes, the LiDAR model can only identify the target categories involved in the training model. Therefore, in actual target box recognition, traditional clustering algorithms are usually used to identify target boxes. However, the recognition method of traditional clustering algorithms may also involve the identification of some virtual targets. Based on this, the target post-processing algorithm based on in-box point cloud provided in this application is also used to filter virtual clustered target boxes obtained by traditional clustering algorithms based on the attribute information of the in-box point cloud.

[0121] Specifically, the point cloud data is first clustered using traditional clustering algorithms to obtain the target bounding boxes for each cluster. Figure 2 An example is shown showing the clustered target box recognition results obtained by clustering point cloud data using a traditional clustering algorithm, where the red part represents the clustered target box recognition results obtained by the traditional clustering algorithm.

[0122] Step S102: Determine the proportion of point clouds for different semantic categories of the clustering target box, determine the size of the clustering target box, determine the reflectance information of the point cloud within the clustering target box, and determine the first point cloud density of the point cloud located within a set range of the centroid of the clustering target box.

[0123] In this embodiment, after clustering the point cloud data using a traditional clustering algorithm to obtain various clustering target boxes, for each clustering target box, the proportion of point clouds of different semantic categories within each box is determined, as are the dimensions (length, width, and height) of each box, the reflectance information of the point clouds within each box, and the point cloud density within a set range of the centroid of the clustering target box. This point cloud density is the first point cloud density. The set range can be dynamically adjusted according to the specific size of the clustering target box, thereby improving the filtering accuracy of the virtual clustering target boxes. For example, if the clustering target box is relatively large, the corresponding set range is set to a larger value; if the clustering target box is relatively small, the corresponding set range is set to a smaller value.

[0124] The semantic categories include at least: invalid points (including noise and ground categories), background, vehicles, cyclists, and general obstacles. The point cloud reflectance information includes at least: the number of low-reflectance point clouds within the clustering bounding boxes, the maximum reflectance and total reflectance of point clouds within the clustering bounding boxes, the proportion of low-reflectance point clouds within the clustering bounding boxes, and the average reflectance of point clouds within the clustering bounding boxes.

[0125] Step S103: Divide the clustering target box into multiple segments in the vertical direction, and calculate the cloud density of the second point around the center point of the first segment in the direction of movement of the clustering target box.

[0126] In this embodiment, for each cluster target box obtained by clustering point cloud data through traditional clustering algorithms, each cluster target box is divided into multiple segments (preferably 4 segments) in its own vertical direction, and the point cloud density around the center point of the first segment (such as the center point of the first 1 / 4 segment) facing the movement direction of the cluster target box itself is calculated. This point cloud density is called the second density.

[0127] Step S104: Based on the point cloud proportion, the size, the reflectivity information, the first point cloud density, and the second point cloud density, determine the virtual clustering target boxes in each clustering target box.

[0128] In this embodiment, based on the point cloud proportion, size, reflectivity information, first point cloud density and second point cloud density of each cluster target box determined in steps S101 to S13, it is determined which of these cluster target boxes are virtual cluster target boxes.

[0129] Step S105: Filter the dummy clustering target boxes in each clustering target box to obtain the clustering result of the clustering algorithm.

[0130] In this embodiment, the clustering target boxes determined in step S104 are filtered, and the remaining clustering target boxes are the final clustering results obtained by the clustering algorithm.

[0131] In conjunction with the above embodiments, in one implementation, this application also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, determining the reflectance information of point clouds within a clustered target frame includes: determining the number of low-reflectance point clouds within the clustered target frame based on attribute information, and determining the maximum reflectance and total reflectance value of the point clouds within the clustered target frame; determining the proportion of low-reflectance point clouds within the clustered target frame based on the total number of point clouds and the number of low-reflectance point clouds; and determining the average reflectance of the point clouds within the clustered target frame based on the total number of point clouds and the total reflectance value.

[0132] In this embodiment, the collected point cloud data records the reflectance of the point clouds. Based on the reflectance of the point cloud data, point clouds with reflectance lower than a set value are identified as low-reflectance point clouds. Then, the number of low-reflectance point clouds within each cluster target box is determined, where the set value is preferably 10. Simultaneously, based on the reflectance of the point cloud data, the total reflectance value and the maximum reflectance of the point clouds within each cluster target box are determined. The number of low-reflectance point clouds within a single cluster target box is divided by the total number of point clouds within that single cluster target box to obtain the proportion of low-reflectance point clouds within that single cluster target box. Based on the same implementation method, each cluster target box will obtain its own corresponding proportion of low-reflectance point clouds. The total reflectance value within a single cluster target box is divided by the total number of point clouds within that single cluster target box to obtain the average reflectance of the point clouds within that single cluster target box.

[0133] In conjunction with the above embodiments, in one implementation, this application also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, step S104 may include: determining the clustered target frame as a dummy clustered target frame when the proportion of low-reflectivity point clouds corresponding to the clustered target frame is a set value; determining the clustered target frame as a dummy clustered target frame when the proportion of low-reflectivity point clouds corresponding to the clustered target frame exceeds a first reflectivity threshold and the corresponding average reflectivity does not exceed a second reflectivity threshold; determining the clustered target frame as a dummy clustered target frame when the proportion of low-reflectivity point clouds corresponding to the clustered target frame exceeds a third reflectivity threshold and the corresponding average reflectivity does not exceed a fourth reflectivity threshold and the corresponding maximum reflectivity does not exceed a fifth reflectivity threshold; and determining the dummy clustered target frame as a dummy clustered target frame when the proportion of point clouds representing noise semantic categories within the clustered target frame exceeds a category threshold. The system determines the size relationship between the stated size and a first size threshold, the density relationship between the first point cloud density and a first density threshold, and the quantity relationship between the total number of point clouds and a total number threshold. Based on the size relationship, the density relationship, and the quantity relationship, it determines whether the clustering target box is a dummy clustering target box. If the proportion of point clouds of noise semantic categories within the clustering target box does not exceed the category threshold, it determines whether the clustering target box is a dummy clustering target box based on the proportion of point clouds of background semantic categories within the clustering target box, the stated size, and the total number of point clouds. It also determines whether the clustering target box is a dummy clustering target box based on the proportion of point clouds of general obstacle semantic categories within the clustering target box, the location of the clustering target box, and the stated size.

[0134] In this embodiment, if the proportion of low-reflectivity point cloud corresponding to a clustering target box obtained by the traditional clustering algorithm is within a set value, the clustering target box is determined to be a dummy clustering target box and will be filtered out. The set value is preferably 1. If the proportion of low-reflectivity point cloud corresponding to a clustering target box obtained by the traditional clustering algorithm exceeds a first reflectivity threshold (preferably 0.9) and the corresponding average reflectivity does not exceed a second reflectivity threshold (preferably 3), the clustering target box is determined to be a dummy clustering target box and will be filtered out. If the proportion of low-reflectivity point cloud corresponding to a clustering target box obtained by the traditional clustering algorithm exceeds a third reflectivity threshold (preferably 1) and the corresponding average reflectivity does not exceed a fourth reflectivity threshold (preferably 0.5) and the corresponding maximum reflectivity does not exceed a fifth reflectivity threshold (preferably 3), the clustering target box is determined to be a dummy clustering target box and will be filtered out. The judgment criteria in this part mainly target the situation where noise such as water vapor and dust is mistakenly identified as target boxes. The reflectivity of noise such as water vapor and dust is generally low. For virtual targets such as water vapor near sprinkler trucks, virtual targets of dust at the rear of vehicles, virtual targets of water vapor in rainy weather, and virtual targets of vehicle exhaust, this application filters virtual targets of noise such as dust and water vapor based on the reflectivity information of the point cloud within the box.

[0135] In this embodiment, since semantic segmentation may misdetect general obstacles as noise or vice versa, this application determines virtual clustered target boxes by combining information such as the position and size of the target box, the number of point clouds within the box, the point cloud density around the center point and centroid of the target box, and the proportion of point clouds of each semantic category within the box. Specifically, if the proportion of point clouds of noise semantic categories within a clustered target box obtained by a traditional clustering algorithm exceeds a category threshold (preferably 0.6), the size relationship between the size of the clustered target box and a first size threshold, the density relationship between the first point cloud density of the clustered target box and a first density threshold, and the quantitative relationship between the total number of point clouds in the clustered target box and a total number threshold are determined. Then, based on the size relationship, density relationship, and quantitative relationship corresponding to the clustered target box, it is determined whether the clustered target box is a virtual clustered target box. The first size threshold includes a length threshold and a width threshold for the target box. These thresholds can be set according to the actual application scenario and are not specifically limited here. The first density threshold can also be set according to the actual application scenario and is not specifically limited here. The total number threshold can also be set according to the actual application scenario and is not specifically limited here. If the size of the cluster target box is not within the first size threshold range, and / or the density is greater than the first density threshold, and / or the total number of points in the cluster target box is greater than the total number threshold, then the cluster target box is determined to be a dummy cluster target box and will be filtered out. In other words, if at least one of these three relationships is satisfied, the cluster target box will be determined to be a dummy cluster target box. If the proportion of point cloud representing noise semantic category within a clustering target box obtained by traditional clustering algorithm does not exceed the category threshold (preferably 0.6), then the clustering target box is determined to be a dummy clustering target box based on the proportion of point cloud representing background semantic category (or noise and background semantic category) within the clustering target box, the size of the clustering target box, and the total number of point clouds within the clustering target box. Furthermore, the clustering target box is also determined to be a dummy clustering target box based on the proportion of point cloud representing general obstacle semantic category within the clustering target box, the location of the clustering target box, and the size of the clustering target box.Specifically, if the proportion of background semantic category (or background and noise semantic category) of the point cloud within the clustering target box exceeds a certain threshold (preferably 0.9), and the size of the clustering target box exceeds a certain set threshold (i.e., the size of the clustering target box is large), and the number of point clouds within the target clustering box is lower than a certain threshold (or the height of the clustering target box above the ground exceeds a certain threshold), then the clustering target box is determined to be a virtual clustering target box and will be filtered out. If the proportion of general obstacle semantic category of the point cloud within the clustering target box exceeds a certain threshold (preferably 0.9), and the clustering target box is not located in this lane and the size of the clustering target box exceeds a certain threshold (i.e., the size is large), then the clustering target box is determined to be a virtual clustering target box and will be filtered out.

[0136] In conjunction with the above embodiments, in one implementation, this application also provides a target post-processing method based on in-frame point clouds. This target post-processing method based on in-frame point clouds further includes: determining whether there are small-sized single-cluster target boxes within a preset range around the vehicle category cluster target box; and if small-sized single-cluster target boxes exist within the preset range around the vehicle category cluster target box, determining that the vehicle category cluster target box is a dummy cluster target box.

[0137] In this embodiment, for each vehicle category clustering target box obtained by the traditional clustering algorithm, it is determined whether there are small-sized single clustering target boxes within a preset range around the single vehicle category clustering target box. If there are small-sized single clustering target boxes within the preset range of the single vehicle category clustering target box, then the single vehicle category clustering target box is determined to be a dummy clustering target box and will be filtered out. The preset range can be set according to the actual application scenario and is not specifically limited here. Similarly, the small size is set according to the actual application scenario and is not specifically limited here.

[0138] In this embodiment, as Figure 5As shown, one optional implementation of the target post-processing method based on in-box point clouds provided in this application is as follows: Point cloud data is processed using both a target recognition model and a traditional clustering algorithm. The target recognition model identifies multiple target boxes, and the traditional clustering algorithm also clusters multiple target boxes. For the multiple target boxes identified by the target recognition model, dummy target boxes are filtered based on the reflectivity information of the point clouds within the target boxes and the distribution information of the point clouds within the target boxes. For the multiple target boxes clustered by the traditional clustering algorithm, dummy target boxes are filtered based on the quantity and proportion of point cloud data obtained from each semantic category within the target boxes, as well as the reflectivity information of the point clouds within the target boxes and the point cloud density information around the centroid and center point of the target boxes. For multiple target boxes identified by the target recognition model, after filtering, the boundary lines of the remaining target boxes are corrected. First, the point cloud data is analyzed and processed to determine the drivable area points of the target boxes located in the remaining target boxes after filtering. Then, based on the drivable area points of the target boxes, the convex hull of the individual target boxes is determined. Finally, when the boundary lines of the individual target boxes need to be corrected, the boundary lines of the individual target boxes are corrected based on the convex hull.

[0139] Based on the same inventive concept, one embodiment of this application provides a target post-processing system based on in-frame point clouds, such as... Figure 6 As shown, the system 600 includes:

[0140] The target recognition module 601 is used to perform target recognition on point cloud data through a target recognition model to obtain multiple target boxes;

[0141] Analysis and processing module 602 is used to determine the target drivable area points located in the target box by analyzing and processing the point cloud data;

[0142] The convex hull point set determination module 603 is used to determine the convex hull of the target frame and the convex hull point set corresponding to the convex hull based on the target drivable area points in the target frame, wherein the convex hull points in the convex hull point set are all the target drivable area points in the target frame;

[0143] The extreme convex hull point determination module 604 is used to determine the maximum vertical convex hull point, the maximum horizontal convex hull point, and the minimum horizontal convex hull point in the set of convex hull points of the target box;

[0144] The right convex hull point set determination module 605 is used to construct the right convex hull point set of the target frame when the target frame is located on the left side of the vehicle, based on the order of the convex hull points in the target frame being located between the longitudinal maximum convex hull point and the lateral minimum convex hull point, and the right convex hull points whose longitudinal distance from the lateral minimum convex hull point satisfies the first condition.

[0145] The left convex hull point set determination module 606 is used to construct the left convex hull point set of the target box when the target box is located on the right side of the vehicle, based on the left convex hull points in the target box being located in the order of the longitudinal maximum convex hull point and the transverse maximum convex hull point, and whose distance from the transverse maximum convex hull point in the longitudinal direction satisfies the second condition.

[0146] The first correction module 607 is used to correct the right boundary line of the target frame by using the right convex hull points in the set of right convex hull points of the target frame when the target frame is located on the left side of the vehicle and turns to the left.

[0147] The second correction module 608 is used to correct the left boundary line of the target frame by using the left convex hull points in the set of left convex hull points of the target frame when the target frame is located on the right side of the vehicle and turns to the right.

[0148] Optionally, the analysis and processing module 602 includes:

[0149] The drivable area point determination module is used to determine drivable area points in the point cloud data by analyzing and processing the point cloud data;

[0150] The target drivable area point determination module is used to determine the target drivable area point located within the target frame based on the location of the target frame and the location of the drivable area point.

[0151] Optionally, the system 600 further includes:

[0152] The virtual target bounding box determination module is used to determine the virtual target bounding box among the plurality of target bounding boxes based on the attribute information of the target bounding boxes;

[0153] The filtering module is used to filter the dummy target boxes in the multiple target boxes to obtain the remaining target boxes;

[0154] The analysis and processing module 602 is used to determine the target drivable area points located in the remaining target boxes after filtering by analyzing and processing the point cloud data.

[0155] Optionally, the system 600 further includes:

[0156] The target bounding box determination module is used to determine the target bounding boxes that need orientation correction from all the target bounding boxes based on the attribute information of the target bounding boxes;

[0157] The convex hull point set determination module 603 is used to determine the convex hull of the target frame that needs to be oriented and the convex hull point set corresponding to the convex hull, based on the target drivable area points in the target frame that need to be oriented.

[0158] Optionally, the first correction module 607 includes:

[0159] The traversal module is used to traverse the right convex hull points in the set of right convex hull points of the target box when the target box is located on the left side of the vehicle and the vehicle is turning left, and connect the currently traversed right convex hull point with the horizontal minimum convex hull point to form the right prediction boundary line.

[0160] The first correction submodule is used to end the traversal and correct the right boundary line of the target box with the right predicted boundary line whose deviation meets the set conditions when the angle deviation between the obtained right predicted boundary line and the right boundary line of the target box meets the set conditions.

[0161] The second correction module 608 includes:

[0162] The traversal module is used to traverse the left convex hull points in the set of left convex hull points of the target box when the target box is located on the right side of the vehicle and turns to the right, and connect the currently traversed left convex hull point with the largest lateral convex hull point to form the left prediction boundary line.

[0163] The second correction submodule is used to end the traversal and correct the left boundary line of the target box with the left predicted boundary line whose deviation meets the set conditions when the angle deviation between the obtained left predicted boundary line and the left boundary line of the target box meets the set conditions.

[0164] Optionally, the system 600 further includes:

[0165] The clustering module is used to cluster point cloud data using clustering algorithms to obtain various cluster target boxes;

[0166] The first attribute determination module is used to determine the proportion of point clouds of different semantic categories of the clustering target box, as well as the size of the clustering target box, the reflectance information of the point cloud within the clustering target box, and the first point cloud density of the point cloud located within a set range of the centroid of the clustering target box.

[0167] The second attribute determination module is used to divide the clustering target box into multiple segments in the vertical direction and calculate the cloud density of the second point around the center point of the first segment in the direction of movement of the clustering target box.

[0168] The virtual clustering target box determination module is used to determine the virtual clustering target boxes in each clustering target box based on the point cloud proportion, the size, the reflectivity information, the first point cloud density, and the second point cloud density.

[0169] The filtering module is used to filter the dummy clustering target boxes in each clustering target box to obtain the clustering result of the clustering algorithm.

[0170] Optionally, a reflectance information determination module is used to determine the reflectance information of the point cloud within the clustering target bounding box; the reflectance information determination module includes:

[0171] The first reflectivity information determination module is used to determine the number of low reflectivity point clouds within the clustering target box, as well as the maximum reflectivity and total reflectivity of the point clouds within the clustering target box, based on the attribute information of the point clouds within the clustering target box.

[0172] The second reflectivity information determination module is used to determine the proportion of low reflectivity point clouds in the clustering target box based on the total number of point clouds in the clustering target box and the number of low reflectivity point clouds.

[0173] The third reflectance information determination module is used to determine the average reflectance of the point cloud within the clustering target box based on the total number of point clouds and the total reflectance value within the clustering target box.

[0174] Optionally, the virtual clustering target bounding box determination module includes:

[0175] The first virtual clustering target box determination module is used to determine the clustering target box as a virtual clustering target box when the proportion of low reflectivity point cloud corresponding to the clustering target box is a set value.

[0176] The second virtual clustering target box determination module is used to determine the clustering target box as a virtual clustering target box when the proportion of low reflectivity point cloud corresponding to the clustering target box exceeds the first reflectivity threshold and the corresponding average reflectivity does not exceed the second reflectivity threshold.

[0177] The third virtual clustering target box determination module is used to determine the clustering target box as a virtual clustering target box when the proportion of low reflectivity point cloud corresponding to the clustering target box exceeds the third reflectivity threshold and the corresponding average reflectivity does not exceed the fourth reflectivity threshold and the corresponding maximum reflectivity does not exceed the fifth reflectivity threshold.

[0178] The fourth virtual clustering target box determination module is used to determine the size relationship between the size and the first size threshold, the density relationship between the first point cloud density and the first density threshold, and the quantity relationship between the total number of point clouds and the total number threshold when the proportion of point clouds of noisy semantic categories in the clustering target box exceeds the category threshold.

[0179] The fifth virtual clustering target box determination module is used to determine whether the clustering target box is a virtual clustering target box based on the size relationship, the density relationship and the quantity relationship;

[0180] The sixth virtual clustering target box determination module is used to determine whether the clustering target box is a virtual clustering target box based on the proportion of point clouds of background semantic categories within the clustering target box, the size, and the total number of point clouds, when the proportion of point clouds of noise semantic categories within the clustering target box does not exceed the category threshold; and to determine whether the clustering target box is a virtual clustering target box based on the proportion of point clouds of general obstacle semantic categories within the clustering target box, the location of the clustering target box, and the size.

[0181] Optionally, the system 600 further includes:

[0182] The single clustering target box determination module is used to determine whether there are small single clustering target boxes within a preset range around the vehicle category clustering target box;

[0183] The seventh virtual clustering target box determination module is used to determine the vehicle category clustering target box as a virtual clustering target box when there are small-sized single clustering target boxes within a preset range around the vehicle category clustering target box.

[0184] Based on the same inventive concept, one embodiment of this application provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and running on the processor. When the computer program is executed by the processor, it implements the steps of the target post-processing method based on in-frame point cloud as described in the first aspect of this application.

[0185] Based on the same inventive concept, one embodiment of this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a target post-processing method based on in-frame point clouds as described in the first aspect of this application.

[0186] As the system implementation is basically similar to the method implementation, it is described in a relatively simple way. For relevant details, please refer to the description of the method implementation.

[0187] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily necessary for the embodiments of this application.

[0188] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0189] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0190] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0191] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0192] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0193] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0194] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0195] The above provides a detailed description of the target post-processing method, system, and product based on in-frame point clouds provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A target post-processing method based on in-frame point clouds, characterized in that, The method includes: The point cloud data is used to identify targets using a target recognition model, resulting in multiple bounding boxes. By analyzing and processing the point cloud data, the target drivable area points located within the target frame are determined; Based on the target drivable area points in the target box, determine the convex hull of the target box and the set of convex hull points corresponding to the convex hull, wherein the convex hull points in the set of convex hull points are all the target drivable area points in the target box; Determine the vertical maximum convex hull point, the horizontal maximum convex hull point, and the horizontal minimum convex hull point in the set of convex hull points of the target bounding box; When the target box is located on the left side of the vehicle, the right convex hull point set of the target box is constructed by taking the convex hull points in the target box as being located in the order of the longitudinal maximum convex hull point and the transverse minimum convex hull point, and the right convex hull point whose distance from the transverse minimum convex hull point in the longitudinal direction satisfies the first condition. When the target box is located on the right side of the vehicle, the left convex hull point set of the target box is constructed by taking the convex hull points in the target box as being located in the order of the longitudinal maximum convex hull point and the transverse maximum convex hull point, and the left convex hull point whose distance from the transverse maximum convex hull point in the longitudinal direction satisfies the second condition. When the target frame is located on the left side of the vehicle and the vehicle is turning left, the right boundary line of the target frame is corrected using the right convex hull points in the set of right convex hull points of the target frame. When the target frame is located on the right side of the vehicle and the vehicle is turning right, the left boundary line of the target frame is corrected using the left convex hull points in the set of left convex hull points of the target frame. By analyzing and processing the point cloud data, target drivable area points located within the target frame are determined, including: By analyzing and processing the point cloud data, drivable areas in the point cloud data are determined. Based on the location of the target frame and the location of the drivable area point, determine the target drivable area point located within the target frame.

2. The target post-processing method based on in-frame point cloud according to claim 1, characterized in that, Before determining the target drivable area points located within the target frame by analyzing and processing the point cloud data, the method further includes: Based on the attribute information of the target boxes, determine the virtual target boxes among the multiple target boxes; Filter the dummy target boxes among the multiple target boxes to obtain the remaining target boxes; The step of analyzing and processing the point cloud data to determine the target drivable area points located within the target frame includes: By analyzing and processing the point cloud data, the drivable area points of the target located in the remaining target boxes after filtering are determined.

3. A target post-processing method based on in-frame point clouds according to claim 1 or 2, characterized in that, Before determining the convex hull of the target bounding box and the set of convex hull points corresponding to the convex hull based on the target drivable area points in the target bounding box, the method further includes: Based on the attribute information of the target bounding boxes, determine the target bounding boxes that need orientation correction from all the target bounding boxes; The step of determining the convex hull of the target frame and the set of convex hull points corresponding to the convex hull based on the target drivable area points in the target frame includes: determining the convex hull of the target frame that needs to be oriented corrected and the set of convex hull points corresponding to the convex hull based on the target drivable area points in the target frame that need to be oriented corrected.

4. The target post-processing method based on in-frame point cloud according to claim 1, characterized in that, When the target frame is located on the left side of the vehicle and the vehicle is turning left, the right boundary line of the target frame is corrected using the right convex hull points in the set of right convex hull points of the target frame, including: When the target box is located on the left side of the vehicle and the vehicle is turning left, the right convex hull points in the set of right convex hull points of the target box are traversed, and the right convex hull point currently traversed is connected with the horizontal minimum convex hull point to form the right prediction boundary line. If the angle deviation between the obtained right predicted boundary line and the right boundary line of the target box meets the set conditions, the traversal ends and the right predicted boundary line of the target box is corrected with the right predicted boundary line whose deviation meets the set conditions. The step of correcting the left boundary line of the target frame by using the left convex hull points from the set of left convex hull points of the target frame when the target frame is located on the right side of the vehicle and the vehicle is turning right includes: When the target box is located on the right side of the vehicle and turns right, the left convex hull points in the set of left convex hull points of the target box are traversed, and the currently traversed left convex hull point is connected with the largest lateral convex hull point to form the left prediction boundary line. If the angle deviation between the obtained left predicted boundary line and the left boundary line of the target box meets the set conditions, the traversal ends and the left boundary line of the target box is corrected using the left predicted boundary line whose deviation meets the set conditions.

5. The target post-processing method based on in-frame point cloud according to claim 1, characterized in that, The method further includes: The point cloud data is clustered using a clustering algorithm to obtain the target bounding boxes for each cluster. Determine the proportion of point clouds for different semantic categories of the clustering target box, determine the size of the clustering target box, determine the reflectance information of the point cloud within the clustering target box, and determine the first point cloud density of the point cloud within a set range of the centroid of the clustering target box. The clustering target box is divided into multiple segments vertically, and the cloud density of the second point around the center point of the first segment in the direction of movement of the clustering target box is calculated. Based on the point cloud proportion, the size, the reflectivity information, the first point cloud density, and the second point cloud density, the virtual clustering target boxes in each clustering target box are determined; The dummy clustering boxes in each clustering target box are filtered to obtain the clustering result of the clustering algorithm.

6. The target post-processing method based on in-frame point cloud according to claim 5, characterized in that, Determine the reflectance information of the point cloud within the cluster target bounding box, including: Based on the attribute information of the point cloud within the clustering target box, determine the number of low reflectivity point clouds within the clustering target box, as well as the maximum reflectivity and total reflectivity of the point clouds within the clustering target box. The proportion of low-reflectivity point clouds within the clustering target box is determined based on the total number of point clouds within the clustering target box and the number of low-reflectivity point clouds. The average reflectance of the point cloud within the clustering target box is determined based on the total number of point clouds and the total reflectance value within the clustering target box.

7. A target post-processing method based on in-frame point clouds according to claim 6, characterized in that, Based on the point cloud proportion, the size, the reflectivity information, the first point cloud density, and the second point cloud density, the virtual clustering target boxes in each clustering target box are determined, including: When the proportion of low reflectivity point cloud corresponding to the clustered target box is a set value, the clustered target box is determined to be a virtual clustered target box; If the proportion of low reflectivity point cloud corresponding to the clustered target box exceeds the first reflectivity threshold and the corresponding average reflectivity does not exceed the second reflectivity threshold, the clustered target box is determined to be a virtual clustered target box. If the proportion of low reflectivity point cloud corresponding to the clustered target box exceeds the third reflectivity threshold, and the corresponding average reflectivity does not exceed the fourth reflectivity threshold and the corresponding maximum reflectivity does not exceed the fifth reflectivity threshold, the clustered target box is determined to be a virtual clustered target box. When the proportion of point clouds of noisy semantic categories within the clustering target box exceeds the category threshold, the size relationship between the size and the first size threshold is determined, the density relationship between the first point cloud density and the first density threshold is determined, and the quantity relationship between the total number of point clouds and the total number threshold is determined. Based on the size relationship, the density relationship, and the quantity relationship, determine whether the clustering target box is a dummy clustering target box; If the proportion of point clouds representing noise semantic categories within the clustering target box does not exceed the category threshold, the clustering target box is determined to be a virtual clustering target box based on the proportion of point clouds representing background semantic categories within the clustering target box, the size, and the total number of point clouds. Furthermore, the clustering target box is also determined to be a virtual clustering target box based on the proportion of point clouds representing general obstacle semantic categories within the clustering target box, the location of the clustering target box, and the size.

8. The target post-processing method based on in-frame point cloud according to claim 5, characterized in that, The method further includes: Determine whether there are small single clustering boxes within a preset range around the vehicle category clustering target box; If there are small-sized single clustering target boxes within a preset range around the vehicle category clustering target box, the vehicle category clustering target box is determined to be a dummy clustering target box.

9. A target post-processing system based on in-frame point clouds, characterized in that, The system includes: The target recognition module is used to identify targets in point cloud data using a target recognition model to obtain multiple target bounding boxes. The analysis and processing module is used to determine the target drivable area points located in the target box by analyzing and processing the point cloud data; The convex hull point set determination module is used to determine the convex hull of the target frame and the convex hull point set corresponding to the convex hull based on the target drivable area points in the target frame. The convex hull points in the convex hull point set are all the target drivable area points in the target frame. The extreme convex hull point determination module is used to determine the maximum vertical convex hull point, the maximum horizontal convex hull point, and the minimum horizontal convex hull point in the set of convex hull points of the target box. The right convex hull point set determination module is used to construct the right convex hull point set of the target box when the target box is located on the left side of the vehicle, based on the order of the convex hull points in the target box being located between the longitudinal maximum convex hull point and the lateral minimum convex hull point, and the right convex hull points whose distance from the lateral minimum convex hull point in the longitudinal direction satisfies the first condition. The left convex hull point set determination module is used to construct the left convex hull point set of the target box when the target box is located on the right side of the vehicle, based on the left convex hull points in the target box being located in the order of the longitudinal maximum convex hull point and the transverse maximum convex hull point, and whose longitudinal distance from the transverse maximum convex hull point satisfies the second condition. The first correction module is used to correct the right boundary line of the target frame by using the right convex hull points in the set of right convex hull points of the target frame when the target frame is located on the left side of the vehicle and turns to the left. The second correction module is used to correct the left boundary line of the target frame by using the left convex hull points in the set of left convex hull points of the target frame when the target frame is located on the right side of the vehicle and turns to the right. The analysis and processing module includes: The drivable area point determination module is used to determine drivable area points in the point cloud data by analyzing and processing the point cloud data; The target drivable area point determination module is used to determine the target drivable area point located within the target frame based on the location of the target frame and the location of the drivable area point.

10. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and running on the processor, wherein the computer program, when executed by the processor, implements the steps of a target post-processing method based on in-frame point clouds as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a target post-processing method based on in-frame point clouds as described in any one of claims 1 to 8.

Images